Spain's National Court, which deals with corruption and financial crime cases, said he had been found guilty of embezzlement when he headed up Caja Madrid and Bankia, at a time when both groups were having difficulties.
The case caused an outrage in Spain, where it was uncovered at the height of a severe economic crisis that left many people struggling financially - made all the worse because Bankia later had to be nationalised and injected with more than 22 billion euros in public funds.
Rato, who is also a former Spanish economy minister, remains at liberty pending a possible appeal.
He was on trial with 64 other former executives and board members at both banks accused of misusing 12 million euros between 2003 and 2012 - sometimes splashing out at the height of Spain's economic crisis.
They were accused of having paid for personal expenses with credit cards put at their disposal by both Caja Madrid and Bankia, without ever justifying them or declaring them to tax authorities.
These expenses included petrol for their cars, supermarket shopping, pricey holidays, luxury bags or parties in nightclubs.
According to the indictment, Rato maintained the "corrupt system" established by his predecessor Miguel Blesa when he took the reins of Caja Madrid in 2010.
He then replicated the system when he took charge of Bankia, a group born in 2011 out of the merger of Caja Madrid with six other savings banks, prosecutors said.
Blesa was sentenced to six years in jail.
Rato, 67, had always denied any wrongdoing and said the credit cards were for discretionary spending as part of executives' pay deal.
He told court last October that everything "was completely legal".
Rato will not necessarily go directly to jail if he appeals the ruling, just like the Spanish king's brother-in-law Inaki Urdangarin who has been left free without posting bail following his sentence of six years and three months for syphoning off millions of euros.
Urdangarin's temporary reprieve, also announced on Thursday, made waves in Spain where people have long criticised what is perceived as the impunity of the elite.
IMF chiefs in the dock
Rato was economy minister and deputy prime minister in the conservative government of Jose Maria Aznar from 1996 to 2004, before going on to head up the International Monetary Fund until 2007.
His subsequent career as a banker in Spain was short-lived - from 2010 to 2012 - but apart from the case of the undeclared credit cards, it also led to another banking scandal considered the country's biggest ever.
Thousands of small-scale investors lost their money after they were persuaded to convert their savings to shares ahead of the flotation of Bankia in 2011, with Rato at the reins.
Less than a year later, he resigned as it became known that Bankia was in dire straits.
The state injected billions of euros but faced with the scale of Bankia's losses and trouble in other banks, it asked the European Union for a bailout for the entire banking sector and eventually received 41 billion euros.
Rato and others were put under investigation, accused of misleading small investors in the listing of Bankia, which has since paid out 1.2 billion euros in compensation.
He is the third former IMF chief to get into trouble with the law.
His successor Dominique Strauss-Kahn was tried in 2015 on pimping charges in a lurid sex scandal, and was acquitted.
And Christine Lagarde, who took over from Strauss-Kahn and is the current IMF chief, was found guilty of negligence over a massive state payout to a tycoon when she was French finance minister.
By Marianne Barriaux