To the din of whistles and drums, the demonstrators snaked through the central streets of the Spanish capital under a blazing sun, waving red and white union flags and carrying placards reading "Austerity Destroys and Kills" and "Spending Cuts Are Robbery".
"We hope the government realizes that you can't reverse the situation with cuts, cuts and more cuts. It is time to change course and do more to foster growth," said Diana Rodriguez, 47, a former bank teller who has been unemployed for nearly two years.
Demonstrations were held in 28 other cities across Spain, including Barcelona, the country's second-largest city, and the Atlantic port of Vigo in the northwest.
They were organized by Spain's three largest unions, the Workers' Commissions, the General Union of Workers and the Workers' Trade Unionist Federation which urged Rajoy to press for a change to the European Union's focus on austerity during a summit of leaders of the bloc at the end of the month.
"The only thing which the policy of cuts which is being imposed by the more rancid right wing of the European Union is achieving is to rob us of rights which took us many years to gain," the secretary general of the branch of the General Union of Workers in the northern region of Galicia, Rosa Arcos, said before the start of a march attended by several hundred people in Santiago de Compostela.
Rajoy's conservative government has introduced a range of spending cuts and tax hikes, as well as pension reform, to shore up strained public finances since it took office in December 2011 following a landslide general election win.
It has vowed to find 150 billion euros ($195 billion) in savings between 2012 and 2014 as part of the painful austerity programme that has sparked mass demonstrations.
"We demand a change to these policies which are leading us to a future of more poverty and economic depression," said the leader of the Catalan branch of the Workers' Commissions, Joan Carles Gallego, at the march in Barcelona.
The Spanish economy, the eurozone's fourth-largest, shrank by 0.5 percent during the first three months of 2013, its seventh consecutive quarterly decline, while the unemployment rate soared to a record 27.16 percent, the highest level since the re-birth of Spanish democracy after the death of General Francisco Franco in 1975.
For those between the ages of 16 and 24, joblessness has soared to 57.22 percent.
"The government has to change course. It is obvious that the path they are taking us on will just dig us deeper into this mess," said Carlos Lago, a 32-year-old civil servant who carried a sign reading "Without Bread, There Is No Peace".