The number of graft accusations to have hit the PP, other parties, unions and companies over the years have made corruption a main concern of Spaniards after sky-high unemployment, opinion polls show.
In the latest development, the former PP regional president of Madrid was arrested last month as part of a probe into embezzlement of public money.
Earlier in April, the PP president of the southeastern region of Murcia was also forced to resign as he too is being investigated for alleged corruption.
"The most corrupt is (Mariano) Rajoy," banners read at the protest on Madrid's central Puerta del Sol square, referring to the conservative prime minister.
"It's the main problem in the country," said Jose Martin, 59, a former employee at Bankia, a bank that was nationalised during the crisis and whose former executives have been found guilty of misusing funds and are under probe for fraud.
"They are emptying state coffers," added his wife Carmen Juarez, a 56-year-old civil servant, pointing to "clear spending cuts in education, in health. It's obscene."
Dolores Sanz, a 69-year-old former hospital worker, said many in Spain were struggling to make ends meet after a severe economic crisis.
"It's all linked: if there is corruption, the money goes and there is no money for important things like people's salaries," she said.
Spain last year scored its worst ranking in Transparency International's annual corruption perceptions index.
The PP has been particularly hard hit by graft allegations, but its rival Socialists have also been affected.
Such is public anger over the issue that many voters have flocked to two relatively new parties -- the far-left Podemos and the centre-right Ciudadanos.
As a result, although the PP still won a general election last year, it failed to retain the absolute majority it gained in 2011, and Rajoy now heads a minority government.