His ruling Popular Party, which won an absolute majority four years ago but is running neck-and-neck with the Socialists in polls ahead of a December 20th general election, fears a similar scenario could emerge in Spain.
"This would be the first time in Portugal's democratic history that the party which won the elections does not govern," Rajoy said as he arrived at an annual congress of the European People's Party (EPP) which groups conservative parties from across the European Union.
Portugal's general election on October 4th ended in a deadlock.
The governing centre-right alliance, which has overseen unpopular austerity measures, won the most seats but lost its absolute majority in parliament.
The country's Socialist opposition leader, Antonio Costa, said on Tuesday that he was in a position to form a government backed by the radical Left Bloc, linked to Greece's anti-austerity party Syriza, and the Communist party.
In Spain, polls show the Socialists could form a majority in parliament if they join forces with new anti-austerity party Podemos, a close ally of Syriza, or the new centre-right party Ciudadanos.
"A coalition between the Socialists and the Podemos there (in Portugal) would be very negative," Rajoy said.
Like Portuguese Prime Minister Pedro Passos Coelho, Rajoy has imposed unpopular economic reforms and spending cuts.
The Portuguese economy "went through an extremely difficult period and now it is recovering," said Rajoy.
"There has already been some growth, jobs are being created and reversing this course would be a shame" he added.