"Our voters voted to get rid of Rajoy," Socialist party head Pedro Sanchez told reporters after a 30-minute meeting with the conservative leader, which he said he "could have perfectly done without."
Spain has been without a government for eight months following two inconclusive elections. Rajoy's conservative Popular Party (PP) has won both ballots but failed to reach an absolute majority.
Rajoy on Sunday clinched a deal with the upstart Ciudadanos, a liberal party, to support him in this week's confidence vote but still needs the backing of the Socialist party to prevent a third round of elections.
To win Ciudadanos support, Rajoy agreed a series of anti-corruption measures and a reform of electoral law, which will favour small parties like the liberals, as well as a string of proposals such as a wage hike for low-income families and a boost in education and health budgets.
But Sanchez told reporters that the PP was intent on "pursuing the same policies which have brought on unemployment and inequality" since it came to power in 2011.
With Ciudadanos' support and the vote of a deputy from the Canary Islands, Rajoy has secured 170 votes in the 350 seat lower house, six short of the majority needed in Wednesday's vote to form a government.
In a second confidence vote to take place on Friday, Rajoy only needs a simple majority but for that he would need the Socialists to abstain.
If there is no breakthrough two months after Wednesday's parliamentary ballot, vote-weary Spaniards will be asked to return to the polls on December 25th.
Sanchez said that in choosing the date of the confidence vote to ensure the Christmas Day election, Rajoy was "attempting to blackmail the political parties and the whole of Spain."
"The Socialist party does not give in to blackmail," he said.
Rajoy for his part said he would keep on trying to convince the Socialist party to change its mind.
"The lack of government is starting to undermine the country's reputation," he said. "And there is serious risk that we will be seen as jokers."
Twitter even introduced an emoji especially for the investiture debate and vote.