It was Catalan separatist lawmakers -- along with Basque nationalists -- who swung the balance in favour of a no-confidence vote brought Friday against Rajoy due to corruption woes afflicting his party.
As opposition leader, 46-year-old economist Sanchez was deeply critical of Catalonia's secession bid last October.
He backed Rajoy's imposition of direct rule on the region after separatist leaders declared Catalan independence. But he softened his tone as Catalan separatist lawmakers in the national parliament backed his no-confidence motion. He has said he wants to "build bridges" with the new regional separatist government.
The fall of the conservatives in Madrid, along with the start of a new executive in Catalonia, could allow "the situation to improve," says Joan Botella, politics professor at the Autonomous University of Barcelona.
Sanchez's speech in parliament was not particularly ambitious where the wealthy divided region is concerned, he says. But he could profit from divisions within the separatist camp.
The more moderate want to ditch the unilateral route to independence after several separatist leaders were jailed over their role in the secession bid. Meanwhile radicals like deposed president Carles Puigdemont, who is currently in Germany in self-exile, are in favour of keeping the conflict alive.
Sanchez's swearing-in on Saturday coincided with that of new Catalan president Quim Torra's executive, which automatically triggers the end of direct rule.
"The pro-independence movement is declining, there are internal divisions and with the new government Puigdemont is going to be less important," says Botella. "So maybe Sanchez will be lucky."
Things have not gone off to a good start.
Torra, a hardline separatist, criticised for writing tweets or articles deemed offensive to Spaniards, was described by Sanchez as a "racist" and "supremacist."
Sanchez even dubbed Torra the Jean-Marie Le Pen of Spanish politics, referring to the controversial former far-right leader in France.
"The worst insults came from him and people from his (Socialist) party," said a regional presidency source, who wished to remain anonymous. "We don't have a good opinion at all, but Rajoy was clearly worse. At least he's of fresh disposition, and can defuse the situation and allow some detente."
Since Rajoy came to power in 2011, ties between Madrid and Barcelona have consistently worsened. The tensions escalated still further in the autumn with police violence during a banned independence referendum on October 1, a subsequent failed proclamation of a republic and the imposition of direct rule.
Even if Sanchez supported Rajoy in his response to the Catalan crisis, generally speaking the Socialists have been more attuned to Catalan claims than the conservative Popular Party (PP).
Since 2013, they have called for a federal reform of Spain to give semi-autonomous regions more power and Sanchez has even defined Catalonia as a "nation."
"Even if the PSOE (Socialist party) has tried hard to look like the Popular Party... in Catalonia we know there are differences," Joan Tarda, a lawmaker with the separatist ERC party, said this week in the national parliament.
Adding to this, the main supporter of the Socialists in the no-confidence vote was far-left Podemos, which defends the right to hold an independence referendum in Catalonia.
"There could be a change, maybe not substantial but more like an offer for dialogue, the will to reopen channels to find a negotiated solution to the conflict," says Berta Barbet, editor of political analysis website Politikon.
But any concessions to Catalan separatists could be rejected by members of Sanchez's own party, the PP and especially Ciudadanos, the centre-right party that is on the rise in opinion polls for its aggressive anti-independence stance.
"There will be very strong opposition," acknowledges Barbet. "But if there is a moment when the PSOE could make a different offer to de-escalate the conflict, it's now, because part of Spanish society has realised how important it is to resolve the Catalan issue."