Rajoy gave Carles Puigdemont, who has led the semi-autonomous region of Catalonia for three months, a copy of revered Spanish author Miguel de Cervantes' masterpiece "Don Quixote" before they began their closed-door talks at his official residence in Madrid.
Puigdemont was elected president of Catalonia's government in January with the task of fast-tracking the Spanish region towards independence from Madrid.
"I'm going with good will, but with reasonable scepticism too," Puigdemont told reporters before heading the meeting.
There is little likelihood that anything will come of the meeting in Madrid as Puigdemont's position is radically opposed to that of Rajoy, who wants Catalonia to remain Spanish and refuses to allow a Scotland-style referendum on the issue.
Spain is also mired in political uncertainty as parties are unable to agree on a coalition government four months after inconclusive general elections, and Rajoy is currently only acting prime minister.
But the talks are still significant as the last meeting at this level dates back to July 2014 when then Catalonian president Artur Mas had asked for an independence referendum and more investment in the region.
Far from bringing them closer, though, the meeting at the time had triggered higher tensions between Madrid and Barcelona, the regional capital.
In November of that year, Catalonia's government went ahead and organized a symbolic poll in which 80 percent of the 2.3 million who cast their ballot in the 7.5-million-strong region voted for independence.
Then last September, a pro-independence faction won regional parliamentary elections, giving them an absolute majority.
Separatist lawmakers subsequently passed a motion in parliament calling for complete independence, although this was ruled illegal by Spain's Constitutional Court.
One of Spain's 17 semi-autonomous regions with its own language and customs, Catalonia already enjoys a large degree of freedom in education, health and policing.
But fed up after years of demands for greater autonomy on the taxation front - complaining it pays more to Madrid than it gets back - the region veered towards separatism.
Puigdemont is expected to ask Rajoy for more investment in Catalonia, greater leeway in the management of his heavily indebted region's deficit, and a referendum.
"The 16 times we have asked him will not stop us asking a 17th time," he said on Wednesday in front of the regional parliament, before travelling to Madrid.