His Popular Party (PP) was the big winner of the election, the second in six months, which played out amid the turbulence from Britain's shock vote to leave the European Union.
The party, which portrayed itself as the guardian of stability, won 137 seats in the 350-strong lower house of parliament - 14 more than in December and more than what pre-election polls predicted.
All other parties lost votes or seats, in some cases both.
The Socialists held on to second place like in December, though with just 85 seats, their worst score in modern history.
The far-left Unidos Podemos coalition came third with 71 seats, reversing pre-election expectations that it could leapfrog over the Socialists and replace them as the main left-wing force.
Market-friendly party Ciudadanos, meanwhile, finished fourth as in December with 32 seats, down from 40, as many of the party's voters moved back to the PP.
"We won and we demand the right to govern," Rajoy said as he looked down from a tall podium on a crowd of supporters waving blue flags and shouting "yes we can!" - stealing Podemos' key catchphrase.
"It's been hard, it's been difficult, it's been complicated, but we put up a fight for Spain," he added.
The PP celebrated their win. Photo: AFP
'Listen to voters'
But while the PP boosted its seats, it still faces that same challenges to form a government as after the December polls when Podemos and Ciudadanos uprooted the country's two-party dominance.
Without a majority, it will need to seek the outright or tacit support of other parties to get a coalition or minority government through.
The combined total number of seats if PP teams up with Ciudadanos, their natural ally, is not enough to form a majority centre-right government so they may need to court the Socialists too or smaller regional parties.
But other parties have been reluctant to back the PP, which has been tainted by a string of corruption scandals and anger over high unemployment and the steep public spending cuts it has put in place.
The general election in December resulted in a parliament so splintered that parties failed to agree on a coalition, and this is what prompted Sunday's repeat vote.
After failing to form a government after the last polls, political leaders will be under more pressure this time to form some sort of government.
Top-selling daily newspaper El Pais, which is traditionally close to the Socialists, urged the party in an editorial on Monday to allow the PP to govern.
"To facilitate the formation of a government, the Socialists should listen to the mandate of voters that it remain in the opposition and allow the party that has the necessary votes to govern to do so," the newspaper wrote.
After the last elections the Socialists tried to form a coalition government with Ciudadanos but were not able to pass a required vote of confidence in parliament.
Return to growth
Throughout the campaign - and with greater insistence after the Brexit vote - the PP had hammered away at the need for stability in reference to the rise of Unidos Podemos, which like Greece's ruling Syriza party rejects EU-backed austerity.
Rajoy has argued that since the PP came to power in 2011, it has brought Spain back to growth and overseen a drop in unemployment - though at 21 percent it is still the second highest rate in the EU after Greece.
But his rivals retort that inequalities have risen, the jobs created are mainly unstable, and they point to the repeated corruption scandals to have hit the PP.
Despite this, though, many voters still went for the PP led by Rajoy, who has portrayed himself as a safe pair of hands.
"The PP is the only party that guarantees us stability and economic growth," said Andres Alvarez, a 23-year-old advertising executive, as he celebrated outside the party's Madrid headquarters.
By Daniel Silva / AFP