The 63-year-old veteran politician is quitting as head of a party he has led since 2004 after he was ousted as prime minister in June in a parliamentary no-confidence vote tabled by his Socialist rival Pedro Sanchez, angered at the PP's repeated corruption woes.
"I come to bid farewell to the party, and also to say thank you," Rajoy said, visibly moved, as he started speaking to hundreds of PP members gathered in a Madrid hotel.
Among them were some of the 3,082 delegates who on Saturday will elect his successor, choosing between Rajoy's former right-hand woman Soraya Saenz de Santamaria and Pablo Casado, a 37-year-old lawmaker.
"I've lived through the history of our organisation with intensity," he said, recalling when he started as an activist four decades ago, putting up election posters in Galicia, his home region in Spain's northwest.
The former prime minister also praised his government's track record.
"Spain is much better off than when we arrived" at the end of 2011, replacing the Socialist government at a time of devastating economic crisis, he said.
The economy has grown over three percent for three years in a row, and Spain's jobless rate has fallen dramatically, even if at 16.7 per cent it remains the second highest in the eurozone after Greece.
Before he took to the podium, a video was broadcast in the room with many prominent leaders bidding him farewell, including Germany's Angela Merkel, Britain's Theresa May, India's Narendra Modi and even Portugal's Socialist Prime Minister Antonio Costa.
And towards the end of his speech, Rajoy said cryptically, "I'm stepping aside, but I'm not going away."