"We have suffered similar tragedies in our country and we know that there is no country in the world, no matter how strong or how many resources it has, that can tackle on its own these events," he told a rally in the Mediterranean city of Alicante.
"It is a response that has to come from Europe and words alone are not enough anymore. We have to act and we Europeans risk damaging our credibility if we are not able to prevent these tragic situations which are happening every day."
The European Union announced an emergency meeting of foreign and interior ministers, but gave no date for the gathering, after the latest migrant disaster off the coast of Libya.
The Local Italy: 700 feared dead in Mediterranean shipwreck
Spain's foreign ministry called illegal migration "one of the biggest challenges of the 21st century" which it recalled "is not new".
In a statement it said "renewed efforts" to tackle the problem should focus on curbing illegal immigration and human trafficking, regulating legal immigration and boosting economic development in the nations where migrants come from as well as in those where they chose to live.
Spain is struggling with an influx of migrants, although on a smaller scale than Italy and Greece, and Madrid regularly calls on the EU to provide more resources for Europe's border control agency Frontex.
Rajoy vowed that Spain would continue to push the EU to better handle migration and its sometimes tragic consequences.
Despite the danger, thousands of migrants fleeing war and hardship try to cross the 15-kilometre (nine-mile) Strait of Gibraltar from Morocco to Spain on makeshift boats and inflatable dinghies each year.
Spain also faces growing migrant pressure at its two enclaves in north Africa, Ceuta and Melilla, which have the European Union's only land borders with Africa.
Around 16,000 migrants tried to storm the Melilla border fence last year and nearly 5,000 made it over, according to Spanish authorities. In 2013 roughly 3,000 managed to cross.
The Local Sweden: Relatives in Sweden fear for refugees lost at sea