The proposed reform, fiercely opposed by dockers who threatened to stage a nationwide strike that would hurt exports if it went ahead, was shot down with 175 votes against, 142 in favour and 33 abstentions.
It is the first major defeat for Rajoy's Popular Party, which has since October headed a minority government that has just 137 seats in the 350-seat lower house of parliament.
The cabinet gave the green light to the decree more than two years after the EU Court of Justice ruled Spain must reform the sector, or face sanctions.
Currently, domestic or foreign companies can only hire dockers to load and unload ships from specific, already-established Spanish groups known as Sagebs that select, train and provide personnel, and no other firm.
The decree would allow companies to contract workers wherever they want.
"It is a European directive that we must follow," Rajoy said ahead of the vote in parliament on the decree.
"We have delayed its approval, we have given all the time in the world to reach an agreement."
Spain's 6,150 dockers fear that opening up the sector to competition will put their jobs and salaries at risk.
Over 60 percent of Spain's exports pass through the country's 46 main ports.
Spain, with its nearly 6,000 kilometres of coastline, is also a key transit point for exports from Europe to the rest of the world.
The International Dockworkers Council (IDC), an umbrella group of 91 unions in 41 countries, has also threated to stage coordinated strikes around the world in a show of support for Spanish dockers.