Brazilian midfielder Fernando deflected Gareth Bale's cross into his own net after 20 minutes at the Santiago Bernabeu for the only goal of the tie after the first leg ended goalless.
Fernandinho came closest to taking City to their first ever Champions League final when his shot clipped the outside of the post a minute before half-time.
Real beat Atletico 4-1 after extra time in Lisbon two years ago to win the competition for a 10th time and will be hoping for a repeat when they face Diego Simeone's men in Milan on May 28th.
"It is another final. It has been very difficult to get there," Madrid defender Sergio Ramos -- whose 93rd-minute header sent the previous final against Atletico into extra time - told BeIN Sports Spain.
"We have shown we know how to win a final and we are going to try and do it
City's performance over the two legs left much to be desired, but goalkeeper Joe Hart claimed the difference between the sides had been the luck Real enjoyed in the only goal of the game.
"We could easily be in the final but they managed the game well," he told BT Sport.
"They've beaten us with a very lucky goal but it is what it is."
City manager Manuel Pellegrini's time at the club will come to an end without European glory as he will be replaced at the end of the season by Pep Guardiola.
And the Chilean lamented his side's lack of creativity to cause Madrid problems over the two legs.
"We worked very well the whole game and that is why Real Madrid couldn't create chances, but after that you need creativity and players who can make the difference in the last third and we lacked that."
Real Madrid boss Zinedine Zidane now has the chance to repeat his Champions League success as a player with Real in 2002 in his first season in charge as a coach.
"In a semi-final you always have to suffer, but we've achieved our place in the final," said Zidane.
Real were given a huge boost before kick-off as Cristiano Ronaldo made his return after a three-game absence due to a thigh injury.
By contrast, City were dealt a massive injury blow inside 10 minutes when captain Vincent Kompany was forced off once more with a muscle problem in an injury-disrupted campaign.
Ronaldo was looking to equal his own record of 17 Champions League goals in a season, but sent his first chance well over the bar with a header from Dani Carvajal's cross.
However, Madrid got the luck they needed to swing the tie in their favour with their next attack as Bale was played in on the right of the City box and his intended cross spooned up off Fernando and into the far corner to leave Hart helpless.
The difference in European pedigree between the clubs couldn't have been starker as Madrid were playing in their 27th semi-final, whilst City were in the last four for the first time.
That inexperience looked to be taking hold as City enjoyed decent spells of first-half possession without troubling the Real defence.
However, they came within inches of finding the crucial away goal just before the break when Kevin De Bruyne picked out Fernandinho and the Brazilian midfielder's low driven shot clipped the outside of the post.
The goalmouth action was all in City's box early in the second period as Hart denied Luka Modric and Ronaldo twice before Bale's looping header came back off the bar.
Yet, out of nothing City nearly snatched the goal they needed a minute from time when top scorer Sergio Aguero's first serious effort of the tie dipped onto the roof of the net.
Despite a frantic finale, though, Madrid continued their record of not conceding a goal at home in the Champions League this season to progress to their 14th European Cup final.
By Kieran Canning / AFP