Officers arrested Spanish national Javier Martín-Artajo Rueda on an international detention request issued by the United States for alleged fraud and tax crimes, police said in a statement.
After locating Martín-Artajo, Spanish police spoke to him and managed to get him to appear at a police station where he was arrested on Tuesday morning, police said, without giving further details.
"The detainee is suspected of being responsible for manipulating and inflating the value of positions on his firm's credit portfolio," the police statement said.
The suspect was taken to appear before an investigating judge in Madrid's National Court.
Martín-Artajo is accused of being a senior figure in the 2012 "London Whale" trading scandal involving $6.2 billion in trading losses at US banking giant JP Morgan.
US federal prosecutors filed criminal charges on August 14 against Martín-Artajo and Frenchman Julien Grout, alleging the two men kept false records on the trades, committed wire fraud and submitted false US securities filings.
The charges were the first criminal case to stem from last year's mammoth trading loss. The Securities and Exchange Commission also filed parallel civil charges against the two men alleging securities fraud.
A third ex-JPMorgan official involved in the trades, French national Bruno Iksil — originally identified as the "London Whale" responsible for the trades — was cleared of criminal responsibility after cooperating with prosecutors.
Martín-Artajo, 49, is a Spaniard who usually resides in London, while Grout, 35, resides in his native France.
According to US prosecutors, Martín-Artajo was the most senior of the three employees.
The US complaint documents how the London team allegedly falsified financial records after Martin-Artajo was pressured from higher-ups about losses in early 2012.
Martín-Artajo directed underlings to discount the losses and count positions in ways that departed from company practice of pricing assets at mid-market levels, according to the complaint.
It cites phone calls and emails from the Iksil, who is identified only as a "co-conspirator not named as a defendant."
Iksil is depicted in the complaints as a generally unwilling participant in the cover-up, urging Martín-Artajo to revise pricing practices to more accurately reflect losses and taking steps to document and highlight the growing gap between the division's reported and actual losses.
At one point, Martín-Artajo allegedly fought with Iksil after he reported in an email a single-day trading loss of $40 million. Martin-Artajo erupted at the document, saying putting the information in an email "just creates more tension", according to a recorded phone call.
As a result of the false records, JPMorgan in July 2012 restated losses from its corporate private equity division, which included the London trading operation, by $459 million.
The defendants each face a maximum sentence of 65 years in prison when all the criminal charges are combined, and in addition a fine of $5 million, or twice the gross gain or loss from the offences, whichever is greater.
Civil penalties in the SEC case include paying back ill-gotten gains and a fine.
The debacle led to senior bank resignations, slashed pay for JPMorgan chief executive Jamie Dimon and sparked a plethora of government probes.
The London whale case is one of several regulatory headaches still facing JPMorgan. The bank also faces probes on its sale of mortgage-backed securities, among other issues