The advertisements for "The Man in the High Castle" had completely wrapped the seats, walls and ceilings of one train on the shuttle line that connects Times Square and Grand Central Terminal in midtown Manhattan.
Adam Lisberg, a spokesman for the Metropolitan Transportation Authority, said the train was taken out of service after the evening rush hour so the ads could be removed.
An Amazon representative said earlier in the evening that the company had not requested the ads be pulled, contradicting a transit official who had said the company itself had asked for the removal.
Lisberg would not comment on internal discussions between the MTA and its advertisers but said Cuomo had called the head of the MTA on Tuesday and asked him to ensure the ads were taken down.
Earlier on Tuesday, New York City Mayor Bill de Blasio had also called on Amazon to pull the ads, saying they were "irresponsible and offensive to World War Two and Holocaust survivors, their families, and countless other New Yorkers."
The show, based on a Philip K. Dick novel, depicts an alternate reality in which Nazi Germany and Japan have divided control over the United States after winning World War Two.
The advertisements include a version of the American flag with a German eagle and iron cross in place of the stars, as well as a stylized imperial Japanese flag.
In a statement, Amazon did not directly address the controversy, saying the show is part of its lineup of "high-quality, provocative programming that spurs conversation."
The shuttle train ads had been scheduled to run until December 6. Amazon's 260 subway station posters for the show have not been removed.
The MTA had said the advertisements did not violate the agency's content-neutral guidelines, which ban political ads.
All 10 episodes of the show were released on November 20 on the Amazon Prime streaming service.
Frank Spotnitz, the show's creator, told Entertainment Weekly he agreed the advertisements could be seen as offensive.
"It's very difficult with a show with subject matter like this to market it tastefully," the magazine quoted him as saying. "If they had asked me, I would have strongly advised them not to do it."
© Thomson Reuters 2015