The probe may deal another blow to the U.S. case after a New Zealand court ruled in June that search warrants used in the raid on Dotcom's home earlier this year, requested by the U.S. Federal Bureau of Investigation, were illegal.
Key has asked the government's Intelligence and Security division to investigate "circumstances of unlawful interception of communications of certain individuals by the Government Communications Security Bureau", his office said in a statement on Monday.
Key's spokesman would not comment on whether the "certain individuals" referred to Dotcom, his three colleagues also arrested and facing U.S. charges, or all of them.
"The Bureau had acquired communications in some instances without statutory authority," Key's statement said.New Zealand authorities arrested Dotcom and his colleagues at his rented country estate near Auckland in January, confiscating computers and hard drives, works of art, and cars.
The FBI accuses the flamboyant Dotcom, a 38-year-old German national also known as Kim Schmitz, of leading a group that netted $175 million since 2005 by copying and distributing music, films and other copyrighted content without authorisation.
"I welcome the inquiry by (Key) into unlawful acts by the GCSB," Dotcom said on his Twitter account.
Dotcom maintains that the Megaupload site was no more than an online storage facility, and has accused Hollywood of lobbying the U.S. government to vilify him.
The raid and evidence seizure has already been ruled illegal and a court has ruled that Dotcom should be allowed to see the evidence on which the extradition hearing will be based.
U.S. authorities have appealed against that ruling, and a decision is pending.
Copyright Thomson Reuters 2012