The company said it has approached police after the media outlet, which remains unnamed, asked the e-commerce giant to pay $300,000 for a research report against it to be dropped.
The research is allegedly based on negative reports from a domestic weekly and comments by individuals online, Alibaba said in a statement.
Police in Hangzhou city, where the company is based, registered a case on Monday, state-run Xinhua news agency reported.
Alibaba welcomes supervision from the mass media, but it will not tolerate acts that go beyond the legal boundaries, said Jack Ma, founder and chairman of the company.
The statement did not disclose the media institution's name, but insiders speculated it referred to a Huangzhou-based business magazine.