The move comes after Facebook also increased anti-harassment measures, following a survey which said young victims of online abuse feel social media firms aren't doing enough to fight the problem.
Instagram chief Adam Mosseri said artificial intelligence is being used to detect signs of bullying and then automatically flag content for review by staff from the image-oriented social network.
"This change will help us identify and remove significantly more bullying," Mosseri said in a blog post.
"It's a crucial next step since many people who experience or observe bullying don't report it."
Rolling the technology out across the entire social network is expected to take a few weeks.
In another move, a filter that enables Instagram users to detect and hide bullying remarks from Instagram feeds or profiles was extended to comments on live videos, according to Mosseri.
In the spirit of inspiring people to be nice to one another, Instagram also added a camera effect which can be used to adorn images with hearts or the word "kindness" in a variety of languages.
"Online bullying is complex, and we know we have more work to do to further limit bullying and spread kindness on Instagram," Mosseri said.
Last week Facebook said its users will be able to hide or delete groups of comments, including responses, instead of having to remove them one at a time.
The network also began letting people report bullying or harassment on behalf of friends reluctant to complain to the social network themselves.
A majority of US teenagers say they have been victims of online harassment or bullying, most commonly name-calling, a Pew Research Center survey found in September.
It said that 59 percent of US teens reported online abuse, and 63 percent said it was a major problem for people their age.
The survey found that most young harassment victims feel that teachers, social media firms and politicians are doing a poor or fair job in addressing the problem of cyber bullying -- an issue which has drawn the attention of US First Lady Melania Trump and Britain's Prince William.