According to a report in The Information late on Monday, Giannandrea's role is being split into two.
Jeff Dean, who leads the company's deep learning AI research team Google Brain, would run a separate AI wing in addition to his current position.
"Ben Gomes, who joined Google in 2000 and was running search engineering, will take over the search organisation," CNBC reported.
Giannandrea will continue to work at Google. A Machine Learning (ML) expert, Giannandrea joined Google in 2010 after the company acquired his startup Metaweb Technologies.
He took the role of Senior Vice President of Engineering at Google from Amit Singhal in early 2016, signalling Google's emphasis on weaving ML and AI into Search.
Google CEO Sundar Pichai has been "betting big" on advances in AI and machine learning.
Earlier this year, Pichai said that AI is one of the most profound things that humanity is working on right now and compared it to basic utilities in terms of its importance.
"AI is 'one of the most important things that humanity is working on. It's more profound than, I don't know, electricity or fire," The Verge quoted Pichai as saying.
Pichai also said that AI could be used to help solve climate change issues or to cure cancer.