Little is known outside the company about Zhang, a soft-spoken accountant who goes under the epithet "Xiaoyaozi" at work - the name of a character from a Chinese martial arts novel literally meaning the "free and unfettered one."
But he will have to step out from the shadows after Ma said he would leave his role as chairman in September 2019, handing Zhang the reins of his $420 billion tech giant.
"We don't know much about (Zhang) even though he's been CEO for a while, because he's by nature a very low-key person, low profile. He's happy to not be in the limelight," said Duncan Clark, managing director at Beijing tech advisory BDA and author of "Alibaba: The House that Jack Built".
The handover of power, the first such transition for any of China's major tech firms, will see Zhang, 46, elevated to the top of a sprawling empire that spans e-commerce, payments, fast food delivery and brick-and-mortar retail.
Zhang, who leads Alibaba's investor calls with polish stemming from his years at global auditors Arthur Andersen and PricewaterhouseCoopers, has been CEO since 2015.
His fingerprints are on some of the company's most successful moves: he was a key architect of Alibaba's "Singles Day," the November 11 event that has become the world's largest online shopping festival.
The company has added around $200 billion to its market value since he became CEO, and has now posted nine straight quarters of revenue growth above 50 percent while investing in food delivery and offline retail, and pushing into Southeast Asia.
Crucially, Zhang oversaw the growth of retail platform Tmall, which has grown to be one of the company's most significant revenue drivers, squaring off against major Chinese e-commerce rival JD.com Inc.
But he also inherits some challenges. Despite soaring sales, Alibaba's profit margins have been squeezed by rising competition and heavy investment to fend off rivals. The company's shares are down more than 10 percent this year after peaking in June.
A global trade war is also hampering Alibaba's ambitions for international expansion, including into the United States.
Zhang, who joined Alibaba in 2007, sees Alibaba as an integrated ecosystem that lures consumers with movies, live-streamed sports, fast food deliveries and news.
"Alibaba today is more like all-in-one," he told investors and analysts on a call discussing the company's quarterly earnings in August.
Zhang, who is from Shanghai, said in an interview with local business publication Yicai last year that he returned to the city once a week from Alibaba's Hangzhou headquarters to recharge and spend time with his wife.
He added that he had not bought a house in Hangzhou because he didn't have time. To relax, he enjoys watching soccer and basketball.
Zhang - like Ma - has worked hard to promote the company since its 2014 listing, making appearances at the World Economic Forum in Davos and other global events.
But he doesn't have the same star power that founders in China's large tech companies hold, including Jack Ma, Baidu Inc's Robin Li and Tencent Holdings Ltd's Pony Ma.
Those three frequently headline tech conferences and even government events together. Ma even did a Michael Jackson-themed dance routine at an Alibaba anniversary event last year, complete with a costume.
Zhang and his team, although working under the radar, "are ready" to take over, Ma said in a letter to staff and shareholders.
Ma described Zhang as having "superb talent, business acumen and determined leadership."
"His analytical mind is unparalleled, he holds dear our mission and vision, he embraces responsibility with passion, and he has the guts to innovate and test creative business models," Ma wrote.
© Thomson Reuters 2018