Using the software means voice commands like "change the channel" are transmitted to a third-party service that converts the speech to text and then sends the command to the television.
"Please be aware that if your spoken words include personal or other sensitive information, that information will be among the data captured and transmitted to the third party," the policy said.
It's not the first time Smart TVs have been called out for potential "Big Brother" style invasions of privacy.
In 2013, another South Korean electronics conglomerate LG acknowledged that its smart TVs were collecting data about owners' viewing habits without their permission.
The company said it had fixed the problem with an update.
In a statement Tuesday, Samsung stressed that it took privacy issues "very seriously" and employed numerous safeguards to prevent the unauthorised the use of consumer data.
The statement noted that the voice recognition feature on its Smart TVs was an option and could simply be switched off.
"Should consumers enable the voice recognition capability, the voice data consists of TV commands, or search sentences, only," it added.