Microsoft has filed a patent application regarding an artificial intelligence capable of altering the spoken content in real time during video broadcasts. This concept also includes the ability to adjust lip movements to match the newly generated speech.
The system, referred to by the service Patentlyze, would operate with a slight delay and has several potential applications. It could be used to translate speeches, eliminate verbal tics, and censor inappropriate terms, in addition to allowing the replacement of information presented during a presentation.
In the documentation, Microsoft characterizes this technology as a 'near real-time enhancement' mechanism intended for event presentations. The operation involves simultaneous analysis of both the audio and the images from the broadcast to identify segments requiring modification.
Upon detecting an expression that fits pre-configured rules, the AI would create an alternative version of the audio and adjust the mouth area to preserve synchronization. Although similar functionalities already exist in AI dubbing tools on platforms like Instagram, Microsoft's system promises to operate with a latency ranging between 1 and 10 seconds.
The patent claims cover everything from minor corrections, such as automatic removal of stutters or incorrect pronunciations, to filtering profanity and offensive language according to viewer preferences, as well as instant speech translation.
Additionally, the document describes a functionality that places the technology in the realm of deepfakes: the possibility of swapping information spoken by the speaker. According to Microsoft, this substitution would be based on data obtained through internet searches during the transmission itself.
This capability raises concerns about authenticity, as the system could produce multiple versions of a statement that was never made by the original presenter. Microsoft submitted the application on April 29 of last year, and the document was released by the United States Patent Office last week. However, according to Patentlyze, the claims were rejected in the initial preliminary review, although this does not end the process, as the company has the option to amend the application.
There is no indication that this technology is being developed for a specific product. As with other intellectual property registrations, the mere existence of the patent does not guarantee that the described system will be commercially launched.
