**Synthesia **has implemented content moderation measures in response to the controversial use of its deepfake avatars. Recognizing the technology’s potential for misuse, such as spreading disinformation or engaging in scams, the company has instituted policies to restrict the creation of content that goes against its terms of service. These terms prohibit the avatars from speaking on subjects like politics, religion, race, or sexuality. The moderation process involves both human and algorithmic systems, aiming to prevent the deepfakes from reading malicious scripts. Additionally, Synthesia has recently mandated that news content can only be created through enterprise accounts, where the identity of the users must be verified. The number of content moderators at the company has increased significantly, accounting for around 10 percent of its 230-person workforce, as part of a broader shift in the industry to move content moderation to the point of creation rather than distribution.
https://www.wired.co.uk/article/synthesia-ai-deepfakes-it-control-riparbelli