#kddicom
**Japanese telecommunications operator KDDI**, in collaboration with the XR Development team and the Google Partner Innovation team, has enhanced the realism of virtual avatars for VTubers using AI rendering and MediaPipe’s Face Landmarker solution. The solution detects facial landmarks and outputs blendshape scores to render a 3D face model that mimics the user. The technology successfully detects facial features and extracts 52 blendshapes in real-time using MediaPipe’s Python package.
The 52 blendshape float values are stored in Firebase’s Realtime Database and are continuously updated in real-time to capture dynamic changes in facial expressions. The extracted blendshape data is then transmitted to the database, providing a continuous flow of real-time data and a seamless user experience.
Additionally, the blendshape data from the Firebase Realtime Database is transmitted to Google Cloud’s Immersive Stream for XR instances in real time, enabling real-time streaming of facial animation with minimal latency.
In terms of future plans, KDDI aims to further eliminate the boundary between the real and virtual worlds, enabling users to enjoy experiences such as live music, art, conversations, and shopping anytime, anywhere. KDDIâ??s αU service is also shaping an ecosystem for the Web3 era, supporting a new generation of users who move between real and virtual worlds with ease.
https://developers.googleblog.com/2023/07/mediapipe-enhancing-virtual-humans-to-be-more-realistic.html
https://developers.googleblog.com/2023/07/mediapipe-enhancing-virtual-humans-to-be-more-realistic.html