Notes:
Various developers have created a multitude of projects focused on lip synchronization technology and its applications, ranging from converting 2D photos into 3D lip-synced versions, creating photo-realistic lip sync from text, using AI avatars for videos, automatic 2D mouth animation from voice recordings, tools for different platforms like Unity3D, Autodesk Maya, and Blender, and assistive technologies like a mouth-operated joystick for individuals with limited hand function. Additionally, some projects provide solutions for games, animation creation software, and other multimedia integrations.
Resources:
See also:
100 Best Annosoft Lipsync Videos | 100 Best Blender Lipsync Videos | 100 Best Cinema 4D Lipsync Videos | 100 Best Daz3d Lipsync Videos | 100 Best Flash Lipsync Videos | 100 Best Oculus Lipsync Videos | 100 Best Papagayo Lipsync Videos | 100 Best SALSA LipSync Videos | 100 Best ToonBoom Lipsync Videos | 100 Best Unity3d Lipsync Assets | 100 Best Unity3d Lipsync Videos | 100 Best UnrealEngine Lipsync Videos | Automatic Lip Synchronization (Lipsync)
[47x Sep 2023]
- 01010101lzy/kk-lipsync .. an experimental lip sync project built for koikatsu and ai shoujou. (1)
- abhi7410/dass_project .. this project aims to turn 2d photographs provided by the user into 3d versions that can be lip-synced with an audio file of the user’s choice.
- acvictor/obama-lip-sync .. an implementation of obamanet: photo-realistic lip-sync from text.
- ajay-sainy/wav2lip-gfpgan .. high quality lip sync.
- amtsai96/learning-lip-sync-from-audio .. learning lip sync of obama from speech audio.
- android-iceland/lipsync-avatar .. use your own video to generate ai avatar video
- anxdpanic/plugin.video.youtube .. watch your favorite youtube content on kodi.
- automattic/vu-vrm .. lip-sync vrm avatar client for zero-webcam mic-based vtubing
- danielswolf/rhubarb-lip-sync .. rhubarb lip sync is a command-line tool that automatically creates 2d mouth animation from voice recordings.
- denchisoft/vtubestudio .. vtube studio api development page.
- dhall5/blender-game-engine-lip-sync .. a bge lip sync plugin using shapekeys and animation timelines along with “large vocabulary continuous speech recognition engine julius”
- google/lipsync .. lipsync system employs tensorflow.js facemesh for real-time facial key point detection
- guanjz20/stylesync .. official code of cvpr ’23 paper “stylesync: high-fidelity generalized and personalized lip sync in style-based generator”
- hecomi/mmd4mecanim-lipsync-plugin .. lipsync and tts plugin for mmd4mecanim.
- hecomi/ulipsync .. mfcc-based lipsync plug-in for unity using job system and burst compiler
- huailiang/lipsync .. lipsync for unity3d generates mouth animation based on speech and supports fmod
- joaen/maya-auto-lip-sync .. a tool to automatically generate lip sync animations in autodesk: maya using montreal forced aligner.
- libgdx/gdx-video .. a libgdx cross platform video rendering extension.
- lynzrand/kk-lipsync .. an experimental lip sync project built for koikatsu and ai shoujou. (2)
- makersmakingchange/lipsync .. an open-source mouth operated sip and puff joystick that enables people with limited hand function emulate a mouse on their computer and/or smartphone.
- makersmakingchange/lipsync-gaming .. a mouth operated gaming joystick based on the lipsync.
- makersmakingchange/lipsync-macro .. lipsync macro is an open-source device enabling quadriplegics to use touchscreens
- morevnaproject-org/papagayo-ng .. papagayo is a lip-syncing program designed to help you line up phonemes (mouth shapes) with the actual recorded sound of actors speaking.
- mowshon/lipsync .. lip synchronization (wav2lip).
- oobabooga/text-generation-webui .. a gradio web ui for large language models. supports transformers, gptq, llama.cpp (ggml/gguf), llama models.
- opentoonz/opentoonz .. opentoonz – an open-source full-featured 2d animation creation software
- philcryer/lipsync .. lipsync sets up a lightweight service that provides command-line, dropbox like syncing
- powroupi/blender_mmd_tools .. mmd_tools is a blender addon for importing models and motions of mikumikudance.
- prakruthijainys/lipsync .. synchronizing an audio file with a video file.
- renpy/renpy .. the ren’py visual novel engine.
- rtyper/lipsync-pro .. official repo for rogo digital’s lipsync pro for unity – formerly available on the asset store
- rudrabha/wav2lip .. this repository contains the codes of “a lip sync expert is all you need for speech to lip generation in the wild”, published at acm multimedia 2020.
- s-soltys/lipsync .. lip animation app for 3d face models.
- scaredyfish/blender-rhubarb-lipsync .. blender rhubarb lipsync is an addon for blender integrating rhubarb lip sync to automatically generate mouth-shape keyframes from a pose library.
- shiyatzu/oculuslipsyncplugin-ue5 .. oculus lipsync plugin compiled for unreal engine 5. this plugin allows you to synchronize the lips of 3d characters in your game with audio in, using the oculus lipsync technology.
- snehitvaddi/deepfake-using-wave2lip .. a deep learning model to lip-sync a given video with any given audio. it uses gan architecture to orchestrate loss reconstruction or training.
- taekyungki/stylelipsync .. official pytorch implementation of “stylelipsync: style-based personalized lip-sync video generation”.
- ubicastteam/qr-lipsync .. qrlipsync is an audio-video latency (also referred to as lipsync) measurement and validation tool.
- vamdeluxe/lipsync .. a node command line utility to generate virt-a-mate lip sync animation.
- waseemyusuf/houdini-lip-sync .. a houdini shelf tool that creates a simple chop network that can be used to lip-sync automatically
- wzpan/rhubarb-lip-sync-ccc .. cocos creator extension for rhubarb-lip-sync.
- zz-ww/sadtalker-video-lip-sync .. this project implements wav2lip for video lip synthesis based on sadtalkers.