#kaistackr #scatterlabcokr #ludaai
The article presents the latest developments in 3D digital human content, highlighting a novel method for manipulating 3D facial representations. This approach, developed by KAIST and **Scatter Lab** researchers, uses around 300 training frames from a dynamic portrait video, each depicting different facial deformations, to enable text-driven modifications. This method uses HyperNeRF to discern and isolate observed deformations before manipulating facial expressions. It uses multiple spatially variable latent codes, addressing the “linked local attribute problem” and providing a nuanced approach to scene deformations for manipulation tasks. Using an MLP-trained combination of anchor codes, the technique generates numerous position-conditional latent codes, which are enhanced to align with a target text in CLIP embedding space, thereby ensuring the visual characteristics of the text are mirrored in the latent codes.