Real-time Graphics Pipeline & Digital Humans

Notes:

A real-time graphics pipeline is a set of algorithms and computational processes that are used to generate and render graphics in real time. This can include generating the 3D models and textures of objects, as well as lighting and shading the objects to create a realistic and immersive visual experience.

Real-time graphics pipelines are used in a variety of applications, including video games, virtual reality, and simulations. They are particularly important in the context of digital humans, as they enable the creation of realistic and lifelike digital characters that can be used in these applications.

In the context of digital humans, a real-time graphics pipeline is used to generate the 3D model and appearance of the digital character, as well as to animate the character and make it move and behave in a realistic and believable way. This can involve using algorithms and techniques such as motion capture, inverse kinematics, and facial animation to create realistic and expressive digital characters.

Gaming pipeline is a set of algorithms and computational processes that are used to create and render the graphics and other visual elements in a video game. This typically involves generating the 3D models and textures of objects and characters, as well as simulating the behavior and interactions of these objects and characters within the game world. The gaming pipeline is an essential component of any video game, as it enables the creation of the visual elements that make up the game world.
Rendering pipeline is a similar concept to a gaming pipeline, but is typically used in the context of computer graphics and visualization, rather than in video games. A rendering pipeline is a set of algorithms and processes that are used to generate and render 3D graphics and other visual elements, such as lighting and shading, in a way that is realistic and believable. This can involve using techniques such as ray tracing, global illumination, and ambient occlusion to create high-quality, photo-realistic images and animations.
Tele-immersion is a term that refers to the use of technology to create a sense of presence and immersion in a remote environment. This can involve using virtual reality or other immersive technologies to create a real-time, interactive experience that allows users to feel as though they are physically present in the remote environment. Tele-immersion is often used in applications such as teleconferencing, telemedicine, and remote collaboration, and can help to overcome the limitations of traditional communication and collaboration tools.

Wikipedia:

References:

Development of a standardized framework for cost-effective communication system based on 3D data streaming and real-time 3D reconstruction
DDH Huynh – 2017 – tamucc-ir.tdl.org
… 33 20 Streaming and Rendering Pipeline … for AR/VR, this data can be projected in 3D space around the users in real time and significantly enhance the user experience … This will make the avatars can only look like a bust statue of a person. Oculus Avatar also …

A framework for cost-effective communication system for 3D data streaming and real-time 3D reconstruction
D Huynh, SA King, AK Katangur – … of the 3rd International Workshop on …, 2018 – dl.acm.org
… Because this needs to be accomplished in real time, Open Sound Control [18], which has UDP protocol lying underneath is … The user’s avatar and the other participant avatar can be rendered into one single space as shown in Figure … Figure 5: Streaming and Rendering Pipeline …

Real-Time Hair Rendering using Sequential Adversarial Networks
L Wei, L Hu, V Kim, E Yumer… – Proceedings of the …, 2018 – openaccess.thecvf.com
… Given a pair of input/output images for each stage in the rendering pipeline (eg the … as hair modeling with interactive visual feedback, photo ma- nipulation, and image-based 3D avatar rendering … 2018) 2. Cao, C., Wu, H., Weng, Y., Shao, T., Zhou, K.: Real-time facial animation …

Deep appearance models for face rendering
S Lombardi, J Saragih, T Simon, Y Sheikh – ACM Transactions on …, 2018 – dl.acm.org
… Mesh Mesh Encoder Decoder Encoder Decoder Input Animated Avatar … This manual process makes them unsuitable for real-time applications. The primary challenge posed by the traditional graphics pipeline stems form its physics-inspired model of light transport …

Real-Time Collision Detection for Deformable Characters with Radial Fields
SJ Friston, A Steed – IEEE transactions on visualization and …, 2018 – ieeexplore.ieee.org
… The avatars were animated, but the pose was such that body parts did not occlude each … REAL-TIME COLLISION DETECTION FOR DEFORMABLE CHARACTERS WITH RADIAL FIELDS 4 … case, a set of radial fields are applied coincident with the bones of an articulated avatar …

Holographic signing avatars for deaf education
N Adamo-Villani, S Anasingaraju – E-Learning, E-Education, and Online …, 2017 – Springer
… The avatar, seen thorough the AR glasses by the child with the red shirt, is … platform offered two main advantages: (1) Unity has an optimized graphics pipeline that supports … could see the holographic virtual sign language interpreters translating the teacher’s lesson in real time …

Avatar Administered Neuropsychological Testing (AVANT): Stroop Interference Task
TD Parsons, N Russo… – Proceedings of the …, 2017 – journals.sagepub.com
… Anima- tion software was utilized for development of the virtual hu- man avatar and VRST environment. The environments were rendered in real time using a graphics engine with a fully cus- tomizable rendering pipeline, including vertex and pixel shaders, shadows …

Physical Keyboards in Virtual Reality: Analysis of Typing Performance and Effects of Avatar Hands
P Knierim, V Schwind, AM Feit… – Proceedings of the …, 2018 – dl.acm.org
… Avatar Hands The effectiveness of virtual environments has been linked to the subjective … NetNat SDK for streaming position data of bones, joints, and keyboard in real time … The summed up calculated latency caused by motion tracking, rendering pipeline, and HMD never …

Deferred Neural Rendering: Image Synthesis using Neural Textures
J Thies, M Zollhöfer, M Nießner – arXiv preprint arXiv:1904.12356, 2019 – arxiv.org
… synthesis based on imperfect commodity 3D reconstructions at real-time rates, • Neural Textures for novel view synthesis in static scenes and for editing dynamic objects, • which is achieved by an end-to-end learned novel deferred neural rendering pipeline that combines …

A generative model of people in clothing
C Lassner, G Pons-Moll… – Proceedings of the IEEE …, 2017 – openaccess.thecvf.com
… We sidestep the com- monly used complex graphics rendering pipeline and the need for high-quality 3D scans of dressed … A pipeline including 3D avatar generation, 2D pattern design, physical simulation to drape the cloth, and texture mapping is … [48], 3D avatars in virtual …

Real time stable haptic rendering of 3D deformable streaming surface
Y Tian, C Li, X Guo, B Prabhakaran – Proceedings of the 8th ACM on …, 2017 – dl.acm.org
… 3 OVERVIEW The proposed method introduces real-time haptic rendering of 3D deformable object under reconstruction … Our method uses the multi-rate architecture [18] splitting the haptic rendering pipeline into a haptic thread and a simulation thread …

Toward a Mobile Gaming Based-Computation Offloading
F Messaoudi, A Ksentini, P Bertin – 2018 IEEE International …, 2018 – ieeexplore.ieee.org
… Shooter (FPS) games, which are very high resource consuming and real- time interacting … engines, since the 3D- scene undergoes several transformations through the rendering pipeline before getting … The NPC is a humanoid avatar triggered by the player characters when they …

Face2Face: Real-time facial reenactment
J Thies – it-Information Technology, 2017 – degruyter.com
… Tour; Sign up for a free trial; Subscribe. OWV_DGO Logo. Search Close. Advanced SearchHelp. My Content (1) Recently viewed (1). Face2Face: Real-time f… My Searches (0). (0) My CartAdded To Cart. Check Out. Menu: Subjects …

Expecting the unexpected: Training detectors for unusual pedestrians with adversarial imposters
S Huang, D Ramanan – Proceedings of the IEEE …, 2017 – openaccess.thecvf.com
… In this section, we describe our rendering pipeline for gener- ating synthetic data … the background image index, the number of 3D models, and for each model, an index specifying the avatar ID and … 3D game engine design: a practical approach to real-time computer graphics …

The AR-Rift 2 Prototype
A Steed, YW Adipradana… – 2017 IEEE Virtual Reality …, 2017 – ieeexplore.ieee.org
… A second problem is that the camera im- agery has higher latency than the real-time generated imagery … We support the Oculus Touch for interaction with the scene. A default scene including avatar animation, interaction with objects and AR/VR switching is available …

Inversefacenet: Deep single-shot inverse face rendering from a single image
H Kim, M Zollhöfer, A Tewari, J Thies… – arXiv preprint arXiv …, 2017 – pure.mpg.de
… Inverse Rendering (of Faces) The goal of inverse render- ing is to invert the graphics pipeline, ie, to recover the geometry, reflectance (albedo) and illumination from images or videos of a scene – or, in our case, a face … Thies et al. [45] recently proposed a real-time inverse ren …

Augmented virtual environment: fusion of real-time video and 3D models in the digital earth system
H Jian, J Liao, X Fan, Z Xue – International journal of digital earth, 2017 – Taylor & Francis
… The 3D rendering pipeline deals with the scene vertex by vertex and fragment by fragment, which makes it difficult … Draw the trajectory of the virtual moving object in real time … 3D model that can display the moving object in the virtual scene, for example, a virtual human model or …

Faceforge: Markerless non-rigid face multi-projection mapping
C Siegl, V Lange, M Stamminger… – IEEE transactions on …, 2017 – ieeexplore.ieee.org
… 5 FACE RECONSTRUCTION A core component of our system is the reconstruction and non-rigid real-time tracking of a human face … The synthesized face images ˆCi are generated using a full perspective projection ?(?x) in the standard graphics pipeline …

Inversefacenet: Deep monocular inverse face rendering
H Kim, M Zollhöfer, A Tewari, J Thies… – Proceedings of the …, 2018 – openaccess.thecvf.com
… 2. Related Work Inverse Rendering (of Faces) The goal of inverse render- ing is to invert the graphics pipeline, ie, to recover the ge- ometry, reflectance (albedo) and illumination from images or videos of a scene – or, in our … [60] recently proposed a real-time inverse rendering …

Android STAR: An Efficient Interaction-Preserving Record-Replay System For Messenger App Usage Surveillance
LP Yuan, P Liu, S Zhu – Proceedings of the 11th ACM Conference on …, 2018 – dl.acm.org
… It can be a text message, a tag, a timestamp, a picture, an avatar, an emoji, or an icon … STAR Recorder (Fig. 4) is a service module integrated with An- droid graphics pipeline … It consists of three components: a real-time composer, a bitmap handler, and a root service …

Use of photogrammetry in video games: a historical overview
N Statham – Games and Culture, 2018 – journals.sagepub.com
In 2014, the developers of The Vanishing of Ethan Carter revealed that the stunning settings in the game were created using photogrammetry, a modeling technique…

Combining GPU-generated linear trajectory segments to create collision-free paths for real-time ambient crowds
O Barut, M Haciomeroglu, EA Sezer – Graphical Models, 2018 – Elsevier
… which is highly overlapping with the requirements of the aforementioned real-time ambient crowds … then improved their paradigm by exploiting the readily available rendering pipeline of the … Moreover, virtual humans walking on linear paths without any maneuvers might have a …

Double logarithmic arithmetic technique for low-power 3-D graphics applications
DM Ellaithy, MA El-Moursy, GH Ibrahim… – … Transactions on Very …, 2017 – ieeexplore.ieee.org
… applications in mobile workstations due to their benefits for gaming, advertising, marketing, and avatars … These functions use most of the clock cycles in real-time 3-D graphics systems … Basically, the 3-D graphics rendering pipeline is composed of vertex shader and pixel shader …

Deepwrinkles: Accurate and realistic clothing modeling
Z Lahner, D Cremers, T Tung – Proceedings of the …, 2018 – openaccess.thecvf.com
… highest possible level of detail at runtime, we propose to revisit the traditional rendering pipeline of 3D … Here, a virtual shirt is seamlessly added on top of an animated virtual human (eg, scanned … de Aguiar, E., Sigal, L., Treuille, A., Hodgins, J.: Stable spaces for real-time clothing …

Interactive molecular graphics for augmented reality using HoloLens
C Müller, M Krone, M Huber, V Biener… – Journal of integrative …, 2018 – degruyter.com
… All frame rates were measured by averaging over 500 ms periods. Following each of these periods, we additionally obtained statistics from the graphics pipeline comprising the geometry load and shader invocations of the last frame of the period …

TransparentHMD: revealing the HMD user’s face to bystanders
C Mai, L Rambold, M Khamis – … of the 16th International Conference on …, 2017 – dl.acm.org
… This information is then used to render a 3D-face model projection in real time on the image … inner part of the HTC Vive 5. Furthermore we are able to let the avatar look into … Projection in Rendering Pipeline To create a mapping from 3D-coordinates to 2D-coordinates that can be …

A direct 3D object tracking method based on dynamic textured model rendering and extended dense feature fields
L Zhong, M Lu, L Zhang – … on Circuits and Systems for Video …, 2018 – ieeexplore.ieee.org
… and the image. In order to render realistic template images from a 3D object model, the OpenGL-based rendering pipeline includes a Modelview Transform and a Projective Transform for all of the vertices [22]. Firstly, a Modelview …

Photorealistic rendering: a survey on evaluation
H Kolivand, MS Sunar, SY Kakh, R Al-Rousan… – Multimedia Tools and …, 2018 – Springer
… the interaction between virtual lighting and real ones was the main target of this rendering pipeline … could be addressed by casting virtual shadows on realm objects in real-time using a … same work but a statistical parametric approach for video-realistic text-driven talking avatar …

A skeleton-free fall detection system from depth images using random decision forest
A Abobakr, M Hossny, S Nahavandi – IEEE Systems Journal, 2018 – ieeexplore.ieee.org
… Fig. 5. Articulating a virtual human model … 4. This rendering pipeline could be considered as a generative model that could synthesize a virtually infinite set of images … The features being computationally efficient is an additional constraint to ensure real-time perfor- mance …

Rotation blurring: use of artificial blurring to reduce cybersickness in virtual reality first person shooters
P Budhiraja, MR Miller, AK Modi, D Forsyth – arXiv preprint arXiv …, 2017 – arxiv.org
… our solution to be that it should have in- significant computational load and work solely with the rendering pipeline … provides motion cues helps to reduce SS in driving simula- tors [18]; the avatar also enhances … 17] describe an algorithm for implementing a real-time adaptive blur …

Virtual Reality Based Immersive Telepresence System for Remote Conversation and Collaboration
Z Tan, Y Hu, K Xu – International Workshop on Next Generation Computer …, 2017 – Springer
… avatars, before sending them along with other scene objects to the rendering pipeline; while the … have been constantly making progress to propose much more effective real-time lip synchronization … Using the handheld controller to drive the avatar still has its advantage of better …

Deep incremental learning for efficient high-fidelity face tracking
C Wu, T Shiratori, Y Sheikh – SIGGRAPH Asia 2018 Technical Papers, 2018 – dl.acm.org
… Inspired by the conventional graphics pipeline – which combined traditional blendshapes with corrective blendshapes – we introduce a Decomposition Layer in … Many existing approaches use holistic linear models for real-time facial tracking, which optimize rigid head pose and …

VR Enhanced Interactive Intubation Simulation
Y Meng – Proceedings of the International Conference on …, 2018 – csce.ucmss.com
… Abstract— This paper introduces a new haptic rendering pipeline for neonatal endotracheal intubation … directly impacts the training or surgical planning results, while real- time interactive haptic … nature and oversimplified mapping between haptic device and its virtual avatar …

Qoe studies on interactive 3d tele-immersion
K Desai, S Raghuraman, R Jin… – … on multimedia (ISM), 2017 – ieeexplore.ieee.org
… is influenced by several factors such as the quality of the “live” 3D avatars of the … comprehensive survey of these efforts can be found in [7]. Approaches for real time 3D reconstruction … the cameras, the framework uses a rendering pipeline and an interaction pipeline, as shown in …

Synthesizing a scene-specific pedestrian detector and pose estimator for static video surveillance
H Hattori, N Lee, VN Boddeti, F Beainy… – International Journal of …, 2018 – Springer
… into the data synthesis pipeline to generate realistic renders of virtual humans; (2) we … The rendering pipeline can precisely control the following variations in human appearance: gender … Our virtual human database consists of 139 different models spanning gender, clothing color …

Deep spherical harmonics light probe estimator for mixed reality games
BAD Marques, EWG Clua, CN Vasconcelos – Computers & Graphics, 2018 – Elsevier
… This substitution of the avatar’s synthetic upper limbs to the real user’s upper limbs is not straightforward … Thus it’s not suitable for real-time environments. Knetch et al … The environment map is employed in a rendering pipeline to render virtual objects into real scenes …

Virtual 3-D Walkthrough for Intelligent Emergency Response
N Saxena, V Diwan – … of International Conference on Computer Vision and …, 2017 – Springer
… able to access various kinds of information interacting with objects or avatars and travel … the capability to rapid development of virtual applications with near real-time rendering powers … and quick; DirectX and OpenGL have highly optimized rendering graphics pipeline, it makes …

Enabling Real-Time Shared Environments on Mobile Head-Mounted Displays
MG Cook – 2017 – otago.ourarchive.ac.nz
… One of the key goals of the work in this thesis is to develop a platform that will enable research into the elements affecting presence for users in a real-time shared environment … a personable avatar to communicate with; one designed for spatial presence by assigning …

Game engine architecture
J Gregory – 2017 – taylorfrancis.com
… 308 7 The Game Loop and Real-Time Simulation 339 …

Evaluating BVH splitting strategies
TT Le – 2017 – trungtuanle.com
… Page 12. Trung Le Evaluating BVH splitting strategies for GPU ray tracing 11 Deferred rendering has gained major popularity in real-time rendering. Some of … It is common to use in games to enclose human shaped player avatar or other non-playable characters …

Appearance Modelling of Living Human Tissues
ALP Nunes, A Maciel, GW Meyer… – Computer Graphics …, 2019 – Wiley Online Library
… a survey of research targeting living digital human tissue representations, particularly for rendering purposes. Material modelling of living human tissue has many applications, from increasing realism in the entertainment industry and photorealistic digital humans to medical …

Virtual and Augmented Reality
V Trivedi – How to Speak Tech, 2019 – Springer
… as you move your head around, it’s scrambling to render the scene for you in real time … building VR/AR content, one must take the necessary precautions to optimize the rendering pipeline to avoid … move in a realistic manner, or for the legs to move when your avatar walks around …

A measurement study on achieving imperceptible latency in mobile cloud gaming
T Kämäräinen, M Siekkinen, A Ylä-Jääski… – Proceedings of the 8th …, 2017 – dl.acm.org
… Using LTE, only the USB and gyro control options fall under the limit. 4 OPTIMIZING SERVER PLACEMENT: CASE EUROPE In this section we measure the delay in the wired network path of the cloud gaming pipeline. This is the last piece missing from the total delay …

Surface Motion Capture Animation Synthesis
A Boukhayma, E Boyer – IEEE transactions on visualization and …, 2018 – ieeexplore.ieee.org
Page 1. 1077-2626 (c) 2018 IEEE. Personal use is permitted, but republication/ redistribution requires IEEE permission. See http://www.ieee.org/ publications_standards/publications/rights/index.html for more information. This …

Sprite tree: an efficient image-based representation for networked virtual environments
M Zhu, G Morin, V Charvillat, WT Ooi – The Visual Computer, 2017 – Springer
… But these meshes may require a high bandwidth for real-time downloading, such as the Thai Statue mentioned above … based rendering algorithms [24, 26] does not depend on the scene complexity and resolution, unlike rendering through a geometry-based rendering pipeline …

Web-based medical data visualization and information sharing towards application in distributed diagnosis
Q Zhang – Informatics in Medicine Unlocked, 2019 – Elsevier
… Mobeen and Feng designed a single-pass rendering pipeline for mobile devices, where they used 2D texture to simulate 3D texture for … this system could effectively transmit data between server to client computers and launch the program on client computers in near real time …

Firefly: Virtual Illumination Drones for Interactive Visualization
S Stoppel, MP Erga, S Bruckner – IEEE transactions on …, 2019 – ieeexplore.ieee.org
… To achieve interactive performance, we employ a parallel rendering pipeline for the light path evaluations … 18] used real-time shadow maps … focused on the emotions induced by different lighting setups and proposes a model for the expression of emotions in virtual humans with a …

Practical Dynamic Facial Appearance Modeling and Acquisition
R JÉRÉMY, A GHOSH – 2018 – disneyresearch.s3.amazonaws.com
… illumination) and employed the acquired data for building an efficient real-time rendering technique … These approaches focus on easily creating a believable digital avatar rather than accurate … 4.1 Inverse Rendering At the core, our inverse rendering pipeline estimates optimal …

A Bibliography of Publications in ACM SIGGRAPH Computer Graphics: 2000–2009
NHF Beebe – 2017 – ctan.math.utah.edu
… automatic [CS01, HSKK01]. Automating [SM01]. Avatar [Lap00]. Avatars98 [DG00] … readers [GA04d]. Real [PHWF01, SR00, Ama08, DDCB01, Lar06, LYS01, PMTH01, Umb06, Wes06d]. Real-time [PHWF01, SR00, Ama08, DDCB01, Lar06, PMTH01]. real-world [LYS01] …

Kinect Programming with Direct3D 11
J Zink – GPU Pro 360 Guide to 3D Engine Design, 2018 – books.google.com
… Each of these vertices is then assembled into a triangle by the graphics pipeline and passed to the … resulting combined scene can then be rendered and presented to the user in real time from the … for the user in the scene, it becomes quite easy to render an avatar that appears in …

Novel Hybrid-Cast Approach to Reduce Bandwidth and Latency for Cloud-Based Virtual Space
X Hou, Y Lu, S Dey – ACM Transactions on Multimedia Computing …, 2018 – dl.acm.org
… and com- municate in the same classroom/gallery session, rendered as avatars in the … we de- velop a common view extraction algorithm by exploring the 3D rendering pipeline, and (iii … an automated and smart grouping algorithm, with fast and accurate real-time evaluation with …

Fragmentation Guided Human Shape Reconstruction
Y Zhang, X Luo, W Yang, J Yu – IEEE Access, 2019 – ieeexplore.ieee.org
… traditional motion analysis [1], to visual special effects [2], and to photorealistic avatars in virtual … coarser geometry compared with multi-view camera system but can achieve real-time performance … This can be easily achieved by reusing the rendering pipeline, eg, under OpenGL …

Mobile Augmented Reality Applications in Education
I Sural – Mobile technologies and augmented reality in open …, 2017 – igi-global.com
… live video stream via a weak perspective projection camera model and an OpenGL rendering pipeline … information mod- els with the situation at the construction site in real time, as well … capture, and train virtual creatures, called Pokémon (Figure 5). Players use avatar which can …

Crowd simulation in an emergency situation
D Roca López – 2017 – upcommons.upc.edu
… 31 4.2 Graphics Pipeline ….. 32 4.3 Summary of Optimizations ….. 33 … 12 Chapter 2.2.1 Customizing Avatars Figure 10: Customizing avatar’s body with adobe fuse ….. 13 …

Image Quality-Driven Level of Detail Selection on a Triangle Budget
L Arlebrink, F Linde – 2018 – diva-portal.org
… will only differ in how the LOD versions are selected, the rest of the rendering pipeline will be … mesh, a pre-computing step to determine view-dependent LOD ver- sions, and a real-time stage where … are positioned at the normal height on the Y-axis for a humanoid avatar in a first …

Collaborative design model review in the AEC industry
JG Cárcamo, H Trefftz, DA Acosta, LF Botero – International Journal on …, 2017 – Springer
… eg user’s avatar collision with the floor and walls objects to give the impression … This collaborative virtual tool should enable real-time visualisation of the BIM models produced by … called CineBox, with support of full stereoscopic pipeline, fully linear rendering pipeline and import …

Assessment of an augmented reality apparatus for the study of visually guided walking and obstacle crossing
K Binaee, GJ Diaz – Behavior research methods, 2019 – Springer
… The simultaneous use of motion-capture for real-time tracking of the subject’s position during locomotion … resolve this through the use of motion tracking to animate a virtual avatar visible inside … axis defined by the HMD and the viewing frustum within the rendering pipeline yield a …

Performance analysis of game engines on mobile and fixed devices
F Messaoudi, A Ksentini, G Simon… – ACM Transactions on …, 2017 – dl.acm.org
… depending on whether the player is immersed in the scene or the avatar representing the … The rendering pipeline (see Section 2.3) is the part of the game engine that generates one … fighting game is different from an FPS engine, an MMOG engine, a Real-Time Strategy (RTS …

Multimodal serious games technologies for cultural heritage
F Liarokapis, P Petridis, D Andrews… – Mixed Reality and …, 2017 – Springer
… The programmable rendering pipeline of modern consumer graphics hardware allows the use of new … Visitors enter a specifically designed on-site kiosk, wherein real-time video images and … system allows the ren- dering of realistic 3D simulations of virtual humans, animals and …

A short survey on modern virtual environments that utilize AI and synthetic data
M Korakakis, P Mylonas, E Spyrou – 2018 – aisel.aisnet.org
… are accommodated within networked computers, allowing users to participate via personalized avatars … convolutional neural network that utilizes an on-the-fly rendering pipeline generating realistic … of deep learning algorithms in video games within the Real-Time Strategy (RTS …

Improving the visual momentum of tethered viewpoint displays using spatial cue augmentation
RY Tara, WC Teng – Intelligent Service Robotics, 2017 – Springer
… viewer to have unobstructed views of the target, which includes the avatar and its … The presented approach was directly implemented in the GPU rendering pipeline using the OpenGL … M, Gedikli S, Navab N (2012) Adaptive neighborhood selection for real-time surface normal …

IBIS: an OR ready open-source platform for image-guided neurosurgery
S Drouin, A Kochanowska, M Kersten-Oertel… – International journal of …, 2017 – Springer
… kinds of volumetric rendering techniques, and polygonal models are rendered using a traditional 3D rendering pipeline … track the checkerboard pattern of the calibration device in real time, helping the … b The segmented tumor and avatar showing location of tumor (virtual image) …

Gaze Estimation with Graphics
EW Wood – 2017 – repository.cam.ac.uk
… 107 7 Analysis-by-synthesis in real time 109 … Since eye gaze is an important part of any facial performance it should be captured along with the rest of the face. Beyond games, facial performance capture is also important for remote communication with lifelike 3D avatars …

Generating 3D Faces by Tracking and Pose Estimation in Video Streams
Q Wang – 2018 – pdfs.semanticscholar.org
… 44 Figure 24 3D transformations in rendering pipeline ….. 45 … 48 Figure 28 Result of real time rendering ….. 49 … variety of 3D avatars. Many video game and movie companies used MSQRD to promote …

Model-driven simulations for computer vision
VSR Veeravasarapu, C Rothkopf… – 2017 IEEE Winter …, 2017 – ieeexplore.ieee.org
… Especially, the impact of modeling errors and com- putational rendering approximations, due to choices in the rendering pipeline, on trained CV systems generalization performance is … However, these game engines were developed with real time rendering for gaming purposes …

An overview of serious game engines and frameworks
B Cowan, B Kapralos – Recent Advances in Technologies for Inclusive …, 2017 – Springer
… of user-created content forces the Second Life server to send every user’s avatar and objects … Engine 4 features a 64-bit color HDR (High Dynamic Range) rendering pipeline with a … known for having some of the most realistic lighting effects, including real-time global illumination …

LumiWatch: On-Arm Projected Graphics and Touch Input
R Xiao, T Cao, N Guo, J Zhuo, Y Zhang… – Proceedings of the 2018 …, 2018 – dl.acm.org
… graphics. Owing to the shallow angle of projection, our graphics pipeline must rectify interfaces to the complex, non-planar geometry of the arm … order. The system ran the tap detector in real-time, record- ing any tap event received …

Realistic Simulation of Cultural Heritage
C Kiourt, G Pavlidis, A Koutsoudis… – International Journal of …, 2017 – igi-global.com
… A simple comparative example of using this technique in the presented system is shown in Figure 2. There are several variations of real-time ambient occlusion … Object space methods operate on 3D scene representation, and build DoF effects directly into the rendering pipeline …

Multi-scale Dynamic Capture for High Quality Digital Humans
K Nagano – 2017 – search.proquest.com
… meet the increasing demands in high quality digital avatars for AR/VR applications, which allow … Such models have proven useful in recent work for real-time performance … reconstruction of a personalized avatar from mobile free-form videos [69], medium scale dynamic …

Virtual Environment Grocery Store
TD Parsons, T McMahan, P Melugin… – The Role of Technology …, 2017 – books.google.com
… In summary, VE-based neuropsychological assessments allow for real-time measurement of multiple neurocognitive abilities in order to assess complex sets of skills and … stand behind a shopping cart, visual- izing the VAP-S from a first-person perspective (ie, no avatar is visible …

Legible Visualization of Semi-Transparent Objects using Light Transport
D Murray – 2018 – tel.archives-ouvertes.fr
… Keywords Rendering, GPU, Rasterization, Real-Time, Global Illumination, Volume Rendering, Scattering … 19 1.10 Graphics pipeline … TM ). (b) A footage of the movie Avatar (courtesy of XXth Century Fox TM ). (c) Physically-plausible rendering …

Haptic display for virtual reality: progress and challenges
D WANG, Y GUO, S LIU, Y ZHANG… – Virtual Reality & …, 2019 – researchgate.net
… 71]. The problem of hand-based haptic rendering can be conceptualized as the following pipeline. First, the joint angles of the hand avatar are measured in real time from a glove. With the predefined configuration and geometric …

Design and development of a collaborative virtual reality environment
M McMenamin – 2018 – theseus.fi
… 47 6.5 Avatar Customization ….. 48 … In Spaces, users are represented by self- created avatars and they can spend time with their friends … (Unity 2018b). Unity provides a highly-optimized stereoscopic rendering pipeline that is …

Real-Time Large Crowd Rendering with Efficient Character and Instance Management on GPU
Y Dong, C Peng – International Journal of Computer Games …, 2019 – hindawi.com
… In their experiment, they demonstrated a real-time rendering rate for a crowd composed of 15.3 million … However, their approach was based on the simulation of single virtual humans … With visibility culling, a renderer is able to reject a character from the rendering pipeline if it is …

A fast framework construction and visualization method for particle-based fluid
F Zhang, Z Wang, J Chang, J Zhang, F Tian – EURASIP Journal on Image …, 2017 – Springer
… details, it incurs much performance cost in sampling, and hardly achieve real-time rendering with … for fluid rendering, frame buffer is turned on after a certain rendering pipeline is chosen … His research focuses on a number of topics relating to 3D virtual human modeling, animation …

Building a Large Database of Facial Movements for Deformation Model-Based 3D Face Tracking
D Sibbing, L Kobbelt – Computer Graphics Forum, 2017 – Wiley Online Library
… Their system involves an expensive search in the database and does not run in real time … and occlusion maps to enhance the visual appearance of a virtual avatar [IBP15] … for each image sequence produced by each camera, we use the OpenGL rendering pipeline [AMHH08] to …

Low Latency Displays for Augmented Reality
PC Lincoln – 2017 – search.proquest.com
… CISCO • Avatar – Office of Naval Research Award N00014-09-1-0813: “3D Display and Capture of Humans for … In early graphics systems, this type of latency was typically expressed as the depth of the rendering pipeline and the …

Unity Virtual Reality Projects: Learn Virtual Reality by Developing More Than 10 Engaging Projects with Unity 2018
J Linowes – 2018 – books.google.com
… Table of Contents Options for voice chat 409 Using Oculus platform and avatars 410 Oculus platform entitlement check 411 Adding a local avatar 412 Adding remote avatars Building and sharing custom VRChat rooms 413 415 Preparing and building the world Summary 416 …

Network Streaming and Compression for Mixed Reality Tele-Immersion
RN Mekuria – 2017 – dare.ubvu.vu.nl
… error and delay resilient transmission techniques that can handle large frames in real time … include 3D visual reconstruction, 3D user analysis for embodying an avatar and 3D … Desired functionalities include emotion understanding and conversational capabilities of avatars …

Occlusion Culling in Batched Ray Traversal
ML Molenaar – 2018 – dspace.library.uu.nl
… 1. Introduction Rendering large scenes is an ongoing challenge in the computer graphics industry. Real-time visual- ization of complex meshes and photo realistic ren- dering of large scenes on machines without suf- ficient system memory remains an open problem …

Mobile augmented reality survey: From where we are to where we go
D Chatzopoulos, C Bermejo, Z Huang, P Hui – Ieee Access, 2017 – ieeexplore.ieee.org
… and mobile advertising market is more than $732 million [1]. By considering all the definitions from various researchers in the past [2], [3], [4], [5], [6], we can conclude that MAR: 1) combines real and virtual objects in a real environment, 2) is interactive in real time, 3) registers …

Underwater excavation serious game
P Kou?il – 2017 – is.muni.cz
… Also, while this work will focus only on real-time simulations of fluids, most of the approaches and principles discussed in this section are applicable to both interac- tive applications and offline rendering. 11 Page 23. 2. Related work and history …

Procedural Generation of 2D Levels for a Motion-based Platformer Game in Unity with Large Amount of Movement
N Erhard-Olsson – 2018 – diva-portal.org
… it is also important to match the motion that generates the movement of the avatar as closely … in terms of speed and predictable qualitative results, since it does not run in real-time nor does … by Omar Shehata, a shader is simply a program that runs in the graphics pipeline and tells …

A critical analysis of the evaluation practice in medical visualization
B Preim, T Ropinski, P Isenberg – … on Visual Computing for Biology and …, 2017 – hal.inria.fr
… with streamsurfaces and evenly- spaced volume seeding [HQK06] A Pipeline for Computer Aided Polyp Detection [RHD ? 06] Real-Time Illustration of Vascular Structures [LWP ? 06] Full Body Virtual Autopsies using a State-of-the-art Volume Rendering Pipeline [BHW ? 07 …

Game Development Using Python
JR Parker – 2018 – books.google.com
… We observe, for example, that the arrow keys, or sometimes the keys “W,” “A,” “S,” and “D,” are used to move the player’s avatar, but not the keys “R,” “G,” “N,” and “M”; there are … Not all computer games are high-resolution, immersive (ie, virtual reality), real-time games …

Interactive And Immersive Coastal Hydrodynamics
S Tavakkol – 2019 – researchgate.net
… The simulation and visualization are performed on the GPU, which enables the software to run faster than real-time. Celeris supports … tsunamis and wind-waves, there is no solution for a faster than real-time wave forecasting in coastal regions …

Data-Driven Modeling and Rendering of Force Responses from Elastic Tool Deformation
A Abdulali, R Rakhmatov, T Ogay, S Jeon – Sensors, 2018 – mdpi.com
… In such medical and haptic-enabled simulators, the physical interaction with a virtual human tissue or organ usually happens through the medium of interaction … In order to utilize the proposed model in a real-time simulation, we have designed a rendering pipeline where a …

Improving Usability of Mobile Applications Through Speculation and Distraction Minimization
K Lee – 2017 – deepblue.lib.umich.edu
… 81 4.1.2 Scheduling application-initiated interactions in the OS 82 4.1.3 Quantifying attention for real-time scheduling . . . 84 4.2 Overview … I show that by quantifying the driver’s available attention and the attention demand of an interaction, real-time …

Designing In-Headset Authoring Tools for Virtual Reality Video
C Nguyen – 2017 – search.proquest.com
… as Lookat.io [69] support adding annotations to VR video on a web browser but do not allow. users to review or discuss in real-time in VR headsets. 16 … In. contrast, 3D objects, UIs, or avatars are complex polygons processed in a full rendering. 17. pipeline …

Computational Methods for Immersive Perception
Q Sun – 2018 – search.proquest.com
… 89. 6.3.3 Real-time Optimization and Redirection . . . . . 93 … Papadopoulos, Bireswar Laha, Arie Kaufman. Poster: Buyers Satisfaction. in A Virtual Fitting Room Scenario Based on Realism of Avatar. IEEE. Symposium on 3D User Interfaces, 2015 …

Image Design and Animation
G Onwodi, F Osang, E Jituboh, O Olekan – 2017 – dspace.col.org
Page 1. COURSE MATERIAL Image Design and Animation National Open University of Nigeria Faculty of Sciences, Department of Computer Science Page 2. Page 3. Copyright This course has been developed as part of the …

Development and Validation of a Hybrid Virtual/Physical Nuss Procedure Surgical Trainer
MF Obeid – 2017 – digitalcommons.odu.edu
… 4 augmenting a physical manikin with a collocated virtual human avatar. They indicate that … “An interactive haptic-incorporated real-time hybrid training simulator for non-trocar-based minimally invasive surgeries such as the Nuss procedure can be developed utilizing and …

Mid-air hand interaction with optical tracking for 3D modelling
J Cui – 2019 – dr.ntu.edu.sg
Page 1. This document is downloaded from DR-NTU, Nanyang Technological University Library, Singapore. Title Mid-air hand interaction with optical tracking for 3D modelling Author(s) Cui, Jian Citation Cui, J. (2019). Mid-air …

A Data-Driven Approach For Automatic Visual Speech In Swedish Speech Synthesis Applications
J Hagrot – 2019 – diva-portal.org
… A The rendering pipeline … produced by this project will, given text input, drive the geometry of the face of a 3D character model in a real-time chat bot … The final application allows a customer, eg a service provider or business, to provide its clients with a chat bot avatar rendered in …

Haptic Feedback in Room-Scale VR
V Computing, P Erler – 2017 – pdfs.semanticscholar.org
… Padget and Hamm recommend avoiding arms or even full-body avatar because pose estimation … propose a sound model for real-time use, which can simulate sound reflections [PUK16] … The rest of the rendering pipeline works like surface splatting but with a Gaussian weighting …

Development Of A Library Of Intelligent Agents For VR Applications
A Bottino, E Battegazzorre – 2019 – webthesis.biblio.polito.it
… L’efficienza è intesa in termini di utilizzo delle risorse computazionali, ed è un requisito avere una buona frame rate in real time anche quando un … In this simulation, created using the free game-engine Unity, the patient takes control of an avatar of himself via a VR headset and …

The reality editor: an open and universal tool for understanding and controlling the physical world
VMJ Heun – 2017 – dspace.mit.edu
Page 1. The Reality Editor: An Open and Universal Tool for Understanding and Controlling the Physical World by Valentin Heun Diplom Designer Bauhaus Universitit Weimar, Germany 2009 Master of Science in Media Arts …

Current Status of Applying Artificial Intelligence in Digital Games
Y Miyake – Handbook of Digital Games and Entertainment …, 2017 – Springer
… AI in digital games is also real-time system and restricted by the limit of update time … Character AI A virtual human or animal in digital games is called “character.” A character controlled by a … The graphics pipeline should be updated in one frame in game, but AI pipeline needs not …

Empowering medical personnel to challenge through simulation-based training
JA White – 2017 – etheses.bham.ac.uk
Page 1. EMPOWERING MEDICAL PERSONNEL TO CHALLENGE THROUGH SIMULATION-BASED TRAINING by Jamie White A thesis submitted to The University of Birmingham For the degree of DOCTOR OF PHILOSOPHY (Ph.D.) Supervisors: Prof …

An Architecture for Designing Content Agnostic Game Mechanics for Educational Burst Games
T Baron – 2017 – search.proquest.com
… Altering the difficulty of the game in real time in response to the affective state of the player was shown to increase engagement. Potential causes of the issues with CAGE games and … 112 Real Time Affective State Analysis ….. 113 …

Multiview video: Acquisition, processing, compression, and virtual view rendering
O Stankiewicz, G Lafruit, M Doma?ski – Academic Press Library in Signal …, 2018 – Elsevier
… rendering under any perspective viewpoint becomes possible at relatively low cost at the client side, through the OpenGL/DirectX 3D rendering pipeline … Processing throughput—depending on the target usage (real time or off-line), some of the processing has to be done on the …

Comparing a Clipmap to a Sparse Voxel Octree for Global Illumination
E Arnebäck – … University of Technology and University of …, 2017 – pdfs.semanticscholar.org
… The surgeon does this by performing surgery inside a virtual human body … In his PhD thesis, Crassin developed GigaVoxels. It is a GPU-based rendering pipeline that uses voxels to render detailed scenes in interactive or real-time frame rates [21] …

Academic Press Library in Signal Processing, Volume 6: Image and Video Processing and Analysis and Computer Vision
R Chellappa, S Theodoridis – 2017 – books.google.com
… 2D Textures. In this way, rendering under any perspec- tive viewpoint becomes possible at relatively low cost at the client side, through the OpenGL/DirectX 3D rendering pipeline. Page 28. 5 1.1 Multiview video FIG. 1.2 From …