In progress
Lorem ipsum dolor sit amet consectetur adipiscing elit sed do eiusmod tempor incididunt ut labore
Ongoing work at MPI-INF
Publications
Papers on speech-driven facial animation, gaze, and animator-centric tools. The full list is also on Google Scholar.
In progress
Lorem ipsum dolor sit amet consectetur adipiscing elit sed do eiusmod tempor incididunt ut labore
Ongoing work at MPI-INF
ACM SIGGRAPH MIG, 2026
Constructing Dynamic Conversations with Neural Back-channel Generation and Voice-line Scheduling
This paper turns independently recorded voice lines into a two-speaker conversation, using a neural scheduler and a back-channel predictor to give animators control over turn-taking and listener responses.
Constructing Dynamic Conversations with Neural Back-channel Generation and Voice-line Scheduling Pan, Yifang, Chris Landreth, and Karan Singh. (2026). "Constructing Dynamic Conversations with Neural Back-channel Generation and Voice-line Scheduling." The 19th ACM SIGGRAPH Conference on Motion, Interaction, and Games (MIG 2026).
@inproceedings{pan2026voiceline,
author = {Pan, Yifang and Landreth, Chris and Singh, Karan},
title = {Constructing Dynamic Conversations with Neural Back-channel Generation and Voice-line Scheduling},
booktitle = {The 19th ACM SIGGRAPH Conference on Motion, Interaction, and Games (MIG '26)},
year = {2026},
publisher = {ACM},
doi = {10.1145/3828647.3857292}
}
ACM CHI, 2026
FAME: Exploring Expressive Facial Avatars for Lyrical and Non-Lyrical Music Visualization for d/Deaf Individuals
This paper presents FAME, a design probe that conveys music to d/Deaf and Hard of Hearing users through expressive facial animation, instrument highlights, and synchronized captions.
FAME: Exploring Expressive Facial Avatars for Lyrical and Non-Lyrical Music Visualization for d/Deaf Individuals Yoo, Suhyeon, Yifang Pan, Ashish Ajin Thomas, Karan Singh, and Khai N. Truong. (2026). "FAME: Exploring Expressive Facial Avatars for Lyrical and Non-Lyrical Music Visualization for d/Deaf Individuals." Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. 1-19.
@inproceedings{yoo2026fame,
author = {Yoo, Suhyeon and Pan, Yifang and Thomas, Ashish Ajin and Singh, Karan and Truong, Khai N.},
title = {FAME: Exploring Expressive Facial Avatars for Lyrical and Non-Lyrical Music Visualization for d/Deaf Individuals},
booktitle = {Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems},
series = {CHI '26},
year = {2026},
pages = {1--19},
publisher = {ACM},
doi = {10.1145/3772318.3790402}
}
ACM SIGGRAPH, 2025
Model See Model Do: Speech-Driven Facial Animation with Style Control
This paper introduces a speech-driven facial animation system with controllable style parameters for generating diverse and expressive character animations.
Model See Model Do: Speech-Driven Facial Animation with Style Control Pan, Yifang, Karan Singh, and Luiz Gustavo Hafemann. (2025). "Model See Model Do: Speech-Driven Facial Animation with Style Control." SIGGRAPH Conference Papers 2025. 1-10.
@inproceedings{pan2025msmd,
author = {Pan, Yifang and Singh, Karan and Hafemann, Luiz Gustavo},
title = {Model See Model Do: Speech-Driven Facial Animation with Style Control},
booktitle = {Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers},
series = {SIGGRAPH Conference Papers '25},
year = {2025},
pages = {1--10},
publisher = {ACM},
doi = {10.1145/3721238.3730672}
}
IEEE TVCG, 2025
Head-EyeK: Head-eye Coordination and Control Learned in Virtual Reality
This paper explores head-eye coordination patterns learned through virtual reality interactions for improved animation control and realistic character behavior.
Head-EyeK: Head-eye Coordination and Control Learned in Virtual Reality Pan, Yifang, Ludwig Sidenmark, and Karan Singh. (2025). "Head-EyeK: Head-eye Coordination and Control Learned in Virtual Reality." IEEE Transactions on Visualization and Computer Graphics. 1-11.
@article{pan2025headeyek,
author = {Pan, Yifang and Sidenmark, Ludwig and Singh, Karan},
title = {Head-EyeK: Head-Eye Coordination and Control Learned in Virtual Reality},
journal = {IEEE Transactions on Visualization and Computer Graphics},
year = {2025},
volume = {31},
number = {10},
pages = {9039--9051},
publisher = {IEEE},
doi = {10.1109/TVCG.2025.3589333}
}
ACM SIGGRAPH, 2024
S3: Speech, Script and Scene Driven Head and Eye Animation
This paper presents S3, a multi-modal approach for generating realistic head and eye animations driven by speech, script, and scene context information.
S3: Speech, Script and Scene Driven Head and Eye Animation Pan, Yifang, Rishabh Agrawal, and Karan Singh. (2024). "S3: Speech, Script and Scene Driven Head and Eye Animation." ACM Trans. Graph.. 43(4).
@article{pan2024s3,
author = {Pan, Yifang and Agrawal, Rishabh and Singh, Karan},
title = {S3: Speech, Script and Scene driven Head and Eye Animation},
journal = {ACM Transactions on Graphics},
year = {2024},
volume = {43},
number = {4},
pages = {1--12},
publisher = {ACM},
doi = {10.1145/3658172}
}
ACM SIGGRAPH Asia, 2022
VOCAL: Vowel and Consonant Layering for Expressive Animator-Centric Singing Animation
This paper introduces VOCAL, a novel approach for creating expressive singing animations through vowel and consonant layering techniques designed for animator-centric workflows.
VOCAL: Vowel and Consonant Layering for Expressive Animator-Centric Singing Animation Pan, Yifang, Chris Landreth, Eugene Fiume, and Karan Singh. (2022). "VOCAL: Vowel and Consonant Layering for Expressive Animator-Centric Singing Animation." SIGGRAPH Asia 2022 Conference Papers. 1-9.
@inproceedings{pan2022vocal,
author = {Pan, Yifang and Landreth, Chris and Fiume, Eugene and Singh, Karan},
title = {VOCAL: Vowel and Consonant Layering for Expressive Animator-Centric Singing Animation},
booktitle = {SIGGRAPH Asia 2022 Conference Papers},
series = {SA '22},
year = {2022},
pages = {1--9},
publisher = {ACM},
doi = {10.1145/3550469.3555408}
}