Artwork

תוכן מסופק על ידי Itzik Ben-Shabat. כל תוכן הפודקאסטים כולל פרקים, גרפיקה ותיאורי פודקאסטים מועלים ומסופקים ישירות על ידי Itzik Ben-Shabat או שותף פלטפורמת הפודקאסט שלו. אם אתה מאמין שמישהו משתמש ביצירה שלך המוגנת בזכויות יוצרים ללא רשותך, אתה יכול לעקוב אחר התהליך המתואר כאן https://he.player.fm/legal.
Player FM - אפליקציית פודקאסט
התחל במצב לא מקוון עם האפליקציה Player FM !

Guy Gafni - NerFACE

32:44
 
שתפו
 

Manage episode 319391776 series 3300270
תוכן מסופק על ידי Itzik Ben-Shabat. כל תוכן הפודקאסטים כולל פרקים, גרפיקה ותיאורי פודקאסטים מועלים ומסופקים ישירות על ידי Itzik Ben-Shabat או שותף פלטפורמת הפודקאסט שלו. אם אתה מאמין שמישהו משתמש ביצירה שלך המוגנת בזכויות יוצרים ללא רשותך, אתה יכול לעקוב אחר התהליך המתואר כאן https://he.player.fm/legal.

PAPER TITLE:
Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction
AUTHORS:
Guy Gafni Justus Thies Michael Zollhöfer Matthias Nießner
Project page: https://gafniguy.github.io/4D-Facial-Avatars/
CODE:
💻https://github.com/gafniguy/4D-Facial-Avatars
ABSTRACT:
We present dynamic neural radiance fields for modeling the appearance and dynamics of a human face. Digitally modeling and reconstructing a talking human is a key building-block for a variety of applications. Especially, for telepresence applications in AR or VR, a faithful reproduction of the appearance including novel viewpoint or head-poses is required. In contrast to state-of-the-art approaches that model the geometry and material properties explicitly, or are purely image-based, we introduce an implicit representation of the head based on scene representation networks. To handle the dynamics of the face, we combine our scene representation network with a low-dimensional morphable model which provides explicit control over pose and expressions. We use volumetric rendering to generate images from this hybrid representation and demonstrate that such a dynamic neural scene representation can be learned from monocular input data only, without the need of a specialized capture setup. In our experiments, we show that this learned volumetric representation allows for photo-realistic image generation that surpasses the quality of state-of-the-art video-based reenactment methods.
RELATED PAPERS:
📚Representing Scenes as Neural Radiance Fields for View Synthesis
📚Deep Video Portraits
📚Nerfies: Deformable Neural Radiance Fields
📚AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis
CONTACT:
-----------------
If you would like to be a guest, sponsor or just share your thoughts, feel free to reach out via email: talking.papers.podcast@gmail.com
TIME STAMPS
-----------------------
00:00
00:07 Intro
00:27 Authors
01:16 Abstract / TLDR
02:54 Motivation
12:24 Related Work
13:20 Approach
17:10 Results
27:05 Conclusions and future work
32:12 Outro
#talkingpapers #CVPR2021 #NeRF
#machinelearning #deeplearning #AI #neuralnetworks #research #computervision #artificialintelligence #FacialAvatars
Recorded on April, 2nd 2021.
SUBSCRIBE AND FOLLOW:
🎧Subscribe on your favourite podcast app: https://talking.papers.podcast.itzikbs.com
📧Subscribe to our mailing list: http://eepurl.com/hRznqb
🐦Follow us on Twitter: https://twitter.com/talking_papers
🎥YouTube Channel: https://bit.ly/3eQOgwP
🎧Subscribe on your favourite podcast app: https://talking.papers.podcast.itzikbs.com

📧Subscribe to our mailing list: http://eepurl.com/hRznqb

🐦Follow us on Twitter: https://twitter.com/talking_papers

🎥YouTube Channel: https://bit.ly/3eQOgwP

  continue reading

32 פרקים

Artwork

Guy Gafni - NerFACE

Talking Papers Podcast

0-10 subscribers

published

iconשתפו
 
Manage episode 319391776 series 3300270
תוכן מסופק על ידי Itzik Ben-Shabat. כל תוכן הפודקאסטים כולל פרקים, גרפיקה ותיאורי פודקאסטים מועלים ומסופקים ישירות על ידי Itzik Ben-Shabat או שותף פלטפורמת הפודקאסט שלו. אם אתה מאמין שמישהו משתמש ביצירה שלך המוגנת בזכויות יוצרים ללא רשותך, אתה יכול לעקוב אחר התהליך המתואר כאן https://he.player.fm/legal.

PAPER TITLE:
Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction
AUTHORS:
Guy Gafni Justus Thies Michael Zollhöfer Matthias Nießner
Project page: https://gafniguy.github.io/4D-Facial-Avatars/
CODE:
💻https://github.com/gafniguy/4D-Facial-Avatars
ABSTRACT:
We present dynamic neural radiance fields for modeling the appearance and dynamics of a human face. Digitally modeling and reconstructing a talking human is a key building-block for a variety of applications. Especially, for telepresence applications in AR or VR, a faithful reproduction of the appearance including novel viewpoint or head-poses is required. In contrast to state-of-the-art approaches that model the geometry and material properties explicitly, or are purely image-based, we introduce an implicit representation of the head based on scene representation networks. To handle the dynamics of the face, we combine our scene representation network with a low-dimensional morphable model which provides explicit control over pose and expressions. We use volumetric rendering to generate images from this hybrid representation and demonstrate that such a dynamic neural scene representation can be learned from monocular input data only, without the need of a specialized capture setup. In our experiments, we show that this learned volumetric representation allows for photo-realistic image generation that surpasses the quality of state-of-the-art video-based reenactment methods.
RELATED PAPERS:
📚Representing Scenes as Neural Radiance Fields for View Synthesis
📚Deep Video Portraits
📚Nerfies: Deformable Neural Radiance Fields
📚AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis
CONTACT:
-----------------
If you would like to be a guest, sponsor or just share your thoughts, feel free to reach out via email: talking.papers.podcast@gmail.com
TIME STAMPS
-----------------------
00:00
00:07 Intro
00:27 Authors
01:16 Abstract / TLDR
02:54 Motivation
12:24 Related Work
13:20 Approach
17:10 Results
27:05 Conclusions and future work
32:12 Outro
#talkingpapers #CVPR2021 #NeRF
#machinelearning #deeplearning #AI #neuralnetworks #research #computervision #artificialintelligence #FacialAvatars
Recorded on April, 2nd 2021.
SUBSCRIBE AND FOLLOW:
🎧Subscribe on your favourite podcast app: https://talking.papers.podcast.itzikbs.com
📧Subscribe to our mailing list: http://eepurl.com/hRznqb
🐦Follow us on Twitter: https://twitter.com/talking_papers
🎥YouTube Channel: https://bit.ly/3eQOgwP
🎧Subscribe on your favourite podcast app: https://talking.papers.podcast.itzikbs.com

📧Subscribe to our mailing list: http://eepurl.com/hRznqb

🐦Follow us on Twitter: https://twitter.com/talking_papers

🎥YouTube Channel: https://bit.ly/3eQOgwP

  continue reading

32 פרקים

כל הפרקים

×
 
Loading …

ברוכים הבאים אל Player FM!

Player FM סורק את האינטרנט עבור פודקאסטים באיכות גבוהה בשבילכם כדי שתהנו מהם כרגע. זה יישום הפודקאסט הטוב ביותר והוא עובד על אנדרואיד, iPhone ואינטרנט. הירשמו לסנכרון מנויים במכשירים שונים.

 

מדריך עזר מהיר