LOC3DIFF: Local Diffusion for 3D Human Head Synthesis and Editing

Yushi Lan; Feitong Tan; Qiangeng Xu; Di Qiu; Kyle Genova; Zeng Huang; Sean Fanello; Rohit Pandey; Thomas Funkhouser; Chen Change Loy; Yinda Zhang

doi:10.1007/978-3-031-73650-6_4

LOC3DIFF: Local Diffusion for 3D Human Head Synthesis and Editing

Yushi Lan^*, Feitong Tan, Qiangeng Xu, Di Qiu, Kyle Genova, Zeng Huang, Sean Fanello, Rohit Pandey, Thomas Funkhouser, Chen Change Loy, Yinda Zhang

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We present a novel framework for generating photorealistic 3D human head and subsequently manipulating and reposing them with remarkable flexibility. The proposed approach constructs an implicit representation of 3D human heads, anchored on a parametric face model. To enhance representational capabilities and encode spatial information, we represent semantic consistent head region by a local tri-plane, modulated by a 3D Gaussian. Additionally, we parameterize these tri-planes in a 2D UV space via a 3DMM, enabling effective utilization of the diffusion model for 3D head avatar generation. Our method facilitates the creation of diverse and realistic 3D human heads with flexible global and fine-grained region-based editing over facial structures, appearance and expressions. Extensive experiments demonstrate the effectiveness of our method.

Original language	English
Title of host publication	Computer Vision – ECCV 2024 - 18th European Conference, Proceedings
Editors	Aleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, Gül Varol
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	49-66
Number of pages	18
ISBN (Print)	9783031736490
DOIs	https://doi.org/10.1007/978-3-031-73650-6_4
Publication status	Published - 2025
Externally published	Yes
Event	18th European Conference on Computer Vision, ECCV 2024 - Milan, Italy Duration: Sept 29 2024 → Oct 4 2024

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	15123 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	18th European Conference on Computer Vision, ECCV 2024
Country/Territory	Italy
City	Milan
Period	9/29/24 → 10/4/24

Bibliographical note

Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

ASJC Scopus Subject Areas

Theoretical Computer Science
General Computer Science

Keywords

3D Editing
3D Generation
3D Head

Access to Document

10.1007/978-3-031-73650-6_4

Cite this

Lan, Y., Tan, F., Xu, Q., Qiu, D., Genova, K., Huang, Z., Fanello, S., Pandey, R., Funkhouser, T., Loy, C. C., & Zhang, Y. (2025). LOC3DIFF: Local Diffusion for 3D Human Head Synthesis and Editing. In A. Leonardis, E. Ricci, S. Roth, O. Russakovsky, T. Sattler, & G. Varol (Eds.), Computer Vision – ECCV 2024 - 18th European Conference, Proceedings (pp. 49-66). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 15123 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-73650-6_4

Lan, Yushi ; Tan, Feitong ; Xu, Qiangeng et al. / LOC3DIFF : Local Diffusion for 3D Human Head Synthesis and Editing. Computer Vision – ECCV 2024 - 18th European Conference, Proceedings. editor / Aleš Leonardis ; Elisa Ricci ; Stefan Roth ; Olga Russakovsky ; Torsten Sattler ; Gül Varol. Springer Science and Business Media Deutschland GmbH, 2025. pp. 49-66 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{9ab8aa2231584c32ab4848234a6adef6,

title = "LOC3DIFF: Local Diffusion for 3D Human Head Synthesis and Editing",

abstract = "We present a novel framework for generating photorealistic 3D human head and subsequently manipulating and reposing them with remarkable flexibility. The proposed approach constructs an implicit representation of 3D human heads, anchored on a parametric face model. To enhance representational capabilities and encode spatial information, we represent semantic consistent head region by a local tri-plane, modulated by a 3D Gaussian. Additionally, we parameterize these tri-planes in a 2D UV space via a 3DMM, enabling effective utilization of the diffusion model for 3D head avatar generation. Our method facilitates the creation of diverse and realistic 3D human heads with flexible global and fine-grained region-based editing over facial structures, appearance and expressions. Extensive experiments demonstrate the effectiveness of our method.",

keywords = "3D Editing, 3D Generation, 3D Head",

author = "Yushi Lan and Feitong Tan and Qiangeng Xu and Di Qiu and Kyle Genova and Zeng Huang and Sean Fanello and Rohit Pandey and Thomas Funkhouser and Loy, \{Chen Change\} and Yinda Zhang",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.; 18th European Conference on Computer Vision, ECCV 2024 ; Conference date: 29-09-2024 Through 04-10-2024",

year = "2025",

doi = "10.1007/978-3-031-73650-6\_4",

language = "English",

isbn = "9783031736490",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "49--66",

editor = "Ale{\v s} Leonardis and Elisa Ricci and Stefan Roth and Olga Russakovsky and Torsten Sattler and G{\"u}l Varol",

booktitle = "Computer Vision – ECCV 2024 - 18th European Conference, Proceedings",

address = "Germany",

}

Lan, Y, Tan, F, Xu, Q, Qiu, D, Genova, K, Huang, Z, Fanello, S, Pandey, R, Funkhouser, T, Loy, CC & Zhang, Y 2025, LOC3DIFF: Local Diffusion for 3D Human Head Synthesis and Editing. in A Leonardis, E Ricci, S Roth, O Russakovsky, T Sattler & G Varol (eds), Computer Vision – ECCV 2024 - 18th European Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 15123 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 49-66, 18th European Conference on Computer Vision, ECCV 2024, Milan, Italy, 9/29/24. https://doi.org/10.1007/978-3-031-73650-6_4

LOC3DIFF: Local Diffusion for 3D Human Head Synthesis and Editing. / Lan, Yushi; Tan, Feitong; Xu, Qiangeng et al.
Computer Vision – ECCV 2024 - 18th European Conference, Proceedings. ed. / Aleš Leonardis; Elisa Ricci; Stefan Roth; Olga Russakovsky; Torsten Sattler; Gül Varol. Springer Science and Business Media Deutschland GmbH, 2025. p. 49-66 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 15123 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - LOC3DIFF

T2 - 18th European Conference on Computer Vision, ECCV 2024

AU - Lan, Yushi

AU - Tan, Feitong

AU - Xu, Qiangeng

AU - Qiu, Di

AU - Genova, Kyle

AU - Huang, Zeng

AU - Fanello, Sean

AU - Pandey, Rohit

AU - Funkhouser, Thomas

AU - Loy, Chen Change

AU - Zhang, Yinda

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

PY - 2025

Y1 - 2025

N2 - We present a novel framework for generating photorealistic 3D human head and subsequently manipulating and reposing them with remarkable flexibility. The proposed approach constructs an implicit representation of 3D human heads, anchored on a parametric face model. To enhance representational capabilities and encode spatial information, we represent semantic consistent head region by a local tri-plane, modulated by a 3D Gaussian. Additionally, we parameterize these tri-planes in a 2D UV space via a 3DMM, enabling effective utilization of the diffusion model for 3D head avatar generation. Our method facilitates the creation of diverse and realistic 3D human heads with flexible global and fine-grained region-based editing over facial structures, appearance and expressions. Extensive experiments demonstrate the effectiveness of our method.

AB - We present a novel framework for generating photorealistic 3D human head and subsequently manipulating and reposing them with remarkable flexibility. The proposed approach constructs an implicit representation of 3D human heads, anchored on a parametric face model. To enhance representational capabilities and encode spatial information, we represent semantic consistent head region by a local tri-plane, modulated by a 3D Gaussian. Additionally, we parameterize these tri-planes in a 2D UV space via a 3DMM, enabling effective utilization of the diffusion model for 3D head avatar generation. Our method facilitates the creation of diverse and realistic 3D human heads with flexible global and fine-grained region-based editing over facial structures, appearance and expressions. Extensive experiments demonstrate the effectiveness of our method.

KW - 3D Editing

KW - 3D Generation

KW - 3D Head

UR - http://www.scopus.com/inward/record.url?scp=85210491047&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85210491047&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-73650-6_4

DO - 10.1007/978-3-031-73650-6_4

M3 - Conference contribution

AN - SCOPUS:85210491047

SN - 9783031736490

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 49

EP - 66

BT - Computer Vision – ECCV 2024 - 18th European Conference, Proceedings

A2 - Leonardis, Aleš

A2 - Ricci, Elisa

A2 - Roth, Stefan

A2 - Russakovsky, Olga

A2 - Sattler, Torsten

A2 - Varol, Gül

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 29 September 2024 through 4 October 2024

ER -

Lan Y, Tan F, Xu Q, Qiu D, Genova K, Huang Z et al. LOC3DIFF: Local Diffusion for 3D Human Head Synthesis and Editing. In Leonardis A, Ricci E, Roth S, Russakovsky O, Sattler T, Varol G, editors, Computer Vision – ECCV 2024 - 18th European Conference, Proceedings. Springer Science and Business Media Deutschland GmbH. 2025. p. 49-66. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-73650-6_4