BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment

Kelvin C.K. Chan; Shangchen Zhou; Xiangyu Xu; Chen Change Loy

doi:10.1109/CVPR52688.2022.00588

BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment

Kelvin C.K. Chan, Shangchen Zhou, Xiangyu Xu, Chen Change Loy^*

^*Corresponding author for this work

Nanyang Technological University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

396 Citations (Scopus)

Abstract

A recurrent structure is a popular framework choice for the task of video super-resolution. The state-of-the-art method BasicVSR adopts bidirectional propagation with feature alignment to effectively exploit information from the entire input video. In this study, we redesign BasicVsr by proposing second-order grid propagation and flow-guided deformable alignment. We show that by empowering the re-current framework with enhanced propagation and align-ment, one can exploit spatiotemporal information across misaligned video frames more effectively. The new components lead to an improved performance under a simi-lar computational constraint. In particular, our model Ba-sicVSR++ surpasses BasicVSR by a significant 0.82 dB in PSNR with similar number of parameters. BasicVSR++ is generalizable to other video restoration tasks, and obtains three champions and one first runner-up in NTIRE 2021 video restoration challenge.

Original language	English
Title of host publication	Proceedings - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022
Publisher	IEEE Computer Society
Pages	5962-5971
Number of pages	10
ISBN (Electronic)	9781665469463
DOIs	https://doi.org/10.1109/CVPR52688.2022.00588
Publication status	Published - 2022
Externally published	Yes
Event	2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022 - New Orleans, United States Duration: Jun 19 2022 → Jun 24 2022

Publication series

Name	Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Volume	2022-June
ISSN (Print)	1063-6919

Conference

Conference	2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022
Country/Territory	United States
City	New Orleans
Period	6/19/22 → 6/24/22

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

ASJC Scopus Subject Areas

Software
Computer Vision and Pattern Recognition

Keywords

Low-level vision

Access to Document

10.1109/CVPR52688.2022.00588

Cite this

Chan, K. C. K., Zhou, S., Xu, X., & Loy, C. C. (2022). BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment. In Proceedings - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022 (pp. 5962-5971). (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition; Vol. 2022-June). IEEE Computer Society. https://doi.org/10.1109/CVPR52688.2022.00588

Chan, Kelvin C.K. ; Zhou, Shangchen ; Xu, Xiangyu et al. / BasicVSR++ : Improving Video Super-Resolution with Enhanced Propagation and Alignment. Proceedings - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022. IEEE Computer Society, 2022. pp. 5962-5971 (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition).

@inproceedings{ad8339aa667c4c1aafc19bd769dbd938,

title = "BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment",

abstract = "A recurrent structure is a popular framework choice for the task of video super-resolution. The state-of-the-art method BasicVSR adopts bidirectional propagation with feature alignment to effectively exploit information from the entire input video. In this study, we redesign BasicVsr by proposing second-order grid propagation and flow-guided deformable alignment. We show that by empowering the re-current framework with enhanced propagation and align-ment, one can exploit spatiotemporal information across misaligned video frames more effectively. The new components lead to an improved performance under a simi-lar computational constraint. In particular, our model Ba-sicVSR++ surpasses BasicVSR by a significant 0.82 dB in PSNR with similar number of parameters. BasicVSR++ is generalizable to other video restoration tasks, and obtains three champions and one first runner-up in NTIRE 2021 video restoration challenge.",

keywords = "Low-level vision",

author = "Chan, \{Kelvin C.K.\} and Shangchen Zhou and Xiangyu Xu and Loy, \{Chen Change\}",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022 ; Conference date: 19-06-2022 Through 24-06-2022",

year = "2022",

doi = "10.1109/CVPR52688.2022.00588",

language = "English",

series = "Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition",

publisher = "IEEE Computer Society",

pages = "5962--5971",

booktitle = "Proceedings - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022",

address = "United States",

}

Chan, KCK, Zhou, S, Xu, X & Loy, CC 2022, BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment. in Proceedings - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2022-June, IEEE Computer Society, pp. 5962-5971, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, United States, 6/19/22. https://doi.org/10.1109/CVPR52688.2022.00588

BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment. / Chan, Kelvin C.K.; Zhou, Shangchen; Xu, Xiangyu et al.
Proceedings - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022. IEEE Computer Society, 2022. p. 5962-5971 (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition; Vol. 2022-June).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - BasicVSR++

T2 - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022

AU - Chan, Kelvin C.K.

AU - Zhou, Shangchen

AU - Xu, Xiangyu

AU - Loy, Chen Change

PY - 2022

Y1 - 2022

N2 - A recurrent structure is a popular framework choice for the task of video super-resolution. The state-of-the-art method BasicVSR adopts bidirectional propagation with feature alignment to effectively exploit information from the entire input video. In this study, we redesign BasicVsr by proposing second-order grid propagation and flow-guided deformable alignment. We show that by empowering the re-current framework with enhanced propagation and align-ment, one can exploit spatiotemporal information across misaligned video frames more effectively. The new components lead to an improved performance under a simi-lar computational constraint. In particular, our model Ba-sicVSR++ surpasses BasicVSR by a significant 0.82 dB in PSNR with similar number of parameters. BasicVSR++ is generalizable to other video restoration tasks, and obtains three champions and one first runner-up in NTIRE 2021 video restoration challenge.

AB - A recurrent structure is a popular framework choice for the task of video super-resolution. The state-of-the-art method BasicVSR adopts bidirectional propagation with feature alignment to effectively exploit information from the entire input video. In this study, we redesign BasicVsr by proposing second-order grid propagation and flow-guided deformable alignment. We show that by empowering the re-current framework with enhanced propagation and align-ment, one can exploit spatiotemporal information across misaligned video frames more effectively. The new components lead to an improved performance under a simi-lar computational constraint. In particular, our model Ba-sicVSR++ surpasses BasicVSR by a significant 0.82 dB in PSNR with similar number of parameters. BasicVSR++ is generalizable to other video restoration tasks, and obtains three champions and one first runner-up in NTIRE 2021 video restoration challenge.

KW - Low-level vision

UR - http://www.scopus.com/inward/record.url?scp=85141696791&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85141696791&partnerID=8YFLogxK

U2 - 10.1109/CVPR52688.2022.00588

DO - 10.1109/CVPR52688.2022.00588

M3 - Conference contribution

AN - SCOPUS:85141696791

T3 - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

SP - 5962

EP - 5971

BT - Proceedings - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022

PB - IEEE Computer Society

Y2 - 19 June 2022 through 24 June 2022

ER -

Chan KCK, Zhou S, Xu X, Loy CC. BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment. In Proceedings - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022. IEEE Computer Society. 2022. p. 5962-5971. (Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition). doi: 10.1109/CVPR52688.2022.00588

BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment

Abstract

Publication series

Conference

Bibliographical note

ASJC Scopus Subject Areas

Keywords

Access to Document

Other files and links

Cite this