High dynamic range imaging (HDRI) for real-world dynamic scenes is challenging because moving objects may lead to hybrid degradation of low dynamic range and motion blur. Existing event-based approaches only focus on a separate task, while cascading HDRI and motion deblurring would lead to sub-optimal solutions, and unavailable ground-truth sharp HDR images aggravate the predicament. To address these challenges, we propose an event-based HDRI framework within a self-supervised learning paradigm, i.e., Self-EHDRI, which generalizes the performance of HDRI in real-world dynamic scenarios. Specifically, a self-supervised learning strategy is carried out by learning cross-domain conversions from blurry LDR images to sharp LDR images, which enables sharp HDR images to be accessible in the intermediate process even though ground-truth sharp HDR images are missing. Then, we formulate the event-based HDRI and motion deblurring model and conduct a unified network to recover the intermediate sharp HDR results, where both the high temporal resolution and high dynamic range of events are leveraged simultaneously for compensation. We construct large-scale synthetic and real-world datasets to evaluate the effectiveness of our method. Comprehensive experiments demonstrate that the proposed Self-EHDRI outperforms state-of-the-art approaches by a large margin.
Since existing state-of-the-art HDRI methods and deblurring methods cannot deal with blurry LDR images, we compare our proposed Self-EHDRI with the methods which combine HDRI with event-based deblurring. Two experiments are conducted for cascading approaches, HDRI+Deblurring as cascading HDRev and eSL-Net++ by implementing HDRI as the pre-operation for deblurring, and Deblurring+HDRI as cascading eSL-Net++ and HDRev by implementing deblurring as the pre-operation for HDRI. The upper portion of the image displays the sharp HDR reconstructed effect when the mouse is hovered over it, while the lower portion shows the original input blurry LDR image. This allows for a clearer comparison of the correction effect.
Our method can generate continuous-time sharp HDR reconstructions at arbitrary time from the blurry LDR image and events. We compare our Self-EHDRI with HDRI+Deblurring as cascading HDRev and eSL-Net++ as well as Deblurring+HDRI as cascading eSL-Net++ and HDRev on 15X (1st row), 45X (2nd row), 90X (3rd row) reconstructions (nX represents the reconstructions n sharp HDR frames from one blurry LDR image).
@misc{xiaopeng2024hdr,
title={HDR Imaging for Dynamic Scenes with Events},
author={Li Xiaopeng and Zeng Zhaoyuan and Fan Cien and Zhao Chen and Deng Lei and Yu Lei},
year={2024},
eprint={2404.03210},
archivePrefix={arXiv},
primaryClass={cs.CV}
}