|
| 融合Transformer和卷积神经网络的全局光照估计 |
| Globally lighting estimation by integrating Transformer and Convolutional Neural Networks |
| 投稿时间：2025-06-05 修订日期：2025-11-14 |
| DOI： |
| 中文关键词: AR应用 光照估计 球面谐波系数 Transformer 卷积神经网络 |
| 英文关键词: Augmented Reality(AR) illumination estimation Spherical Harmonic(SH) coefficient Transformer Convolutional Neural Networks |
| 基金项目:安徽教育厅自然科学项目 |
|
| 摘要点击次数: 45 |
| 全文下载次数: 0 |
| 中文摘要: |
| 增强现实(Augmented Reality,AR)技术的目标是实现虚拟物体与现实世界的视觉一致性。由于视角受限和场景光照复杂多变,现有方法在估计全景照明时往往未能兼顾局部光照一致性。为解决这一问题,提出一种基于Transformer和深度可分离卷积神经网络的全局光照估计模型EViTLight。该模型能够有效提取场景中的低频光照特征,从单张低动态范围(Low Dynamic Range, LDR)且视角受限(Limited Field of View, LFOV)的图像中估算出表示漫反射的球面谐波(Spherical Harmonics, SH)系数,并通过这些系数构建SH函数来近似场景中的低频光照。在公开数据集上与四种经典光照估计方法相比的实验结果表明,EViTLight模型在均方根误差(RMSE)和结构相似度(DSSIM)指标上分别达到了0.0165和0.0119,性能优于其他方法。此外,定量实验和原型系统的开发验证了该模型的有效性和实用性。 |
| 英文摘要: |
| 【Abstract】The goal of Augmented Reality (AR) technology is to achieve visual consistency between virtual objects and the real world. However, due to the limitations of the camera"s field of view and the complexity of lighting conditions in real scenes, developers often face challenges in accurately estimating low-frequency lighting information when recovering panoramic lighting. To address this issue, we propose a lighting estimation model, EViTLight, based on Transformer and Depthwise Separable Convolutional Neural Networks (CNNs). The model effectively extracts low-frequency lighting features from the scene and estimates Spherical Harmonics (SH) coefficients, representing diffuse reflection, from a single Low Dynamic Range (LDR) image with a Limited Field of View (LFOV). The SH coefficients are then used to approximate low-frequency lighting in the scene through the construction of SH functions. Experimental results on public datasets show that, compared with four classical lighting estimation methods, the EViTLight model achieves superior performance, with a Root Mean Square Error (RMSE) of 0.0165 and a Structural Dissimilarity (DSSIM) of 0.0119. Furthermore, both quantitative experiments and the development of a prototype system validate the effectiveness and practicality of the proposed model. |
|
View Fulltext
查看/发表评论 下载PDF阅读器 |
| 关闭 |
|
|
|
