AWS Builder Center

Lens Flare Artifact Removal: Bridging Traditional Methods with Deep Learning Models (Part-4)

This study comprehensively examines how flare removal techniques have progressed over time. We start with the basics, like hardware fixes, think anti-reflective coatings and lens hoods, and move on to older computational tricks like PSF-based deconvolution and image inpainting. From there, we get into the newer world of deep learning models, including U-Net, Generative Adversarial Networks (GANs), and Transformer-based architectures, which are pushing the envelope in automatically reducing flare effects.

Series: Lens Flare Artifact Removal: Bridging Traditional Methods with Deep Learning Models (4 articles)

  1. …
  2. 4
    Lens Flare Artifact Removal: Bridging Traditional Methods with Deep Learning Models (Part-4) This article
Pavuluri
DATASETS IN FLARE REMOVAL 
The dataset by comprises a total of 5,001 images, out of which 3,000 are synthetic and 2,001 are real-world captures. The dataset includes a basic variety of light sources but does not explicitly cover reflective flares, streaks, or glare components. It was one of the earlier contributions used primarily for daytime flare removal training tasks. In contrast, the Flare7K dataset comprises 7,000 synthetic images and was developed to support nighttime flare removal. This dataset is remarkably comprehensive, including a wide range of flare artifacts such as reflective flare, streaks, and glare. It offers greater variability with 25 different light source positions and 10 types of flare patterns, making it well-suited for training deep learning models to handle diverse and complex flare conditions. With data augmentation and testing on PSNR and SSIM basis, their approach showed clear improvements over traditional methods.
According to Table VII. Although smaller with only 962 images, the Flare-R dataset comprises entirely real-world images. It includes 962 unique light source positions and nine flare types, making it valuable for evaluating real-world generalization. However, it lacks examples of reflective flares, streaks, and glare, which limits its use for models requiring exposure to these specific artifacts during training. This comparative analysis underscores the trade-off between realism and variety: while synthetic datasets like Flare7K provide controlled diversity, real datasets like Flare-R better represent actual image acquisition conditions. An ideal flare removal model benefits from exposure to both synthetic diversity and real-world authenticity during training and evaluation.

COMPARISON OF FLARE REMOVAL METHODS
The tables below compare datasets, methods, and performance metrics such as PSNR, SSIM, and LPIPS.
TABLE V: Comparison of Flare Removal Datasets 
Dataset NameApproachesType SizeFlare Types CoveredReal vs. Synthetic Strengths Limitations 
Custom Semi-Synthetic DatasetU-Net (daytime), FlareNet Semi-Synthetic5,001 images Various flare artifacts Mixed (Real + Synthetic)Models' real-world opticsSynthetic flares may not match real-world diversity
Flare7KMFDNet, Sparse-UFormer, Baseline U-NetSynthetic 7,000 imagesScattering & reflective flaresSynthetic Large-scale dataset for nighttime flare removalLacks real captured flare images 
Flare7K++MFDNet, UFormer, MPRNetSynthetic 7,000 imagesNighttime flare artifactsSynthetic Well-structured for training deep models Limited real-world validation 
Custom DatasetSparse-UFormerReal & Synthetic UnknownLens flare patternsMixedIntroduces Sparse-UFormer Dataset details not fully disclosed
Custom Veiling Glare DatasetClassical DeconvolutionReal SmallVeiling glare only Real Focuses on glare removalLimited dataset size
Pretrained Image Processing Transformer DatasetFF-Former, UformerLarge-Scale Millions of imagesMultiple image processing tasksReal Versatile for multiple tasksNot specifically optimized for flare removal
Flare7K++MFDNet, RestormerReal + Synthetic Expanded Flare7KMore diverse nighttime flare cases MixedImproved real-world generalizationStill contains synthetic elements 

TABLE VI: Comparison of some of the features of the Datasets used in Flare removal
Dataset Number SyntheticReal Type (s+r)Light SourceReflective FlareStreakGlare
Semi-Synthetic Dataset5001300020012+1NNNN
Flare 7K70007000025+10YYYY
Flare -R9620962962+9YNNN

Table VI presents a comparative summary of three prominent datasets commonly used in lens flare removal research: the dataset introduced by Semi-Synthetic Dataset, Flare7K, and Flare-R. These datasets vary in terms of their size, composition (synthetic vs. real images), and the types of flare artifacts included. (N: No, Y: Yes) Semi-Synthetic Dataset, Flare7K, and Flare-R were selected due to their prevalence in literature and coverage of synthetic/real-world flare types. Other datasets lacked public availability or standardized benchmarks.**** 

TABLE VII: COMPARISON OF FLARE REMOVAL APPROACHES ON FLARE7K DATASET
Method UsedNetwork Type Key TechniquesStrengths Limitations 
Baseline ModelCNN-basedSynthetic flare generation + supervised learning Establishes a strong baseline Limited to synthetic data 
MFDNet Dual-Branch CNN -basedMulti-frequency decomposition (Laplacian Pyramid) Efficient separation of flare components Struggles with mixed-frequency flares 
Sparse-UFormerTransformer-based Sparse attention + U-shaped TransformerBalances computational efficiency with strong learningSparse attention may miss subtle flare components
Hybrid ModelCNN + TransformerReal + synthetic training + adaptive learning Improved real-world generalizationStill contains synthetic elements

PERFORMANCE AND EVALUATION
To determine how well flare removal models work, we look at both objective performance scores and how their architectures compare. In this section, we'll dig into how recent flare removal methods perform, primarily focusing on MFDNet. We will see how it stacks up against other well-known options like Uformer, MPRNet, and Restormer. Table V highlights key attributes such as dataset type, size, flare types covered, and real vs. synthetic composition. While synthetic datasets like Flare7K provide large-scale training data, real or mixed datasets offer better generalization to real-world flare conditions. Limitations include a lack of real diversity, small size, or incomplete documentation. We test these models on both synthetic data and real-world images to see how well they generalize and stay reliable across different scenarios.**** 

QUANTITATIVE PERFORMANCE ANALYSIS
When we're trying to see how well flare removal algorithms perform, as shown in Table VIII, we usually look at three main evaluation metrics that everyone agrees on:**** 
  1. PSNR: This checks how close the reconstructed image is to the original. So, a higher PSNR means the image is restored more accurately.**** 
  2. SSIM: This focuses on how similar the images look to our eyes, especially in terms of structure and detail. The higher the SSIM, the better it is at keeping that intact. 
  3. LPIPS: This one is about perception, based on how deep neural networks see images. Lower LPIPS values mean the processed image looks more like the original.

TABLE VIII: COMPARISON OF FLARE REMOVAL METHODS THAT USE FLARE7K++ DATASET
Method Real-World PSNR (dB)Synthetic PSNR (dB) SSIM (x30) LPIPS (x500) 
Uformer 26.2829.450.883 0.054
MPRNet26.7429.970.8820.048
Restormer26.1429.870.8780.050
MFDNet26.9830.790.8950.019

Figure 6. Normalized Performance Comparison of Image Restoration Methods.

Models like Uformer and Restormer, both based on transformer tech, get pretty good PSNR and SSIM scores, especially on synthetic data. Still, they’re a little behind when it comes to making the images look totally natural, as shown by their higher LPIPS scores. According to Figure 6, Higher SSIM and lower LPIPS reflect better visual quality and similarity to ground truth. The chart highlights distinct structural and perceptual restoration strengths across the tested approaches. Then there's MPRNet [26], which uses a multi-stage CNN, giving a decent overall performance that balances complexity and general performance. But still, MFDNet beats it in every category.
As shown in Figure 7, Darker shades represent better scores, enabling quick visual comparison. The chart highlights MFDNet's consistently strong performance across PSNR and SSIM, with clear distinctions among methods in both synthetic and real-world conditions. Lighter shades are preferable for LPIPS, where lower values indicate better perceptual quality. All in all, these numbers really show how MFDNet shines when it comes to reducing glare, keeping detail, and restoring the overall look of real-world images.

COMPARATIVE ARCHITECTURAL ANALYSIS
To see how MFDNet stacks up, let's compare it to older flare removal methods. Table IX gives a quick rundown of how traditional models, U-Net-based approaches, Transformer-based techniques, and frequency-decomposed models perform across four key areas: how well they remove flares, how well they keep details intact, their computational demands, and how easily they can be scaled up or adapted.**** 

Figure 7. Heatmap visualization of flare removal performance across multiple metrics and methods.

TABLE IX: COMPARISON OF FLARE REMOVAL APPROACHES
FeaturesClassical Methods U-Net-Based Models Transformer-Based Models Multi-Frequency Processing 
Flare Removal Efficiency  Low Moderate High Very High  
Detail PreservationBlurring issues  Some loss of detailsGood Excellent (Retains fine details)
Computational Cost Low Moderate HighOptimized for real-time applications 
Scalability to High-Resolution Images Limited Moderate Good Supports 4K resolution 

MFDNet’s Superior Attributes: 
  1. Precision in Flare Removal: Thanks to its dual-path approach that processes both low- and high-frequency parts with Laplacian Pyramid decomposition, MFDNet can more precisely identify and zap out flare artifacts compared to pixel-based or patch-based techniques. 
  2. Preservation of Texture and Colour: MFDNet avoids over smoothing and color issues, unlike older models or U-Net-based ones. This is because of its HFRM, which helps preserve fine details and keeps natural color transitions looking good. 
  3. Computational Efficiency: While Transformer models tend to need a lot of power and memory, MFDNet gets comparable or even better results with fewer parameters. That means it's more practical for real-time use or working with high-res images. 
  4. Application-Specific Optimization: MFDNet is designed especially for night-time flare removal, which is often trickier because of artificial lighting effects. This focus makes it more effective for real-world jobs like surveillance, self-driving cars, or mobile photography. 
Overall, MFDNet offers a more balanced and efficient approach to flare removal, especially in situations that demand quick and reliable performance.

Future Research Directions 
Although substantial progress has been made in flare removal, current solutions face significant limitations when applied to real-world scenarios. This section highlights ongoing challenges and outlines key directions for future research aimed at enhancing both model robustness and practical deployment.
  1. Real-World Generalization: Most current models are trained on synthetic or semi-synthetic datasets that fail to capture the full diversity of flare patterns encountered in natural environments. As a result, these models struggle to generalize across different lighting conditions, camera types, and optical configurations. Future work should focus on developing domain adaptation strategies and training across heterogeneous datasets to improve generalization. 
  2. Handling Extreme Flare Conditions: Despite advances in architecture design, flare removal under severe conditions—such as intense veiling glare or complex reflective artifacts—remains a challenge. Hybrid frameworks that combine deep learning with physics-based modeling could help reconstruct high-fidelity outputs in these edge cases. 
  3. Color and Detail Preservation: Transformer-based models, while effective in modeling global features, may sometimes over-smooth textures or distort color tones. Future approaches should explore improved perceptual loss functions, attention mechanisms, and CNN-transformer hybrid architectures to ensure better texture and color retention. 
  4. Real-Time Optimization: Deploying flare removal models in real-time applications such as autonomous vehicles, mobile phones, or AR/VR systems is limited by computational complexity. Techniques such as model pruning, knowledge distillation, and sparse attention should be investigated to develop lightweight and efficient architectures capable of operating under strict latency constraints. 
  5. Dataset Benchmarking and Expansion: The reliance on synthetic datasets has led to models that perform well in controlled settings but poorly on real-world data. The community must prioritize building large-scale, diverse, and publicly available real-world flare datasets to enable consistent benchmarking and meaningful evaluation. 
  6. Differentiating Flares from Valid Light Sources: An ongoing issue in flare removal is the misclassification of genuine light sources (e.g., headlights or street lamps) as artifacts. Integrating multi-modal data—such as depth or semantic segmentation—can aid in distinguishing between intended lighting and flare. 
  7. Cross-Device and Cross-Domain Adaptation: Most flare removal models are optimized for specific sensors or lenses and do not generalize across devices. Incorporating meta-learning and style-transfer techniques could allow models to adapt across a broader spectrum of imaging conditions without requiring retraining. 
  8. AR/VR Applications: As augmented and virtual reality platforms gain popularity, the importance of real-time flare suppression in immersive environments becomes critical. Adapting flare removal pipelines to support stereoscopic rendering and 3D spatial awareness will help enhance user immersion and visual consistency in AR/VR systems. 

CONCLUSION
Flare removal techniques have advanced significantly with deep learning, supported by diverse datasets and improved evaluation methods. Among the compared approaches, MFDNet demonstrates strong performance in flare removal, detail preservation, and computational efficiency. However, challenges such as real-world generalization, extreme flare conditions, and real-time deployment remain, highlighting the need for more robust, adaptable, and efficient models.

Series: Lens Flare Artifact Removal: Bridging Traditional Methods with Deep Learning Models (4 articles)

  1. …
  2. 4
    Lens Flare Artifact Removal: Bridging Traditional Methods with Deep Learning Models (Part-4) This article
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article