Lens Flare Artifact Removal: Bridging Traditional Methods with Deep Learning Models (Part-3)

FlareNet uses GANs to remove flare from images. GANs have a generator that tries to produce flare-free images and a discriminator that evaluates whether the image is real or generated. This adversarial training helps FlareNet generate high-quality outputs that resemble clean images in real-world environments. The model is trained on synthetically generated flare images, including different flare patterns. The synthetic dataset is created by simulating the real-world scenarios. The generator is based on a U-Net like structure and the discriminator uses convolutional layers to differentiate clean images from corrupted ones. They trained using loss functions like adversarial loss, perceptual loss, and Structural Similarity Index Measure (SSIM) loss to get optimized results.

FlareNet uses GANs to generate very realistic, flare-free images through adversarial training. U-Net-based generator and convolutional discriminator preserve fine details and improve image quality. It performs well on diverse synthetic flare patterns because it has a rich training set. GAN training is unstable and difficult to optimize, and has the tendency to require sensitive balancing between the generator and the discriminator. The model relies heavily on synthetic data, which might limit performance against actual flare variations. It is computationally expensive due to adversarial training. 

Vanilla GANs are not practical for flare removal, in the sense that they offer no input-output control - i.e., there is no guarantee that the output produced is in any meaningful sense related to the input flare-tarred image. This makes them unsuitable to applications that require structural consistency, such as flare correction.

Conditional GANs (cGANs), enable context-aware generation by conditioning both the discriminator and generator on the input image. This is suitable for flare removal, especially in supervised cases with paired data. Techniques such as FlareNet essentially go on to implicitly extend the cGAN approach to learning explicit mappings from flared to clean images, preserving scene content but removing flare artifacts.

CycleGAN is a deep learning model based on unsupervised learning that is effective for removing flare when paired training data is unavailable. It does not require matched clean and flare image pairs. It learns to translate images from flare to clean images by training a generator and a cycle-consistency network. The data comprised two unpaired image pairs, one with flare artifacts and the other with clean images. It was trained through cycle-consistency loss such that an image translated to the clean domain could be reversed to the flare domain without compromising significant details. The power of CycleGAN is that it can generalize across different flare conditions without supervision, and hence it is highly suitable for real-world applications.

CycleGAN works without paired flare and clean images, making it ideal when such data does not exist. Unsupervised learning with cycle-consistency loss allows it to generalize well for various flares and conditions while preserving essential image details. Unsupervised training may lead to less precise removal of flares than from supervised models. The model can also cause color shifts or artifacts since it does not learn pixel-to-pixel mappings explicitly, and training may be unstable or require careful tuning.

TABLE III: Comparative Analysis of GAN-Based Flare Removal Approaches 

As shown in Table III, GAN-based flare removal methods address the core challenge of distinguishing artifacts from true illumination differently. FlareNet (paired) works precisely on synthetic flares but struggles with real-world variations. CycleGAN (unpaired) handles uncontrolled flares but may preserve artifacts or remove valid lights. Light-Guided GAN (cGAN) introduces physics-based light segmentation, enabling selective flare removal while preserving true lighting – advancing from blind cleaning to semantic-aware processing.

MFDNet Applies Laplacian Pyramids for the Parallel Processing of Low-Frequency and High-Frequency Components. MFDNet uses Laplacian Pyramid decomposition, which decomposes images into various frequency bands, enabling independent processing of low-frequency and high-frequency components. Low-frequency components hold global illumination data, whereas high-frequency components separate edge and fine detail. Such isolation allows MFDNet to eliminate flare artifacts without warping image information. The MFDNet dataset contains real images with flare artifacts, which makes it robust to varying lighting conditions. Multi-task learning was utilized to train the model, with different loss functions for low- and high-frequency components. Through reconstructing images from various frequency levels, MFDNet achieved superior performance compared to standard CNNs and transformers. It achieved state-of-the-art results on flare removal benchmarks, retaining both fine textures and global scene coherence.

MFDNet's Laplacian Pyramid decomposition allows it to process the low-frequency and high-frequency components separately, effectively removing flare while preserving global illumination and fine details. Individually tailored loss functions per frequency band using multi-task learning yield reconstruction quality improvements. MFDNet achieves state-of-the-art, outperforming traditional CNNs and transformers for preserving texture and scene coherence. The multi-frequency parallel processing comes at the cost of model complexity and can require more computational resources. It can be difficult to balance losses over various frequency components, and training may be more intensive than for simpler architectures.

Fourier-based techniques utilize Fast Fourier Convolution (FFC) to eliminate flare while maintaining fine image details. FFC is unlike conventional CNNs, which work in the spatial domain and instead transform images into the frequency domain, making separating unwanted artifacts easier. This technique is especially efficient in flare removal since flares will likely possess clear frequency signatures that can be eliminated. The training dataset for Fourier-based models contains real and synthetically produced flare images, with a variety of artifact patterns. Training included spectral loss functions, which punished undesirable frequency components while preserving natural image structures.

Fourier-based methods showed promising results, particularly for high-resolution images, where CNNs and transformers struggled with preserving fine textures. The ability to operate in the frequency domain made these methods computationally efficient and robust to different flare intensities.

MFDNet's use of Laplacian Pyramid Decomposition enables independent processing of the low- and high-frequency components, preventing flares while retaining high details. It outperforms transformer-based models in preserving details and is facilitated by a heterogeneous training set and advanced loss functions like perceptual and adversarial losses. The two-stream architecture has the potential to increase model size and computational cost. It might also be sensitive to tuning for trading off processing among frequency components, and adversarial losses during training may be unstable and harder to optimize.

MPRNet is a combined architecture for image restoration tasks like de-raining, deblurring, and denoising. It employs a multi-stage progressive network comprising encoder-decoder networks and a full-resolution final stage to enhance details. Supervised attention modules and cross-stage feature fusion are optimized features that improve feature learning and context sharing. MPRNet achieves state-of-the-art performance on multiple restoration benchmarks. 

Fourier-based techniques with FFC efficiently separate flare artifacts from the frequency domain, making it possible to remove undesirable patterns without discarding fine details of the image. They are computationally efficient and perform well in high-resolution images where CNNs and transformers are likely to fail, invariant to flare intensity variations. Frequency-domain operationalization is sometimes difficult in the optimal capture of intricacies in spatial relations. It is thus expected to contribute to less-than-optimal elimination of non-discriminant frequency-signature flares directly. Design and learning of spectral loss functions are also likely to be more complex than with spatial-domain approaches.

Flare removal techniques have evolved significantly from traditional hardware-based solutions and classical image processing approaches to advanced deep learning models. Among these, MFDNet has recently emerged as a state-of-the-art method, particularly excelling in the domain of nighttime flare removal. It distinguishes itself through its multi-frequency decomposition strategy, which enables efficient separation and suppression of flare artifacts while preserving essential image details. In Figure 4, nighttime lens flare is illustrated as comprising two main types: scattering and reflective flares. Scattering flares-such as glare, shimmer, and streaks-are typically caused by dust particles, sensor noise, or lens imperfections. Reflective flares result from internal reflections within the lens system and often appear as repeated iris-shaped patterns.

MFDNet was proposed as a lightweight yet high-performance model tailored for nighttime photography applications. It introduces several key innovations:

The architecture of MFDNet integrates frequency-domain decomposition with deep learning-based restoration modules. MFDNet employs a Laplacian Pyramid to decompose the input image into low- and high-frequency components. The low-frequency layer primarily captures global illumination and large-scale flare, while the high-frequency layer retains edges and fine textures. This module uses a combination of CNNs and Transformers to detect and suppress flare patterns in the low-frequency band. It captures long-range dependencies, ensuring robust flare recognition without compromising the overall brightness and tone of the image. As shown in Figure 5, HFRM fuses the processed low-frequency image with the untouched high-frequency details after flare removal. This step ensures structural and textural integrity preservation, avoiding the blurring commonly seen in classical or U-Net-based models.

TABLE IV: Comparative Analysis of Multi-Frequency Flare Removal Approaches 

Table IV compares multi-frequency flare removal approaches with distinct processing strategies. MFDNet uses a Laplacian pyramid for explicit frequency separation, where nighttime reflective flares are its forte. Fourier-based techniques look for periodic degradations using spectral manipulation and MPRNet's hierarchical architecture implicitly addresses multiple degradations through multi-scale processing - illustrating how frequency-conscious designs tackle varying flare behaviors through spatial, spectral, and hybrid methods.

Flare removal has evolved from traditional methods to advanced GAN-based and multi-frequency deep learning approaches, offering improved artifact removal and detail preservation. The next section will discuss the datasets used and the overall performance and evaluation of these approaches.