proposal

1. Proposal Transformer-Based Reflection Removal from Images

Krupoderov Egor
2026

2. Problem Statement

• Problem: Unwanted reflections and glare in images captured through
transparent surfaces
• Consequences:
• Distorted scene perception
• Reduced visual quality
• Impaired performance of object recognition algorithms
• Research Objective: Develop a Transformer-based neural network
architecture for effective reflection removal
2

3. Motivation & Applications

Motivation & Applications
• Key Application Domains:
• Video surveillance systems
• Autonomous vehicle perception
• Digital image restoration and
enhancement
• Critical Impact: Accurate scene
interpretation directly influences
safety and decision-making
reliability
3

4. Related Work & Background

Related Work & Background
• Foundational Architectures:
• U-Net: Encoder-decoder with skip-connections
• ResNet: Residual learning for deep network training
• Transformer: Self-attention for global dependency modeling
• State-of-the-Art Methods:
• Restormer, Uformer, SwinIR (general artifact removal)
• MPRNet (multi-stage progressive refinement)
• Limitation of Existing Approaches: Training on simplified synthetic
data (alpha-blending) limits real-world generalization
4

5. Proposed Method – Overview

• Core Innovations:
• Physically-based rendering pipeline (Blender Cycles engine)
• Transformer-based architecture adapted for reflection removal
• High-dynamic-range (HDR) environment maps for photorealistic reflections
• Expected Outcome: State-of-the-art performance in reflection
removal tasks
5

6. Synthetic Dataset Generation – Scene Setup

• Toolchain: Blender 5.0 with Cycles
rendering engine
• Scene Components:
• Background plane textured with source image
• Virtual camera for image capture
• Semi-transparent glass object positioned
between camera and background
• Glass Material Parameters:
• Index of Refraction (IOR)
• Surface Roughness ∈ [0.0, 0.02]
6

7. Dataset Specifications

• Base Dataset: COCO subset (14,997 diverse images)
• Final Dataset Size: 59,988 image pairs (4 variations per base image)
• Render Resolution: 512 × 512 pixels
• Parameter Randomization:
• Glass tilt angle relative to optical axis
• Surface roughness variation
• Global scene rotation
• HDR environment maps from PolyHaven repository
7

8. Advantages of Physics-Based Data Generation

• Physical Phenomena Modeled:
• Multiple light reflections and refractions
• Angle-dependent glare intensity (Fresnel
effects)
• Reflection blurring correlated with surface
roughness
• Benefits:
• High photorealism of synthetic reflections
• Improved model robustness on real-world
images
• Reduced domain gap between training and
inference
8

9. Neural Network Architecture

• Backbone Structure: U-Net encoder-decoder framework
• Core Building Blocks: Restormer-inspired Transformer modules
• MDTA: Multi-Dconv Head Transposed Attention (spatial self-attention)
• GDFN: Gated-Dconv Feed-Forward Network (context-aware feature
transformation)
• Design Features:
• Large-kernel convolutions at input/output layers for expanded receptive field
• Skip-connections for spatial information preservation
• Balanced modeling of local textures and global dependencies
9

10. Training Strategies & Output Formulation

Training Strategies & Output Formulation
• Two Prediction Paradigms Under Investigation:
• Direct regression of the clean background image
• Prediction of the reflection layer followed by subtraction from input
• Hypothesis: Predicting the sparse reflection component may simplify
optimization and enhance restoration quality
10

11. Loss Functions – Individual Components

• L1 Loss:
Robust to outliers; promotes sharp reconstructions
• Charbonnier Loss
Smooth optimization near zero error; stable for large deviations
11

12. Composite Loss Function

• Total Loss Formulation:
• ℒ