* denotes equal contribution
1 Robotics Research Centre, IIIT Hyderabad, India.
2 Australian Centre for Robotic Vision at the Queensland University of Technology (QUT), Brisbane, Australia.
3 ZF Friedrichshafen TCI, Hyderabad, India
Object detection is challenging under adverse weather conditions (such as fog or low-lighting)
because the objects are partially or not visible, resulting in missed detections.
We tackle this problem by proposing a Gated Differentiable Image Processing (GDIP) block, which
enhances the image by performing weighted Image Processing (IP) operations concurrently on the adverse input image. This
enhanced output is sent to the downstream object detection network, leading to superior detection performance
to other state-of-the-art methods.
We present Gated Differentiable Image Processing (GDIP) block, a domain-agnostic architecture which adaptively enhances adverse images for object detection by performing concurrent gated-weighing of image
GDIP provides flexible concurrent gated weighting of the individual IP operations that result in superior detection performance.
GDIP block can be integrated as single or multi-level enhancement module with an encoder
In addition, GDIP block has a unique advantage with its utility as a training regularizer, which directly improves object detection training for adverse conditions. This
eliminates any image enhancement overhead during inference, unlike other state-of-the-art works, thus resulting in higher throughput.
Using our GDIP block, we propose novel variants as below:
GDIP-Yolo: a single GDIP block (fed with an adverse input and latent embeddings from an encoder)
can be plugged into existing object detection networks (e.g., Yolo) and trained end-to-end with
adverse condition images (fog and low-lighting).
Multi-level GDIP-Yolo (MGDIP-Yolo): a multi-level version of GDIP where an
image is progressively enhanced through multiple GDIP blocks, each guided by a different layer
of the image encoder. Providing access to such multiple feature scales helps it utlilize
the local/global properties to selectively apply Image Processing operations.
GDIP as regularizer: an adaptation of GDIP as training regularizer,
which directly improves object detection training by learning weather-invariant features.
It can be removed during inference, thus saving compute time with improved performance in adverse conditions.
GDIP in the Wild
GDIP runs seamlessly on video sequences in the wild without any need for retraining or fine tuning,
showcasing its ability to detect objects by composing Image Processing operations through gating by training on only synthetic adverse data.
If you have any questions, please reach out to any of the above mentioned authors.