(CVPR 2021 Oral) Open World Object Detection
APACHE-2.0 License
Humans have a natural instinct to identify unknown object instances in their environments. The intrinsic curiosity about these unknown instances aids in learning about them, when the corresponding knowledge is eventually available. This motivates us to propose a novel computer vision problem called: Open World Object Detection, where a model is tasked to:
We formulate the problem, introduce a strong evaluation protocol and provide a novel solution, which we call ORE: Open World Object Detector, based on contrastive clustering and energy based unknown identification. Our experimental evaluation and ablation studies analyse the efficacy of ORE in achieving Open World objectives. As an interesting by-product, we find that identifying and characterising unknown instances helps to reduce confusion in an incremental object detection setting, where we achieve state-of-the-art performance, with no extra methodological effort. We hope that our work will attract further research into this newly identified, yet crucial research direction.
See INSTALL.md.
Dataset setup: Follow these instructions.
Some bookkeeping needs to be done for the code, like removing the local paths and so on. We will update these shortly.
Data split and trained models: [Google Drive Link]
All config files can be found in: configs/OWOD
Sample command on a 4 GPU machine:
python tools/train_net.py --num-gpus 4 --config-file <Change to the appropriate config file> SOLVER.IMS_PER_BATCH 4 SOLVER.BASE_LR 0.005
Kindly run replicate.sh
to replicate results from the models shared on the Google Drive.
Kindly check run.sh
file for a task workflow.
If you are interested in Incremental Object Detection, you may have to consider this recent work. We have released the exact training configurations, trained models and logs for all three incremental settings: https://github.com/JosephKJ/iOD
Our code base is build on top of Detectron 2 library.
If you use our work in your research please cite us:
@inproceedings{joseph2021open,
title={Towards Open World Object Detection},
author={K J Joseph and Salman Khan and Fahad Shahbaz Khan and Vineeth N Balasubramanian},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021)},
eprint={2103.02603},
archivePrefix={arXiv},
year={2021}
}