Recognize Any Regions
Haosen Yang, Chuofan Ma, Bin Wen, Yi Jiang, Zehuan Yuan, Xiatian Zhu
2023/11/7: Checkpoints are available on both Google Drive and OneDrive.2023/11/6: Code is available Now!
| Method | Box AP_rare | Box AP_all | Mask AP_rare | Mask AP_all | Download |
|---|---|---|---|---|---|
| RegionSpot-BB | 19.1 | 20.9 | 17.5 | 17.8 | model |
| RegionSpot-BL | 26.0 | 23.7 | 22.8 | 20.2 | model |
| RegionSpot-BL@336px | 26.3 | 25.0 | 23.4 | 21.3 | model |
The installation instruction and usage are in Getting Started with Recognize Any Regions.
First download a model checkpoint. Then the model can be used in just a few lines to get masks from a given prompt:
from regionspot.modeling.regionspot import build_regionspot_model
from regionspot import RegionSpot_Predictor
custom_vocabulary = ['<custom>']
clip_type = <clip_type>
regionspot = build_regionspot_model(checkpoint="<path/to/checkpoint>", custom_vocabulary=custom_vocabulary, clip_type=clip_type)
predictor = RegionSpot_Predictor(regionspot)
predictor.set_image(<your_image>)
masks, mask_iou_score, class_score, class_index = predictor.predict(<input_prompts>)See the demo.py on using RegionSpot with box prompts for more details.
If you use Recognize Any Regions in your research or wish to refer to the baseline results published here, please use the following BibTeX entry.
@inproceedings{yang2023recognize,
title={Recognize any regions},
author={Yang, Haosen and Ma, Chuofan and Wen, Bin and Jiang, Yi and Yuan, Zehuan and Zhu, Xiatian},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2023}
}