LTGC: Long-Tail Recognition via Leveraging Generated Content [Official, CVPR 2024, Oral]

[Project] [Paper]

Overview

Qihao Zhao^*,  Yalun Dai^*,  Hao Li,  Wei Hu,  Fan Zhang,  Jun Liu, 

(BUCT & NTU & SUTD & NWPU, * Equal contribution)

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2024, Oral Presentation

Further information please contact Qihao Zhao and Yalun Dai.

Dataset Preparation

(1) Three bechmark datasets

• Please download these datasets and put them to the /data file.
• ImageNet-LT and Places-LT can be found at here.
• iNaturalist data should be the 2018 version from here.

data
├── ImageNet_LT
│   ├── test
│   ├── train
│   └── val
├── Place365
│   ├── data_256
│   ├── test_256
│   └── val_256
└── iNaturalist 
    ├── test2018
    └── train_val2018

(2) Txt files

data_txt
├── ImageNet_LT
│   ├── ImageNet_LT_test.txt
│   ├── ImageNet_LT_train.txt
│   └── ImageNet_LT_val.txt
├── Places_LT_v2
│   ├── Places_LT_test.txt
│   ├── Places_LT_train.txt
│   └── Places_LT_val.txt
└── iNaturalist18
    ├── iNaturalist18_train.txt
    ├── iNaturalist18_uniform.txt
    └── iNaturalist18_val.txt 

Running Scripts

Before running, please replace your own OPENAI key.

Generated Existing Tail-class Descriptions

python lmm_i2t.py -d $DATASET_PATH -m $MAX_NUMBER -f $CLASS_NUMBER_FILE -exi $EXIST_DESCRIPTION_FILE

Generated Extended Tail-class Descriptions

python lmm_extension.py -exi $EXIST_DESCRIPTION_FILE -m $MAX_GENERATED_IMAGES -ext $EXTEND_DESCRIPTION_FILE

Generated Extended Data using Iterative Evaluation

python draw_i2t.py -ext $EXTEND_DESCRIPTION_FILE -d $DATASET_PATH -t $THRESH -r $MAX_ROUNDS

Citation

@inproceedings{zhao2024ltgc,
  title={LTGC: Long-tail Recognition via Leveraging LLMs-driven Generated Content},
  author={Zhao, Qihao and Dai, Yalun and Li, Hao and Hu, Wei and Zhang, Fan and Liu, Jun},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={19510--19520},
  year={2024}
}