Code to generate "unicl_a#ShopAndDining/unicl_final.pth"

[yuyangdu01]

Thank you for your amazing work. When following your code and paper, I got stuck in the training scripts of UniVQA (CLVQA/run_scripts/mmclvqa/scene/S_run_distilgpt2_wogt_wtt_1.5_S1.sh).

Specifically, the error is "...path/unicl_a#ShopAndDining/unicl_final.pth doesn't exist", as shown in the below figure

I think the error may be related with line 18 of CLVQA/run_scripts/mmclvqa/scene/S_run_distilgpt2_wogt_wtt_1.5_S1.sh (highlighted below), which needs "unicl_a#ShopAndDining/unicl_final.pth" as an resume file. Could you please help provide the scripts for generating "unicl_a#ShopAndDining/unicl_final.pth"? Thanks a lot!

[StanLei52]

Hi,

Thank you for your interest. I am sorry that the README file is not clear since I missed the starting training of each stage. You should train each stage separately first to obtain the model checkpoint of each task sequence. I will update the README as soon as possible.

A workaround to do this is to call this function: gen_standalone. For example, to obtain the checkpoints for stages under the scene setting, call gen_standalone(CUDA_DEVICE, "scene", "unicl")(change to functional if needed) and the program will generate the corresponding commands which follow this format:

 CUDA_VISIBLE_DEVICES=$DEVICE mmf_run dataset=clvqa  \
   model=unicl \
   config=EXP_CONFIG/{}/cl_{}_{}_standalone.yaml \
   env.save_dir=/Users/stan/exp/clvqa/save/stand_alone/{}/{}_{} \
   training.checkpoint_interval=4000 \
   training.batch_size=32 \
   training.callbacks=[] 

Please also modify the path for env.save_dir. Conduct these "stand_alone" experiments first. After the checkpoints are saved, you can proceed for continual learning experiments.