Skip to content

Latest commit

 

History

History
54 lines (44 loc) · 2.44 KB

EXTRACT_FEATURE.md

File metadata and controls

54 lines (44 loc) · 2.44 KB
Raw

Extract Motion Trajectories From Scratch

We also provide scripts to extract motion trajectories. The extraction process can be summerize as:

  • A. Calculate optical flow.
  • B. Extract motion trajectories using the optical flow.

❕ As this process require complex multi-nodes computation, we strongly recommend to download and use the pre-extracted motion trajectories.

A. Calculate Optical Flow

We compile a flow extractor with multi-node and multi-gpu support based on the dense_flow repository. The binary file can be download according to your machine architecture and GPU type.

System Arch GPU Download Link
Linux x86_64 A100/A800 extractor.zip
Linux x86_64 3090 extractor.zip

After you download the extractor file, unzip it into current dir.

unzip -d ./flow extractor.zip

Input the data csv file in data/csv to determine videos to calculate optical flow. For a device with 8 x 3090 GPUs, we set NUM_PROCESS = 8 or 16 and NUM_GPU = 8. You can split the data csv file into several chunks according to the number of your machines to enable multi-node extraction.

GPU memory size is not a crucial factor in the extraction process. The extraction speed is mainly limited by the number of CPU cores, memory bandwidth, and storage speed.

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:./flows/lib ./flows/extract_warp_gpu_parallel -f=data/csv/ucf101/train.csv -d=data/flows -n=NUM_PROCESS -g=NUM_GPU

The extracted flow has a same organization with dataset:

data/flow
└── UCF-101
    └── ApplyEyeMakeup
        └── v_ApplyEyeMakeup_g24_c05.avi
            ├── x.mp4
            └── x.mp4

B. Extract motion trajectories using the optical flow

Now you can extract motion trajectory using the calculated optical flow:

python m3video/extract.py --ds_base=data/csv/ucf101/train.csv --dump_base data/trajs --num_process=NUM_PROCESS --job_type=trajs_dense --traj_len=6 --gpu_num=NUM_GPU

This process will dump the motion trajectories into compressed pickle files under the organization as:

data/trajs
└── UCF-101
    └── ApplyEyeMakeup
        └── v_ApplyEyeMakeup_g24_c05.avi_6.gz