Release of data / checkpoint / demo

[ZhangGongjie]

Hi, awesome job!

I am looking forward to the release of data/checkpoint/demo!

[xjj1999]

I wish I could use this masterpiece sooner too.

[ZCMax]

Thank you for your attention. We will be releasing the checkpoint and demo by the end of this week, but, due to certain constraints, we are unable to release the training data at this time. We'll keep updating the repo and please stay tuned for further updation. @ZhangGongjie @xjj1999

[ZCMax]

We've released our checkpoint on huggingface and the demo script, feel free to try it! And we'll continue to update the evaluation script on various benchmarks in the next week~

[xjj1999]

Hi,
I have designed questions with reference to the paper to test the performance of the model on 3D Visual Grounding and I can't get the desired answer, how should I design the query .

python ./llava/eval/run_llava_3d.py --model-path ./LLaVA-3D-7B --video-path ./demo/scannet/scene0356_00 --query "A rectangular brown door. It is next to a bed. Which object best matches the given description? Please provide its coordinates."

The output is ‘Bed.’

[ZCMax]

Hello, currently the code does not officially support 3D Visual Grounding Task, which requires the extra grounding module to achieve the accurate grounding results. We’ve tried simply output the 3D bounding box of object in text or location token format in the 3D VG cases, and found that it does not work well~ We’ll continue to update the VG code in the next week Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: xjj1999 ***@***.***> Sent: Sunday, October 20, 2024 1:45:53 AM To: ZCMax/LLaVA-3D ***@***.***> Cc: ChaimZhu ***@***.***>; Comment ***@***.***> Subject: Re: [ZCMax/LLaVA-3D] Release of data / checkpoint / demo (Issue #3) Hi, I have designed questions with reference to the paper to test the performance of the model on 3D Visual Grounding and I can't get the desired answer, how should I design the query . python ./llava/eval/run_llava_3d.py --model-path ./LLaVA-3D-7B --video-path ./demo/scannet/scene0356_00 --query "A rectangular brown door. It is next to a bed. Which object best matches the given description? Please provide its coordinates." — Reply to this email directly, view it on GitHub<#3 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AIZ5DAEF4OTRY2HDZND6WTTZ4KLFDAVCNFSM6AAAAABP6WFR7CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMRUGEYDKNJUGM>. You are receiving this because you commented.Message ID: ***@***.***>

[xjj1999]

Thank you for your reply! Again, thank you for such awesome work. As far as I know, LLava-3d is the first work that directly predicts bbox but achieves such performance. I'm looking forward to testing LLava-3d on the 3D Grouding task this week!