SoftGroup for STPLS3D

[thangvubk]

Hi @meidachen,

Thank you for sharing a new interesting outdoor dataset for instance segmentation. Based on your shared code, we have implemented our SoftGroup for STPLS3D, which achieves AP/AP50/AP25 of 46.4/62.0/69.5 on val set, significantly higher than the existing ones. I hope this may serve as a solid baseline for Urban3D@ECCV2022 challenge. Feel free to check https://github.com/thangvubk/SoftGroup.

[meidachen]

Hi @thangvubk ,

Really appreciate your feedback and implementing SoftGroup for STPLS3D!!!

Regarding the performance, we have recently found an issue in the evaluation script we provided in HAIS as well as the CodaLab evaluation. I just fixed it an hour ago and found out that our HAIS baseline was incorrect (the HIAS performance should be 35.1% instead of the 40.4% that is currently reported in the paper and our website). I'm not sure if you are using the same script for evaluation or not, but if you are, could you please re-evaluate it using the updated script?

The bug was at line 226
where I mistakenly used //1 instead of //1000, which made all our previous results incorrect.

Same thing for the codaLab evaluation script at here
where I mistakenly used //1 instead of //10000

I'm working to make the correction for results on all our pages, paper, and codalab evaluation server, etc.

I'm not sure if this will affect your result performance but just want to provide all the information for your consideration.

If possible, could you please share the entire result table with me? I can put it on this GitHub as a baseline and point other people to the SoftGroup repo for implementation details.

Thank you again!! and sorry for any inconvenience, this may be caused.

[thangvubk]

We use our implementation from ScanNet v2 and we modified it to support different datasets. I only set the min_npointfor evaluation of STPLS3D to 10. We use 1000 to get the ids.

https://github.com/thangvubk/SoftGroup/blob/a87940664d0af38a3f55cdac46f86fc83b029ecc/softgroup/evaluation/instance_eval.py#L255.

I am not sure whether anything else need to be taken into consideration. Feel free to have a look.

[thangvubk]

When we change 1000 to 1. The AP/AP50/AP25 are 53.5/68.8/77.7

[meidachen]

Wow!! really nice work!! Could you please share the result table with me? and I will make the update here.

[thangvubk]

Here it is

[meidachen]

Great! Thank you!

[meidachen]

SoftGroup result was uploaded to this repo with all related information. Thanks again for the implementation, and I hope to see you in the Urban3D workshop soon!