Leaderboard for heterogeneous graphs is missing

[amblee0306]

In the Simple-HGN code's README (https://github.com/THUDM/CogDL/tree/master/examples/simple_hgn), it mentioned that there is a leaderboard but the link is not working. Do you know where I can find this heterogeneous graphs leaderboard?

[cenyk1230]

Hi @amblee0306 ,

You can find the cogdl leaderboard from this link. If you want to see the original repo of Simple-HGN, please check this link

[amblee0306]

Hi @cenyk1230

Thanks for your reply, the leaderboard stated that the models were evaluated using macro-f1 scores, can I check how can I change the code such that the macro-f1 scores are returned? Currently when I run the simple-hgn from cogDL, I believe the test_acc is being utilized instead.

[cenyk1230]

Hi @amblee0306 ,

It seems that the results in the leaderboard should be micro-f1 scores (the same as accuracy in the multi-class setting) rather than macro-f1.

[amblee0306]

Hi @cenyk1230

Yes indeed. I would like to get the macro-f1 scores as well. Do you have any advice on how that is possible? Also, the score on the leaderboard, is it only for one instance or it refers to the average micro-f1 score? Cause I ran 10 instances and the average was lower than what was mentioned on the leaderboard.