Skip to content

Commit

Permalink
update
Browse files Browse the repository at this point in the history
  • Loading branch information
kexinhuang12345 committed Oct 1, 2023
1 parent 2644c1b commit d60fa0d
Show file tree
Hide file tree
Showing 23 changed files with 1,148 additions and 318 deletions.
1,201 changes: 1,012 additions & 189 deletions auto_leaderboard.ipynb

Large diffs are not rendered by default.

18 changes: 9 additions & 9 deletions benchmark/admet_group/01caco2/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -367,24 +367,24 @@ <h3 id="leaderboard">Leaderboard</h3>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>0.328 <span>&#177;</span> 0.024 </td>
<td>0.330 <span>&#177;</span> 0.024 </td>
</tr>
<tr>
<td> 6 </td>
<td> Chemprop </td>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>0.340 <span>&#177;</span> 0.016 </td>
</tr>
<tr>
<td> 7 </td>
<td> Euclia ML model </td>
<td><a href="mailto:[email protected]">Euclia</a></td>
<td><a href="https://github.com/euclia/public-models"> GitHub</a>, <a href="https://github.com/euclia/public-models">Paper </a></td>
<td> 50 </td>
<td>0.341 <span>&#177;</span> 0.004 </td>
</tr>
<tr>
<td> 7 </td>
<td> Chemprop </td>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>0.344 <span>&#177;</span> 0.015 </td>
</tr>
<tr>
<td> 8 </td>
<td> RDKit2D + MLP (DeepPurpose) </td>
Expand Down
4 changes: 2 additions & 2 deletions benchmark/admet_group/03pgp/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -407,7 +407,7 @@ <h3 id="leaderboard">Leaderboard</h3>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>0.883 <span>&#177;</span> 0.016 </td>
<td>0.886 <span>&#177;</span> 0.016 </td>
</tr>
<tr>
<td> 11 </td>
Expand All @@ -423,7 +423,7 @@ <h3 id="leaderboard">Leaderboard</h3>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>0.858 <span>&#177;</span> 0.036 </td>
<td>0.860 <span>&#177;</span> 0.036 </td>
</tr>
<tr>
<td> 13 </td>
Expand Down
18 changes: 9 additions & 9 deletions benchmark/admet_group/04bioav/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -367,7 +367,7 @@ <h3 id="leaderboard">Leaderboard</h3>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>0.668 <span>&#177;</span> 0.069 </td>
<td>0.667 <span>&#177;</span> 0.068 </td>
</tr>
<tr>
<td> 6 </td>
Expand Down Expand Up @@ -403,20 +403,20 @@ <h3 id="leaderboard">Leaderboard</h3>
</tr>
<tr>
<td> 10 </td>
<td> Chemprop </td>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>0.581 <span>&#177;</span> 0.024 </td>
</tr>
<tr>
<td> 11 </td>
<td> Morgan + MLP (DeepPurpose) </td>
<td><a href="mailto:[email protected]">Kexin Huang</a></td>
<td><a href="https://github.com/mims-harvard/TDC/tree/master/examples/single_pred/admet"> GitHub</a>, <a href="https://doi.org/10.1093/bioinformatics/btaa1005">Paper </a></td>
<td> 1,477,185 </td>
<td>0.581 <span>&#177;</span> 0.086 </td>
</tr>
<tr>
<td> 11 </td>
<td> Chemprop </td>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>0.579 <span>&#177;</span> 0.023 </td>
</tr>
<tr>
<td> 12 </td>
<td> AttrMasking </td>
Expand Down
5 changes: 3 additions & 2 deletions benchmark/admet_group/05lipo/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -335,15 +335,15 @@ <h3 id="leaderboard">Leaderboard</h3>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>0.466 <span>&#177;</span> 0.006 </td>
<td>0.467 <span>&#177;</span> 0.006 </td>
</tr>
<tr>
<td> 2 </td>
<td> Chemprop </td>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>0.469 <span>&#177;</span> 0.005 </td>
<td>0.470 <span>&#177;</span> 0.009 </td>
</tr>
<tr>
<td> 3 </td>
Expand Down Expand Up @@ -433,6 +433,7 @@ <h3 id="leaderboard">Leaderboard</h3>
<td> 226,625 </td>
<td>0.743 <span>&#177;</span> 0.020 </td>
</tr>

</table>

<p><i class="fas fa-long-arrow-alt-down"></i>: The lower the better.</p>
Expand Down
20 changes: 10 additions & 10 deletions benchmark/admet_group/06aqsol/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -335,7 +335,7 @@ <h3 id="leaderboard">Leaderboard</h3>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>0.762 <span>&#177;</span> 0.020 </td>
<td>0.761 <span>&#177;</span> 0.025 </td>
</tr>
<tr>
<td> 2 </td>
Expand All @@ -347,28 +347,28 @@ <h3 id="leaderboard">Leaderboard</h3>
</tr>
<tr>
<td> 3 </td>
<td> Chemprop </td>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>0.818 <span>&#177;</span> 0.010 </td>
</tr>
<tr>
<td> 4 </td>
<td> RDKit2D + MLP (DeepPurpose) </td>
<td><a href="mailto:[email protected]">Kexin Huang</a></td>
<td><a href="https://github.com/mims-harvard/TDC/tree/master/examples/single_pred/admet"> GitHub</a>, <a href="https://doi.org/10.1093/bioinformatics/btaa1005">Paper </a></td>
<td> 633,409 </td>
<td>0.827 <span>&#177;</span> 0.047 </td>
</tr>
<tr>
<td> 5 </td>
<td> 4 </td>
<td> Basic ML </td>
<td><a href="mailto:[email protected]">Nilavo Boral</a></td>
<td><a href="https://github.com/NilavoBoral/Therapeutics-Data-Commons"> GitHub</a>, <a href="https://www.biorxiv.org/content/10.1101/2022.06.29.115436v1">Paper </a></td>
<td> N/A </td>
<td>0.828 <span>&#177;</span> 0.002 </td>
</tr>
<tr>
<td> 5 </td>
<td> Chemprop </td>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>0.829 <span>&#177;</span> 0.022 </td>
</tr>
<tr>
<td> 6 </td>
<td> GCN </td>
Expand Down
68 changes: 38 additions & 30 deletions benchmark/admet_group/07bbb/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -334,139 +334,147 @@ <h3 id="leaderboard">Leaderboard</h3>
<td> Lantern RADR Ensemble </td>
<td><a href="mailto:[email protected]">Rick Fontenot</a></td>
<td><a href="https://github.com/lanternpharma/tdc-bbb-martins"> GitHub</a>, <a href="https://github.com/lanternpharma/tdc-bbb-martins/blob/main/Lantern_TDC_BBBmartins.pdf">Paper </a></td>
<td> 267,439 </td>
<td>0.962 <span>&#177;</span> 0.003 </td>
<td> N/A </td>
<td>0.915 <span>&#177;</span> 0.002 </td>
</tr>
<tr>
<td> 2 </td>
<td> Lantern RADR Logistic Regression </td>
<td> Lantern RADR Deep Neural Network </td>
<td><a href="mailto:[email protected]">Rick Fontenot</a></td>
<td><a href="https://github.com/lanternpharma/tdc-bbb-martins"> GitHub</a>, <a href="https://github.com/lanternpharma/tdc-bbb-martins/blob/main/Lantern_TDC_BBBmartins.pdf">Paper </a></td>
<td> 456 </td>
<td>0.956 <span>&#177;</span> 0.006 </td>
<td> N/A </td>
<td>0.912 <span>&#177;</span> 0.003 </td>
</tr>
<tr>
<td> 3 </td>
<td> Lantern RADR Deep Neural Network </td>
<td><a href="mailto:[email protected]">Rick Fontenot</a></td>
<td><a href="https://github.com/lanternpharma/tdc-bbb-martins"> GitHub</a>, <a href="https://github.com/lanternpharma/tdc-bbb-martins/blob/main/Lantern_TDC_BBBmartins.pdf">Paper </a></td>
<td> 266,881 </td>
<td>0.949 <span>&#177;</span> 0.004 </td>
<td> ZairaChem </td>
<td><a href="mailto:[email protected]">Gemma Turon </a></td>
<td><a href="https://github.com/ersilia-os/zaira-chem-tdc-benchmark"> GitHub</a>, <a href="https://www.biorxiv.org/content/10.1101/2022.12.13.520154v1">Paper </a></td>
<td> N/A </td>
<td>0.910 <span>&#177;</span> 0.024 </td>
</tr>
<tr>
<td> 4 </td>
<td> Lantern RADR Random Forest </td>
<td><a href="mailto:[email protected]">Rick Fontenot</a></td>
<td><a href="https://github.com/lanternpharma/tdc-bbb-martins"> GitHub</a>, <a href="https://github.com/lanternpharma/tdc-bbb-martins/blob/main/Lantern_TDC_BBBmartins.pdf">Paper </a></td>
<td> 319 </td>
<td>0.928 <span>&#177;</span> 0.002 </td>
<td>0.908 <span>&#177;</span> 0.002 </td>
</tr>
<tr>
<td> 5 </td>
<td> ZairaChem </td>
<td><a href="mailto:[email protected]">Gemma Turon </a></td>
<td><a href="https://github.com/ersilia-os/zaira-chem-tdc-benchmark"> GitHub</a>, <a href="https://www.biorxiv.org/content/10.1101/2022.12.13.520154v1">Paper </a></td>
<td> N/A </td>
<td>0.910 <span>&#177;</span> 0.024 </td>
<td> Lantern RADR SVM </td>
<td><a href="mailto:[email protected]">Rick Fontenot</a></td>
<td><a href="https://github.com/lanternpharma/tdc-bbb-martins"> GitHub</a>, <a href="https://github.com/lanternpharma/tdc-bbb-martins/blob/main/Lantern_TDC_BBBmartins.pdf">Paper </a></td>
<td> 241 </td>
<td>0.905 <span>&#177;</span> 0.007 </td>
</tr>
<tr>
<td> 6 </td>
<td> Lantern RADR Logistic Regression </td>
<td><a href="mailto:[email protected]">Rick Fontenot</a></td>
<td><a href="https://github.com/lanternpharma/tdc-bbb-martins"> GitHub</a>, <a href="https://github.com/lanternpharma/tdc-bbb-martins/blob/main/Lantern_TDC_BBBmartins.pdf">Paper </a></td>
<td> 456 </td>
<td>0.903 <span>&#177;</span> 0.002 </td>
</tr>
<tr>
<td> 7 </td>
<td> SimGCN </td>
<td><a href="mailto:[email protected]">Suman Kalyan Bera</a></td>
<td><a href="https://github.com/KatanaGraph/SimGCN-TDC"> GitHub</a>, <a href="https://github.com/KatanaGraph/SimGCN-TDC/blob/main/Report_SimGCN_for_TDC_Benchmarks.pdf">Paper </a></td>
<td> 1,103,000 </td>
<td>0.901 <span>&#177;</span> 0.007 </td>
</tr>
<tr>
<td> 7 </td>
<td> 8 </td>
<td> ContextPred </td>
<td><a href="mailto:[email protected]">Kexin Huang</a></td>
<td><a href="https://github.com/mims-harvard/TDC/tree/main/examples/single_pred/admet"> GitHub</a>, <a href="https://arxiv.org/abs/1905.12265">Paper </a></td>
<td> 2,067,053 </td>
<td>0.897 <span>&#177;</span> 0.004 </td>
</tr>
<tr>
<td> 8 </td>
<td> 9 </td>
<td> AttrMasking </td>
<td><a href="mailto:[email protected]">Kexin Huang</a></td>
<td><a href="https://github.com/mims-harvard/TDC/tree/main/examples/single_pred/admet"> GitHub</a>, <a href="https://arxiv.org/abs/1905.12265">Paper </a></td>
<td> 2,067,053 </td>
<td>0.892 <span>&#177;</span> 0.012 </td>
</tr>
<tr>
<td> 9 </td>
<td> 10 </td>
<td> RDKit2D + MLP (DeepPurpose) </td>
<td><a href="mailto:[email protected]">Kexin Huang</a></td>
<td><a href="https://github.com/mims-harvard/TDC/tree/master/examples/single_pred/admet"> GitHub</a>, <a href="https://doi.org/10.1093/bioinformatics/btaa1005">Paper </a></td>
<td> 633,409 </td>
<td>0.889 <span>&#177;</span> 0.016 </td>
</tr>
<tr>
<td> 10 </td>
<td> 11 </td>
<td> Chemprop-RDKit </td>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>0.868 <span>&#177;</span> 0.027 </td>
<td>0.869 <span>&#177;</span> 0.027 </td>
</tr>
<tr>
<td> 11 </td>
<td> 12 </td>
<td> AttentiveFP </td>
<td><a href="mailto:[email protected]">Kexin Huang</a></td>
<td><a href="https://github.com/mims-harvard/TDC/tree/main/examples/single_pred/admet"> GitHub</a>, <a href="https://pubmed.ncbi.nlm.nih.gov/31408336/">Paper </a></td>
<td> 300,806 </td>
<td>0.855 <span>&#177;</span> 0.011 </td>
</tr>
<tr>
<td> 12 </td>
<td> 13 </td>
<td> GCN </td>
<td><a href="mailto:[email protected]">Kexin Huang</a></td>
<td><a href="https://github.com/mims-harvard/TDC/tree/main/examples/single_pred/admet"> GitHub</a>, <a href="https://arxiv.org/abs/1609.02907">Paper </a></td>
<td> 191,810 </td>
<td>0.842 <span>&#177;</span> 0.016 </td>
</tr>
<tr>
<td> 13 </td>
<td> 14 </td>
<td> NeuralFP </td>
<td><a href="mailto:[email protected]">Kexin Huang</a></td>
<td><a href="https://github.com/mims-harvard/TDC/tree/main/examples/single_pred/admet"> GitHub</a>, <a href="https://ieeexplore.ieee.org/document/9412489">Paper </a></td>
<td> 480,193 </td>
<td>0.836 <span>&#177;</span> 0.009 </td>
</tr>
<tr>
<td> 14 </td>
<td> 15 </td>
<td> Morgan + MLP (DeepPurpose) </td>
<td><a href="mailto:[email protected]">Kexin Huang</a></td>
<td><a href="https://github.com/mims-harvard/TDC/tree/master/examples/single_pred/admet"> GitHub</a>, <a href="https://doi.org/10.1093/bioinformatics/btaa1005">Paper </a></td>
<td> 1,477,185 </td>
<td>0.823 <span>&#177;</span> 0.015 </td>
</tr>
<tr>
<td> 15 </td>
<td> 16 </td>
<td> Chemprop </td>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>0.820 <span>&#177;</span> 0.112 </td>
<td>0.821 <span>&#177;</span> 0.112 </td>
</tr>
<tr>
<td> 16 </td>
<td> 17 </td>
<td> Basic ML </td>
<td><a href="mailto:[email protected]">Nilavo Boral</a></td>
<td><a href="https://github.com/NilavoBoral/Therapeutics-Data-Commons"> GitHub</a>, <a href="https://www.biorxiv.org/content/10.1101/2022.06.29.115436v1">Paper </a></td>
<td> N/A </td>
<td>0.811 <span>&#177;</span> 0.013 </td>
</tr>
<tr>
<td> 17 </td>
<td> 18 </td>
<td> CNN (DeepPurpose) </td>
<td><a href="mailto:[email protected]">Kexin Huang</a></td>
<td><a href="https://github.com/mims-harvard/TDC/tree/master/examples/single_pred/admet"> GitHub</a>, <a href="https://doi.org/10.1093/bioinformatics/btaa1005">Paper </a></td>
<td> 226,625 </td>
<td>0.781 <span>&#177;</span> 0.030 </td>
</tr>
<tr>
<td> 18 </td>
<td> 19 </td>
<td> Euclia ML model </td>
<td><a href="mailto:[email protected]">Euclia</a></td>
<td><a href="https://github.com/euclia/public-models"> GitHub</a>, <a href="https://github.com/euclia/public-models">Paper </a></td>
Expand Down
4 changes: 2 additions & 2 deletions benchmark/admet_group/08ppbr/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -335,7 +335,7 @@ <h3 id="leaderboard">Leaderboard</h3>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>7.811 <span>&#177;</span> 0.163 </td>
<td>7.788 <span>&#177;</span> 0.210 </td>
</tr>
<tr>
<td> 2 </td>
Expand All @@ -351,7 +351,7 @@ <h3 id="leaderboard">Leaderboard</h3>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>8.312 <span>&#177;</span> 0.180 </td>
<td>8.288 <span>&#177;</span> 0.173 </td>
</tr>
<tr>
<td> 4 </td>
Expand Down
18 changes: 9 additions & 9 deletions benchmark/admet_group/09vdss/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -379,20 +379,20 @@ <h3 id="leaderboard">Leaderboard</h3>
</tr>
<tr>
<td> 7 </td>
<td> Chemprop </td>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>0.491 <span>&#177;</span> 0.046 </td>
</tr>
<tr>
<td> 8 </td>
<td> ContextPred </td>
<td><a href="mailto:[email protected]">Kexin Huang</a></td>
<td><a href="https://github.com/mims-harvard/TDC/tree/main/examples/single_pred/admet"> GitHub</a>, <a href="https://arxiv.org/abs/1905.12265">Paper </a></td>
<td> 2,067,053 </td>
<td>0.485 <span>&#177;</span> 0.092 </td>
</tr>
<tr>
<td> 8 </td>
<td> Chemprop </td>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>0.474 <span>&#177;</span> 0.048 </td>
</tr>
<tr>
<td> 9 </td>
<td> GCN </td>
Expand All @@ -407,7 +407,7 @@ <h3 id="leaderboard">Leaderboard</h3>
<td><a href="mailto:[email protected]">Kyle Swanson</a></td>
<td><a href="https://github.com/chemprop/chemprop"> GitHub</a>, <a href="https://pubs.acs.org/doi/full/10.1021/acs.jcim.9b00237">Paper </a></td>
<td> N/A </td>
<td>0.394 <span>&#177;</span> 0.076 </td>
<td>0.389 <span>&#177;</span> 0.075 </td>
</tr>
<tr>
<td> 11 </td>
Expand Down
Loading

0 comments on commit d60fa0d

Please sign in to comment.