You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The create-gem.py script currently uses the pandasmerge function, which is able to merge on any column. It turns out that the pandasjoin function is substantially faster, but is slightly more restrictive in that you can only join on the index.
join is essentially instantaneous, even for large matrices, where as merge becomes slow with large matrices. This is most important when you are dealing with massive matrices (greater than ~30,000 samples) where merge gets rather slow.
The text was updated successfully, but these errors were encountered:
Description of feature
The create-gem.py script currently uses the
pandas
merge
function, which is able to merge on any column. It turns out that thepandas
join
function is substantially faster, but is slightly more restrictive in that you can only join on the index.join
is essentially instantaneous, even for large matrices, where asmerge
becomes slow with large matrices. This is most important when you are dealing with massive matrices (greater than ~30,000 samples) where merge gets rather slow.The text was updated successfully, but these errors were encountered: