Refactoring tsvtool

[camillebrianceau]

TSVTOOL REFACTORING

#313

• Rename tsvtool --> tsvtools
• Rename extract --> prepare-data
• Rename split/ kfold --> ??

#310 Remove restrictcommand

GETLABELS

#311 Change diagnosis into group and subgroup

#258 For now, each group (CN --> MCI --> AD) has a subgroup( p, s, r, us, uk) and this subgroup represents the evolution of the disease during time_horizon :

• p : progressive
• r: regressive
• s: stable
• us: unstable
• uk: unknown

#233 done

#312 Now there is only one tsv with group and subgroup and the group columns is what was called diagnosis in the old version of get labels.

Now, instead of giving path to the missing_mods directory and merge.tsv file, we only give a BIDS_directory and the pipeline will perform clinica iotools merge-tsvand clinica iotools check_missings_modalities inside the get labels pipeline

The command line for get labels is:

clinicadl tsvtools getlabels BIDS_directory/ get_labels_directory/ -d CN -d pMCI

input : BIDS directory and GETLABELS directory
output : (in getlabels directory) getlabels.json + getlabels.tsv + merge.tsv + missing_mods_directory

SPLIT AND K-FOLD

#314 almost done

CHANGES THAT NEED TO BE DISCUSSED

• uk is unknown subgroup : if there are not enough sessions to assess the reliability of the label but no changes were spotted. I chose to put uk for each last session.
• I added a directory at the beginning of the getlabels function showing the progression of Alzheimer's disease : {CN:0, MCI:1, AD:2 }.
  I added the option --stability_dict, no need to keep it, but if wanted, we can pass a list of diagnoses into the command line in order of the disease progression so we can work with other disesases.
• if you add the option --diagnosis with a group, you will get all subjects for whom, for at least one session, the diagnostic is this group.
• if you add the option --diagnosis with a subgroup, you will get all subjects that have this subgroup

[pep8speaks]

Hello @camillebrianceau! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2022-09-13 09:52:58 UTC

[mdiazmel]

Hi @camillebrianceau ,

Thanks for this huge PR. Requested changes concerns mainly the docstrings and some texts. Also try to add typing hints to the new functions that you create.

About the files in the folder tests/data/tsvtool/, these files will be in the new data ci package? If yes, I suppose that they can be removed from this repository.

[camillebrianceau]

I think that this PR is ready to merge.
Some tests don't pass because the data needed are on dvc but we can test the tsvtools locally.

At the end, what is new ?

TSVTOOL REFACTORING

#313

• Rename tsvtool --> tsvtools
• Rename extract --> prepare-data
• Didn't rename split/ kfold

#310Remove restrict command

GETLABELS

#311 Change diagnosis into group and subgroup
--> Finally we don't have group and subgroup, we keep the diagnosis column but without the labels pMCI and sMCI but we added a new command get-progression to get the subgroup (see the first comment of this PR) in a column name progression.

#312 Now there is only one tsv.

Now, instead of giving path to the missing_mods directory and merge.tsv file, we only give a BIDS_directory and the pipeline will perform clinica iotools merge-tsvand clinica iotools check_missings_modalities inside the get labels pipeline

The command line for get labels is:

clinicadl tsvtools get-labels BIDS_directory/  -d CN -d pMCI

input : BIDS directory
output : (in getlabels directory) labels.json + labels.tsv + merge.tsv + missing_mods_directory

SPLIT AND K-FOLD

314 done

CHANGES THAT NEED TO BE DISCUSSED

• uk is unknown subgroup : if there are not enough sessions to assess the reliability of the label but no changes were spotted. I chose to put uk for each last session.

• I added a directory at the beginning of the getlabels function showing the progression of Alzheimer's disease : {CN:0, MCI:1, AD:2 }.

• I added the option --stability_dict, no need to keep it, but if wanted, we can pass a list of diagnoses into the command line in order of the disease progression so we can work with other disesases.

• if you add the option --diagnosis with a group, you will get all subjects for whom, for at least one session, the diagnostic is this group.

• if you add the option --diagnosis with a subgroup, you will get all subjects that have this subgroup

I know that I still need to write the docs.

[mdiazmel]

Just a couple of changes before merging!

[mdiazmel]

LGTM!
I'll merge.
Note: tests for tsvtools are failing because they need the data provided by PR #350 (to be merged soon).