Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adversarial Platipy #168

Open
SimonBiggs opened this issue Nov 17, 2022 · 8 comments
Open

Adversarial Platipy #168

SimonBiggs opened this issue Nov 17, 2022 · 8 comments

Comments

@SimonBiggs
Copy link
Contributor

SimonBiggs commented Nov 17, 2022

@rnfinnegan, that was a great talk on the work you and @pchlap have been doing in Platipy undergoing approximately anatomically relevant deformations.

As discussed at the conference I had a thought that it might be quite helpful to utilise that functionality within an autosegmentation testing suite. A deformation can be undergone, and it can be used to detect whether or not the segmentation has got worse. The difference in the segmentation quality can then be combined with https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.basinhopping.html to then search for a "worst performing deformation". This can then be used to give feedback to the model developer, in their search for failing cases. It might also be able to be a very useful metric in reporting one aspect of the generisability of a model. Essentially one could have an "Adversarial Dice" metric and an "Adversarial Hausdorff Distance" metric, and others.

If everyone was interested it would be quite nice to pull in @JakeKendrick1, Brani, and Daniel and create the implementation and write a paper together.

As a disclosure from my end, it would be very helpful for me and my product, as having a range of robust autosegmentation testing tools that I can include within my GitHub CI suite would help me make a significantly more robust product. This would also be a nice component to be able to show to the TGA through my regulatory process.


Keen to hear what the next steps in the collaboration are seen to be.

@pchlap
Copy link
Contributor

pchlap commented Nov 18, 2022

Hi @SimonBiggs, this all sounds really exciting. Very keen to progress things on this front.

Since this is shaping up to be a research project (or sub-project) I would suggest setting up a kick-off meeting with everyone interested as well as @LoisHolloway. We'd need to break down the tasks here so that we can properly split up the tasks and make sure we have the appropriate resources to be able to do this. It's also always a good idea to agree on authorship (in particular first authorship) upfront. I think there is a lot in this so there may even be opportunities for more than one paper depending on how things progress.

So I think a meeting would be a good next step to brainstorm and start to formulate all of this. I imagine Shrikant would also be interested to be involved in this.

@SimonBiggs
Copy link
Contributor Author

Sounds good, I'm coming to Sydney next week and could potentially come to Liverpool in person on Thursday. Might that work?

@LoisHolloway
Copy link

LoisHolloway commented Nov 18, 2022 via email

@SimonBiggs
Copy link
Contributor Author

All good. That makes sense. Let's just keep it to video call then and go with another time. Friday's are a preference for me.

@SimonBiggs
Copy link
Contributor Author

SimonBiggs commented Nov 19, 2022

It's also always a good idea to agree on authorship (in particular first authorship) upfront.

If it helps, I personally am not going to be vying for first authorship, and I suspect either you @pchlap, or @rnfinnegan should be taking that position depending on the level of commitment being given.

And @pchlap, it seems to align very well with your PhD. So you might naturally be doing the most work on it, and therefore potentially it should be your first authorship? Anyway, that's much more between you and @rnfinnegan, so I'll leave that for you.

If others are happy with it, I would appreciate the final author position if that's something other's aren't looking to be in.

@pchlap
Copy link
Contributor

pchlap commented Nov 21, 2022

Hi @SimonBiggs, I'll be around on Thursday so you're welcome to come by for a visit. But I think in regards to this project a zoom call to capture everyone (potentially) involved would be important to kick this off. My main concern is ensuring we have the appropriate resources and capacity to progress this in a timely fashion (or map out when we might have that capacity).

I can arrange an appropriate time for a call on our end, a Friday should be manageable (although it won't be this Friday). So we'll try to get:

  • Simon
  • Phil
  • Jake
  • Brani
  • Daniel
  • Shrikant
  • Lois

in the call. Did I miss anyone?

@SimonBiggs
Copy link
Contributor Author

SimonBiggs commented Nov 21, 2022 via email

@pchlap
Copy link
Contributor

pchlap commented Nov 22, 2022

haha of course, he's kind of important. OK I'll try to set something up.

Phil

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants