First pass at generating mermaid diagrams for all workflows

[mvdbeek]

These are pretty cool.

I guess some possible next steps might be

• render the mermaid diagrams in the static site
• render subworkflow steps into the same markdown file
• include diagram in readme.md file
• use hyperlinks to move users to subworkflows
• explore tool tips to maybe show tool ids

[hexylena]

The corresponding GTN version: https://github.com/galaxyproject/training-material/blob/main/_plugins/jekyll-topic-filter.rb#L398-L457,

[mvdbeek]

That's cool, I'll see that we can make that look more alike. The labels and utf-8 chars are nice!

[hexylena]

I thought in the past about making it a service, e.g. on apps.galaxyproject.eu, where we could pass in a URL (or POST) a workflow file, and get back either the raw mermaid text (maybe good for your use case embedding in github), or an iframe that's ready to go (maybe good for the GTN use case.) Not sure if this is worth exploring but, wanted to raise it in case it was interesting to you as well.

[mvdbeek]

I thought about that, but i think the most interesting use is in embedding the diagrams in markdown files and for that the tiny script seems like less moving parts ? We could definitely push this into tool-util for re-use though, and maybe use this as the svg rendering engine instead of the one that we have now in Galaxy (as used in workflow reports) ?

[hexylena]

yeah, similar considerations for me, network requests are sub-optimal. I like the idea of being able to use it for reports, it's a nicer cleaner representation a lot of the time!

[jmchilton]

Not a -1 or anything, you've got to do what you've got to do but I would really appreciate it if you uses abstractions from gxformat2 so this works with YAML workflows 👼 .

[mvdbeek]

Turns out .ga is a lot easier to process than gxformat2. I spent the afternoon processing the gxformat2 workflows, and I think I haven't gotten anywhere. I could just convert yaml workflows to the native representation ...

[mvdbeek]

I've gone the conversion to native route for now. There's a bunch more we could do but let's maybe say this is good enough for a first pass (and maybe some hacking next week) ?

[bgruening]

Looks pretty cool! Is this replacing the embedded workflow version? Or in addition?

[mvdbeek]

There's a lot of complexity to embedding the preview (need to add cors exceptions, allow embedding in nginx and a handle to the workflow, need the tools installed etc) and I also prefer the simpler view here. I don't need to know what each step's input and output is, which results in much cleaner graphs.

[mvdbeek]

Was the 👎 to technical issues (the way we run the IWC this isn't currently possible without committing to writing back workflow ids) or to me preferring the view without explicit input-output connections ? Is there something you would like to see ?

[dannon]

Don't want to preclude more discussion about improvements or suggestions around any eventual/possible role of the live-galaxy embedded renderings, but the challenges Marius mentions and we've discussed are significant and this is fantastic stuff that would be great to go ahead and ship. They work and look great in my local testing, so I'm going to go ahead and merge this and we can continue to iterate on this and other rendering approaches moving forward.

[mvdbeek]

Attention: deployment skipped!

https://github.com/galaxyproject/iwc/actions/runs/12268454675

[jmchilton]

I could just convert yaml workflows to the native representation ...

This is what all the other display stuff like the graph visualization does. I am sorry I didn't make that clear - I didn't mean for that to be a whole afternoon project.