Add workflow to publish on PyPI

[alexdewar]

I've added a CI workflow to auto-publish Autocorpus to PyPI on release, based on the one we use for MUSE. (I simplified it a bit though as there are some steps we don't need and I figure there's not a lot of point in trying to publish to testpypi beforehand -- either it will succeed publishing to actual PyPI or not, so I don't see what that gains us). I'm tagging @dalonsoa as I think he wrote this workflow originally.

The repo is currently not in a state where it could be published as a wheel (see #32), but we won't be making a release before this is sorted, so I think this can be reviewed now. I'll make it a draft PR anyway.

Aside from that, someone in the research team will need to set up an account on PyPI and enable trusted publishing (if you do this, you don't need API tokens or passwords).

Closes #33.

[AdrianDAlessandro]

Seems sensible. Maybe worth cross-checking against this guide?

And with TestPyPI, I think we should use that initially (just run poetry publish locally) to make sure everything works as expected.

[dalonsoa]

I agree that for automatic deployment using TestPyPI is redundant. However, it was there, in the past, so you could check that the uploaded package was working as planned when downloading it. We found issues in SWMManywhere where some files hadn't been bundleded with the wheel, for example. Ideally, what you want to do is to publish pre-reseases to TestPyPI, check that all works, and then publish the full release to PyPI.

[alexdewar]

Seems sensible. Maybe worth cross-checking against this guide?

That's a good resource. I've had a look through and it seems we're ticking all the boxes.

And with TestPyPI, I think we should use that initially (just run poetry publish locally) to make sure everything works as expected.

You mean just locally, rather than having it as part of the workflow?

[alexdewar]

I agree that for automatic deployment using TestPyPI is redundant. However, it was there, in the past, so you could check that the uploaded package was working as planned when downloading it. We found issues in SWMManywhere where some files hadn't been bundleded with the wheel, for example. Ideally, what you want to do is to publish pre-reseases to TestPyPI, check that all works, and then publish the full release to PyPI.

Makes sense. I guess you could maybe achieve the same thing by trying to use the wheel before uploading?

[alexdewar]

I think this is good to go, as long as @AdrianDAlessandro is happy with it. Now that #76 is merged, this is the last piece before you can push to PyPI 🎉

[alexdewar]

(Ok there are a couple more issues that probably should be done first, but you could publish after this 😛)

[AdrianDAlessandro]

And with TestPyPI, I think we should use that initially (just run poetry publish locally) to make sure everything works as expected.

You mean just locally, rather than having it as part of the workflow?

Yes

[AdrianDAlessandro]

I think I'm happy with this. But I have one extra thought! Is there a way to confirm that the version number used in the release tag matches the version number in the pyproject.toml?

[alexdewar]

I think I'm happy with this. But I have one extra thought! Is there a way to confirm that the version number used in the release tag matches the version number in the pyproject.toml?

I don't think so 😞. It doesn't look like the GitHub Action we're using will check anything like that -- it just consumes whatever wheel files you give it.

I suppose we could script that check fairly easily though... It is a potential footgun, after all. I'm happy to have a go, if we think it will be useful.

[dalonsoa]

Not with poetry. To have that done right we will need to use hatch. See the setup in SWMManywhere. https://github.com/ImperialCollegeLondon/SWMManywhere/blob/main/pyproject.toml

[alexdewar]

Not with poetry. To have that done right we will need to use hatch. See the setup in SWMManywhere. https://github.com/ImperialCollegeLondon/SWMManywhere/blob/main/pyproject.toml

That's useful. hatch seems like it's got some nice features.

[AdrianDAlessandro]

I think I'm happy with this. But I have one extra thought! Is there a way to confirm that the version number used in the release tag matches the version number in the pyproject.toml?

I don't think so 😞. It doesn't look like the GitHub Action we're using will check anything like that -- it just consumes whatever wheel files you give it.

I suppose we could script that check fairly easily though... It is a potential footgun, after all. I'm happy to have a go, if we think it will be useful.

Yea, I think it's useful. Maybe we put an extra job before build-wheel that just returns an error code if autocorpus.__version__ does not match ${{ github.ref_name }} and that job is required for the subsequent jobs (build and publish) to run?

[alexdewar]

@AdrianDAlessandro Cool. I've had a go. I tried testing it locally as much as I could and I think it should work.

[alexdewar]

I also fixed a stray typo

[AdrianDAlessandro]

Nice! This looks good to me!

[alexdewar]

Cool. Thanks!