Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support GitHub actions #558

Closed
wants to merge 5 commits into from
Closed

Support GitHub actions #558

wants to merge 5 commits into from

Conversation

AdamDonna
Copy link
Contributor

No description provided.

@AdamDonna AdamDonna changed the base branch from master to add-pre-commit November 23, 2023 08:38
@j-antunes
Copy link
Contributor

@AdamDonna - Is there anything I can help with this PR?

Base automatically changed from add-pre-commit to master November 27, 2023 16:28
@AdamDonna
Copy link
Contributor Author

@j-antunes Yes. If there's an templated solution for this that would be amazing, or if you have experience getting github actions working feel free to add to this PR or create a competing solution 👍

@j-antunes
Copy link
Contributor

@AdamDonna - Just wanted to touch base with you. I started by adding the linting CI to the project.
I created a .flake8 file with some standard rules, after I added them, I got a bunch of linting errors. I went trough all of them, and fixed them. If you look at what I pushed, it looks like a lot of changes, but all of them were only changes to make the linter happy.

I ran the unit tests locally, and they all passed.

I also noticed we are missing a requeriments.txt file 🙃 we should add one.
We also have a .tox .flake8 setup.py file, we should move all of that into the pyproject.toml file.

@AdamDonna
Copy link
Contributor Author

Great work. Did you get pre commit to work with those rules?

Moving to pyproject allows us to copy and modify config from other projects and it looks like the direction other jazzband projects are moving. I support it based that.

Interesting that we don't have a requirements file. I wonder how that was managed historically 🤔

@j-antunes
Copy link
Contributor

Yes, pre-commit also worked. However, I did not change anything on it. Worth revisiting it.

Sounds good about pyproject

I think previously they just installed the packages that they needed. If you look at runtests.py there is not a lot of dependencies. You only need django and dj_database_url.

Then on the CI/CD it looks like they used somr python-django pre-built containers. So those must already have the necessary packages.

@j-antunes j-antunes force-pushed the support-github-actions branch 2 times, most recently from 8c67dff to 01a0f0a Compare November 30, 2023 11:39
@j-antunes j-antunes force-pushed the support-github-actions branch from 4cbc294 to 3fe25ee Compare November 30, 2023 11:53
@j-antunes
Copy link
Contributor

@AdamDonna - I just pushed a new CI/CD to run unit tests. It now runs the file runtests.py which runs all the unit tests for this project.

We are missing running coverage for the unit tests, and running tox. I think it would make sense to run them separate. I think these should only trigger when merging to main.

- name: Set up Python environment
uses: actions/setup-python@v4
with:
python-version: "3.10"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be configured as a matrix so we can test against many python versions?

python-version: "3.10"
- name: Install dependencies
run: |
pip install django==3.2
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be configured as a matrix so we can cover different django version, or add additional steps for each django version?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that would be the next step. To run the different versions of python and django.

I think how I would like to implement this. Would be a sort of multi step. When someone pushs an initial PR it runs the unittests for the oldest version first. If that passes and someone does an initial review of the code. It would trigger running the matrix.

I think this makes sense, to save resources. There is no point in running the test matrix for something that is not yet finished.

@AdamDonna
Copy link
Contributor Author

I'm strongly in favour of running coverage should run on all changes as a part of each push. It helps to identify paths which aren't tested before they're merged, and prompts contributors to add more tests.

Comment on lines +6 to +23
flake8-lint:
runs-on: ubuntu-latest
name: flake8 Lint

steps:
- name: Check out source repository
uses: actions/checkout@v3

- name: Set up Python environment
uses: actions/setup-python@v4
with:
python-version: "3.10"

- name: Install flake8
run: pip install flake8

- name: Run flake8
run: flake8 .
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not add flake8 since ruff is a thing these days.

Besides, adding linting sounds like a different PR than just getting GHA running?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, not to mention the repo already uses pre-commit, so you can just use pre-commit/action :)

https://github.com/YatingMusic/miditoolkit/blob/6f0a8f3a78885e0b6614c70c097230f47a292361/.github/workflows/lint.yml#L1-L17

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused. Are you suggesting to keep using pre-commit instead of ruff?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No – it's just that this PR would be adding additional linting configuration beyond that already implemented in #557; instead of doing lints "manually", one can just run pre-commit, so lints are being configured in one place.

#560 switches the black + isort + flake8 configuration to ruff (and adds the GHA step that runs pre-commit).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay. I see

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a bit more to it. Jazzband enforce pre-commit as a CI step eg:// https://results.pre-commit.ci/run/github/1360247/1701345236.N6ZJwVBqR5qkswtYyt4tsw

So we must have some sort of pre-commit tooling. Running the precommit tooling against file changes in the diff is the most consistent way of enforcing linting.

Switching to Ruff looks fine. It looks easy to back out of in the unlikely event there's issues with it

@@ -0,0 +1,21 @@
name: CI-UnitTests
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm in strong favour of this file being renamed to test which feels like a standard

@AdamDonna
Copy link
Contributor Author

AdamDonna commented Dec 12, 2023

I'm not convinced that we should be changing the linting rules to get Ruff working here. That looks like a concession to get the tool working, rather than getting the tool to work for our configuration. I think that will need some reinvestigation before we move commit to it

@j-antunes
Copy link
Contributor

What would be the impact of changing the linter? I agree with you, that it would be better not to change it, but what I'm thinking is that the project as been out of date for quite some time, that an update/upgrade of this time would make sense.

@akx
Copy link
Contributor

akx commented Dec 15, 2023

@j-antunes

What would be the impact of changing the linter?

TBH, my #560 PR is the one changing the linter - flake8 had been technically in use in this repo before, but not enforced.
As to "why:", Ruff is a lot faster (so faster CI and faster pre-commit, and that of course translates to energy savings) and generally more capable than flake8, and also does isort and black formatting (and pyupgrade and flynt and . . .).

@akx
Copy link
Contributor

akx commented Dec 18, 2023

I think this can be closed with #560 and #561 having been merged?

@j-antunes
Copy link
Contributor

@akx - Yes I agree. Let's close this one.

@akx akx closed this Dec 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants