Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: run medusa in CI #86

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

chore: run medusa in CI #86

wants to merge 6 commits into from

Conversation

0xteddybear
Copy link
Contributor

@0xteddybear 0xteddybear commented Oct 15, 2024

merge after #87

this PR aims to include a short medusa run in CI, to avoid pushing false positives/errors that can be caught within 5min of fuzzing

it's a WIP, and I'm currently torn between 3 approaches: I believe having a docker image all our CI runs in is the way to go, see update below

docker-based action

this is a derivative of crytic/echidna-action, but modified to use eth-security-toolbox which has mostly up-to-date versions of a typical solidity toolchain, including medusa. with this, we don't have to worry about installing/updating medusa or crytic-compile, but have to (inside the action definition) deal with the mismatch between the github runner assuming everything will be executed as root and the image configuring everything for a normal user named ethsec

pros:

  • all tools come in a neat package, dont have to install any dependencies
  • can abstract away implementation details defined above so existing workflows dont need to be modified
  • we'd be maintaining a dockerfile and scripts where we could add features of fix issues as the one described below
  • configuration via action params is possible and would be very clean imo

cons:

  • we could have issues due to tob not updating the tools as fast as we'd like (the image currently has a foundry version from july that doesnt recognize the newly added forge fmt param, so I had to add a foundryup in the Dockerfile )
  • one more repo to maintain for the action
  • having two sets of dev toolchains (the non-cointainerized we already use for most tests and the one inside the container) means we could have compilation/reproducibility issues in one but not the other, which will waste time and cause frustration

running the entire CI inside eth-security-toolbox

The entire CI workflow would run inside a container that already has most tools in it, so we could do away with most 'setup' steps in workflow definitions.

However, due to the root-vs-nonroot issue outlined above, we would have to either modify most ci steps so they walk around that or maintain a fork/extension of the aforementioned image so it makes everything available to root

pros

  • low setup
  • potentially better reproducibility (less left up to gh runner internal state)

cons

  • have to deal with either ugly workarounds or maintaining a docker image of our very own

create an action to install medusa in the non-containerized gh runner environment

this would be similar to how we install foundry in CI: https://github.com/foundry-rs/foundry-toolchain/

pros:

  • minimally disruptive

cons

  • would probably take me a bit longer to implement
  • doesnt help with other dependencies such as crytic-compile (which is required to run anything with medusa) -> workflows will end up being more verbose

@0xteddybear 0xteddybear force-pushed the ci/medusa-gh-action branch 7 times, most recently from 78e64c5 to 3f7335d Compare October 15, 2024 18:10
@0xteddybear 0xteddybear changed the title chore: use medusa instead of echidna and configure gh action for it chore: run medusa in CI Oct 15, 2024
@0xteddybear 0xteddybear requested a review from gas1cent October 16, 2024 13:18
@0xteddybear 0xteddybear force-pushed the ci/medusa-gh-action branch 2 times, most recently from 04d1cb1 to 204ccc5 Compare October 16, 2024 16:13
@0xteddybear
Copy link
Contributor Author

0xteddybear commented Oct 16, 2024

update: I went for the 'run the entire CI inside a container' option, which proved simple enough if we maintain a fork of the docker image, see 0xteddybear/eth-security-toolbox-ci#1

TODO

  • use a non-personal container image
  • set the timeout we think is reasonable (I believe for non-trivial projects should be 5min) chose to set a runs # instead so running time is a red flag for high number of reverts

@0xteddybear 0xteddybear force-pushed the ci/medusa-gh-action branch 4 times, most recently from 825b36e to 65562a7 Compare October 17, 2024 18:24
@0xteddybear 0xteddybear marked this pull request as ready for review October 17, 2024 21:44
@0xteddybear 0xteddybear force-pushed the ci/medusa-gh-action branch 2 times, most recently from 3f49064 to 54c0819 Compare October 22, 2024 15:18
@0xteddybear 0xteddybear force-pushed the ci/medusa-gh-action branch 2 times, most recently from 63faf10 to 4dc57f9 Compare November 28, 2024 15:31
@@ -63,6 +63,22 @@ jobs:

- name: Run tests
run: yarn test:integration
medusa-tests:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tiniest nit: separate the actions with a blank line

Suggested change
medusa-tests:
medusa-tests:

gas1cent
gas1cent previously approved these changes Dec 20, 2024
@0xteddybear
Copy link
Contributor Author

0xteddybear commented Dec 20, 2024

note: will update ci image version on monday, after the nightly tagged release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants