ci: add github actions workflow to test self hosted runner #282

anjannath · 2024-08-29T10:21:33Z

need to use a cloud storage for the backend, to be able to remove the instance after the job
Add mapt destroy job to remove provisioned instance

adrianriobo · 2024-09-11T04:40:40Z

.github/workflows/hosted-runner.yaml

+name: Test Self hosted runner addition
+
+on:
+  workflow_call:


We will need a strategy based on workflow_run to get access to secrets and test the binary build from the forked repo, with the current build you have here it will build it form the head of the main instead of from the PR

Similar to https://github.com/crc-org/ci-definitions/blob/70d016f5fd15e7cbdf8e3056c500654b0196a049/.github/workflows/crc-builder-pusher.yml#L18 stragtey for build / push

we also have to store the backend in a cloud storage to be able to destroy the created resources later

@anjannath ^^ Just added all required secrets for this:

ARM_TENANT_ID

ARM_SUBSCRIPTION_ID

ARM_CLIENT_ID

ARM_CLIENT_SECRET

AZURE_STORAGE_ACCOUNT

AZURE_STORAGE_KEY

Also to use the remote storage for backed-url the url would be something like (notice base name should exactly match mapt-gh-runner-mapt-state):

--backed-url azblob://mapt-gh-runner-mapt-state/${{ github.repository }}-${{ github.run_id }}

Note: This event will only trigger a workflow run if the workflow file is on the default branch.

from: https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#workflow_run

It seems workflow_run only works when the workflow is already in the main branch, so need to test this first on my fork and then update the PR, we'll have to initially merge something and then create follow up PR to fix if any issues found

adrianriobo · 2024-10-22T10:46:04Z

Hey can we test selfhosted runner running crc on it. I am curious on the user running the agent.

I mean when I set up ssh I had to trick the user starting the service, as with default setup it is a system user (privileged) as so crc is not working. How is working with the gh agent?

anjannath · 2024-10-22T11:54:23Z

Hey can we test selfhosted runner running crc on it. I am curious on the user running the agent.

still not able to run jobs on the self-hosted runner from this repo, currently i am stuck with getting a runner registration token (this is a temporary token and only valid for 60mins) we need to have a way to fetch this token from withing the job itself, so we need to add a personal access token which has the permission to fetch the runner registration token for this repository

in the latest push i've added workflows to handle the above but still need to test, i'll need to first test the entire flow on my fork

I mean when I set up ssh I had to trick the user starting the service, as with default setup it is a system user (privileged) as so crc is not working. How is working with the gh agent?

the github runner agent is running as service using the nt authority\network service account, i did a test run here: https://github.com/anjannath/mapt/actions/runs/11459558776/job/31884314010

will have to try running crc start to see if it works with this user, the NetworkService is also a privileged user account i think

adrianriobo · 2024-10-22T15:53:41Z

Yeah my point is, if that is the case may we need to trick the agent (if possible) to start as a user process on startup

anjannath · 2024-10-25T08:51:56Z

@adrianriobo the workflow_call strategy will pick the commit from the PR if we add a step for checkout code in the workflow job

so currently this PR adds to re-usable workflows, provision_hosted_runner and destroy_hosted_runner and a build_on_hosted_runner workflow that makes use of the reusable workflows, but currently it seems the secrets are not accessible to the workflows as they are not part of the repo main branch yet

the build-on-hosted-runner workflow additionally has to fetch a runner registration token which is needed to add a self hosted runner and is an input for the provision_hosted_runner workflow

this adds a workflow to provision a self hosted runner using mapt on azure and run a simple job on that runner to test self-hosted runner feature of mapt

anjannath · 2024-11-06T07:57:47Z

@adrianriobo this is finally ready for review again, got some successful runs on my fork, https://github.com/anjannath/mapt/actions/runs/11698979004

the new build-on-hosted-runner workflow will only run once we merge this PR and i have set the workflow to run only for PRs currently, maybe we should change that and test the self hosted feature only once changes are pushed to main (i.e PR is merged) and not for PRs?

adrianriobo

^^ @anjannath About your comment you can add a filter on PRs were any file of the util/ghactions is modified i.e. https://github.com/crc-org/ci-definitions/blob/main/.github/workflows/crc-builder-builder.yaml#L6C3-L8C79

Beyond that LGTM

adrianriobo · 2024-11-07T14:47:41Z

.github/workflows/provision-hosted-runner.yaml

+              --install-ghactions-runner --ghactions-runner-name "az-runner-${{inputs.operating_system}}-${{github.event.workflow_run.id}}" \
+              --ghactions-runner-repo "${{inputs.runner_repo}}" --ghactions-runner-token ${{steps.fetch_token.outputs.runner_token}}
+
+      - name: wait for runner to start accepting jobs


Out of curiosity why is this needed? I was expecting when it is ready the job test_run_selfhosted_runner will be executed

during testing i found that when the job executed immediately the runner would never pick it up, couldn't figure out why this was happening

i just tried opening a second PR just to test by running a different workflow targeting that runner then it'd work and it'd also pickup the pending test_run_selfhosted_runner job, so figured that if there's a bit of delay between the runner getting added and the next job getting triggered it all works

anjannath · 2024-11-08T09:15:41Z

^^ @anjannath About your comment you can add a filter on PRs were any file of the util/ghactions is modified i.e. https://github.com/crc-org/ci-definitions/blob/main/.github/workflows/crc-builder-builder.yaml#L6C3-L8C79

Beyond that LGTM

actually this cannot be used, since we trigger the build-on-hosted-runner workflow based on if oci-build workflow is finished, and then on the job check if the event was a pull_request or not

openshift-ci bot added the do-not-merge/work-in-progress label Aug 29, 2024

adrianriobo reviewed Sep 11, 2024

View reviewed changes

anjannath force-pushed the ghc-runner branch 5 times, most recently from 0b56135 to ec4f431 Compare October 4, 2024 12:26

anjannath force-pushed the ghc-runner branch 4 times, most recently from 46188ca to ccf8ed1 Compare October 21, 2024 11:40

anjannath force-pushed the ghc-runner branch from ccf8ed1 to bd6c662 Compare October 22, 2024 10:57

anjannath force-pushed the ghc-runner branch 3 times, most recently from 4ee2cfb to 52d854b Compare October 25, 2024 08:42

anjannath changed the title ~~[wip] ci: add github actions workflow to test self hosted runner~~ ci: add github actions workflow to test self hosted runner Oct 25, 2024

openshift-ci bot removed the do-not-merge/work-in-progress label Oct 25, 2024

adrianriobo changed the title ~~ci: add github actions workflow to test self hosted runner~~ WIP: ci: add github actions workflow to test self hosted runner Nov 4, 2024

openshift-ci bot added the do-not-merge/work-in-progress label Nov 4, 2024

anjannath force-pushed the ghc-runner branch from 52d854b to 66f42cd Compare November 6, 2024 07:47

ci: add github actions workflow to test self hosted runner

ba2b502

this adds a workflow to provision a self hosted runner using mapt on azure and run a simple job on that runner to test self-hosted runner feature of mapt

anjannath force-pushed the ghc-runner branch from 66f42cd to ba2b502 Compare November 6, 2024 07:54

anjannath changed the title ~~WIP: ci: add github actions workflow to test self hosted runner~~ ci: add github actions workflow to test self hosted runner Nov 6, 2024

openshift-ci bot removed the do-not-merge/work-in-progress label Nov 6, 2024

adrianriobo approved these changes Nov 7, 2024

View reviewed changes

adrianriobo merged commit 7db5d30 into redhat-developer:main Nov 8, 2024
4 checks passed

This was referenced Nov 11, 2024

Create a gh worfkow for testing gh self hosted runner #256

Closed

[CI] Investigate why a sleep is needed after adding a runner for it to start accepting jobs #339

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: add github actions workflow to test self hosted runner #282

ci: add github actions workflow to test self hosted runner #282

anjannath commented Aug 29, 2024 •

edited

Loading

adrianriobo Sep 11, 2024

adrianriobo Sep 11, 2024

anjannath Sep 24, 2024

adrianriobo Oct 3, 2024

anjannath Oct 4, 2024 •

edited

Loading

adrianriobo commented Oct 22, 2024

anjannath commented Oct 22, 2024 •

edited

Loading

adrianriobo commented Oct 22, 2024

anjannath commented Oct 25, 2024

anjannath commented Nov 6, 2024 •

edited

Loading

adrianriobo left a comment •

edited

Loading

adrianriobo Nov 7, 2024

anjannath Nov 7, 2024

anjannath commented Nov 8, 2024

ci: add github actions workflow to test self hosted runner #282

ci: add github actions workflow to test self hosted runner #282

Conversation

anjannath commented Aug 29, 2024 • edited Loading

adrianriobo Sep 11, 2024

Choose a reason for hiding this comment

adrianriobo Sep 11, 2024

Choose a reason for hiding this comment

anjannath Sep 24, 2024

Choose a reason for hiding this comment

adrianriobo Oct 3, 2024

Choose a reason for hiding this comment

anjannath Oct 4, 2024 • edited Loading

Choose a reason for hiding this comment

adrianriobo commented Oct 22, 2024

anjannath commented Oct 22, 2024 • edited Loading

adrianriobo commented Oct 22, 2024

anjannath commented Oct 25, 2024

anjannath commented Nov 6, 2024 • edited Loading

adrianriobo left a comment • edited Loading

Choose a reason for hiding this comment

adrianriobo Nov 7, 2024

Choose a reason for hiding this comment

anjannath Nov 7, 2024

Choose a reason for hiding this comment

anjannath commented Nov 8, 2024

anjannath commented Aug 29, 2024 •

edited

Loading

anjannath Oct 4, 2024 •

edited

Loading

anjannath commented Oct 22, 2024 •

edited

Loading

anjannath commented Nov 6, 2024 •

edited

Loading

adrianriobo left a comment •

edited

Loading