RUNNER_TOOL_CACHE and/or AGENT_TOOLSDIRECTORY not picked up on self hosted runner
#914
Comments
|
Hello @nvincent-vossloh, |
|
Hello @nvincent-vossloh, Thank you once again for creating this issue. We have investigated and found that the problem is due to environment variables not being available in the context where the action is running. The given logs indicate the inclusion of a path (-I/opt/hostedtoolcache/Python/3.10.14/x64/include/python3.10) which does not exist on the self-hosted runner. You need to ensure that the actions/setup-python action respects the custom tool cache directory by setting the environment variables directly in the workflow, as shown in the job below: you can use any custom directory to install python on your self-hosted runner using AGENT_TOOLSDIRECTORY environment variable as per the setup-python documentation. |
|
Hello @nvincent-vossloh, Please let us know in case of any further concerns on the above information. |
|
Thanks for your feedback, I will try your solution and let you know if that fixed the issue on my side. |
|
The documentation you link to, states the the environment variable could be set in the same shell where the runner is running. Unfortunately that does not work (anymore ?), that is why I opened the issue in the first place. Maybe the runner software does not use the env variable from the shell anymore (did not work with the .env file either). Anyway, unfortunately setting the env variable in the workflow does not fix the issue, the behavior is still the same. On my first try it succeeded, but that was because the pip install step used a cache version of PyGObject, after deleting it the failure to compile still happens. Only setting CPATH seems to work. |
|
Hi @nvincent-vossloh, After further investigation, we found that setting the environment variable in the workflow successfully works when the cached version of PyGObject is available. However, after clearing the cache, the compilation failure reoccurs. Given these circumstances, It appears that only setting CPATH directly works. This is because CPATH explicitly sets the include path for the compiler, ensuring the correct headers are found during the compilation of PyGObject. The default paths set by pip or setup-python may not align with the directory structure on the self-hosted runner, leading to the observed compilation errors. Setting CPATH ensures that the compiler can locate the necessary Python header files, avoiding the "No such file or directory" error for Python.h. This approach provides a direct and reliable way to specify include directories, bypassing any inconsistencies in environment variable handling between the workflow and the self-hosted runner. Thank you for your understanding and patience as we work through this issue. |
|
Do you plan on updating the |
|
Hi @nvincent-vossloh, Thank you for suggesting to include directory to the CPATH environment variable in the setup-python action. We acknowledge the importance of this feature and will explore the possible ways to implement it. We will consider this for a future enhancement. In the meantime, here is an alternative solution to include CPATH in a GitHub Actions workflow: |
Thank you for taking my suggestion into account.
In the end I used this: env:
pytest_python_version: '3.10'
[...]
- uses: actions/setup-python@v5
id: setup-python
with:
python-version: ${{env.pytest_python_version}}
- name: setup CPATH
run: |
echo "python version : ${{ steps.setup-python.outputs.python-version }}"
python_version="${{ env.pytest_python_version }}"
echo "python path : ${{ steps.setup-python.outputs.python-path }}"
CPATH=${pythonLocation}/include/python${python_version}
echo CPATH=${CPATH} >> "${GITHUB_ENV}"which leverage the various env variables set. Thanks again. |
gowridurgad
added
feature request
HarithaVattikuti
added
feature request
Description:
I have exported
RUNNER_TOOL_CACHEandAGENT_TOOLSDIRECTORYin the shell running the self-hosted runner, however apip installin the workflow still uses/opt/hostedtoolcachedirectory.job-log.txt
Action version:
actions/setup-python@v5Platform:
Runner type:
Tools version:
python-version: '3.10'
Repro steps:
From the Self-hosted runner directory:
Expected behavior:
I would expect the
pip installstep to set properly the-Iflags without/opt/hostedtoolcacheActual behavior:
The logs (see above) show that it tried to add an include path (
-I/opt/hostedtoolcache/Python/3.10.14/x64/include/python3.10) which do not exist on the self hosted runner.Workaround I am not fond of
With this workaround I manage to install the lib, however it looks like that pip cached some stuff in the default cache directory:
(
Stored in directory: /home/nicolas/.cache/pip/wheels/94/02/70/6ba5403853459b4afcbb7c0348deece9bbc5853d9e0889dc97)even though the
RUNNER_TOOL_CACHEandAGENT_TOOLSDIRECTORYhave been set up.I took the idea of this workaround from #489 and #485
However I do not understand why the
RUNNER_TOOL_CACHEandAGENT_TOOLSDIRECTORYare not used.Beside, if i run
grep -R /opt/hostedtoolcache *from
_work/_toolI get a lot of match:I don't really know where pip gets the include path from, but I guess from one of the above files which clearly points to an invalid folder.
Thanks in advance for your help, let me know if you need more information.
The text was updated successfully, but these errors were encountered: