Mention if papermill is not installed in failed status #10
Conversation
|
This is tested and ready for review. |
nkaretnikov
added
area: user experience 👩🏻💻
needs: review 👀
|
I'm trying to test this but run into jupyter-server/jupyter-scheduler#519 - I see this issue both locally and on active deployments. I think we may want to pin jupyter-scheduler in the docker images in the meantime to avoid this issue on deployments. |
|
Thanks @nkaretnikov for having a look at this!! and thanks @krassowski for testing it out as well 🚀
|
|
Ok, pinning
I am retrying with Let's encrypt certificate now. |
|
@krassowski Hey Mike, any update here? Let me know if you need help testing. And yes, I did use this in my config, so didn't run into any issues with certs: certificate:
type: lets-encrypt
acme_email: <email>
acme_server: https://acme-v02.api.letsencrypt.org/directory |
|
I did try with lets-encrypt and I am still getting the SSL error. The full traceback extracted from the pod is: |
|
The date looks odd: |
|
It seems it would be coming from here: argo-jupyter-scheduler/argo_jupyter_scheduler/utils.py Lines 124 to 125 in 1d9f1d1 Edit: passing |
This is expected, it a placeholder filename (set to the Unix epoch), so we could find it. It's renamed later to the proper date by the I think the issue might be due to dependencies changing in your deployment since you had to pin the package. Here's how I personally test this. I push the current jupyterlab upstream image to my docker hub (not part of this script) and use it as a base image ( #!/usr/bin/env bash
set -euxo pipefail
BASE_TAG=$1
NEW_TAG=$2
ID=$(docker run -tid nkaretnikov/nebari-jupyterlab-papermill-error-8:${BASE_TAG})
docker exec $ID /bin/sh -c 'conda run --no-capture-output -n default /bin/sh -c "pip install git+https://github.com/nkaretnikov/argo-jupyter-scheduler@papermill-error-8"'
docker exec $ID /bin/sh -c 'conda run --no-capture-output -n default python -c "from argo_jupyter_scheduler import executor"'
docker commit $ID nkaretnikov/nebari-jupyterlab-papermill-error-8:${NEW_TAG}
docker push nkaretnikov/nebari-jupyterlab-papermill-error-8:${NEW_TAG}
docker stop $ID
echo "UPDATE TAG IN NEBARI CONFIGS!"
echo "jupyterlab: docker.io/nkaretnikov/nebari-jupyterlab-papermill-error-8:${NEW_TAG}"On my dockerhub, the upstream base tag is https://hub.docker.com/repository/docker/nkaretnikov/nebari-jupyterlab-papermill-error-8/tags You could try pulling these images and checking whether the package versions are different from yours. If you're going to use this script yourself, you might need to update the GitHub and DockerHub repos. |
Yes, I think this is it. But specifically I think this is related to It would be |
|
Small progress here - after adding |
|
@krassowski I've updated and re-tested the code with a notebook containing parens and spaces. PTAL. |
nkaretnikov
removed
needs: review 👀
Reference Issues or PRs
Fixes #8.
This PR adds additional info to the failed status message if papermill is not installed. This is implemented by capturing the status code of the papermill command and writing it to a file that's accessible by all argo steps of this job, via a shared filesystem in the staging area.
What does this implement/fix?
Put a
xin the boxes that applyTesting
Testing checklist:
papermill_status.txt= 0papermill_status.txt= 1papermill_status.txt= 127Tested on Azure, Nebari 2024.5.1.
Documentation
Access-centered content checklist
Text styling
H1or#in markdown).Non-text content
Any other comments?