Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kaito workspace-controller support for Karpenter nodeclaim #339

Open
philwelz opened this issue Apr 4, 2024 · 1 comment
Open

Kaito workspace-controller support for Karpenter nodeclaim #339

philwelz opened this issue Apr 4, 2024 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@philwelz
Copy link

philwelz commented Apr 4, 2024

Describe the bug

Kaito workspace controller seems to be only compatible with Karpenter version prior v0.33.0 as they deprecated the machine CRD (karpenter.sh/v1alpha5) in that release and the controller seems to relay on that machine CRD for spinning up the workspace.

Steps To Reproduce

  1. Create an AKS cluster with NAP enabled
  2. Install Kaito workspace-controller helm install workspace kaito/workspace --namespace workspace --create-namespace
  3. Add an workspace kubectl apply -f https://raw.githubusercontent.com/Azure/kaito/main/examples/inference/kaito_workspace_phi-2.yaml
  4. Nothing happens, run kubectl api-resouces and see there is no CRD machines karpenter.sh/v1alpha5

Expected behavior

As Kaito per docs supports node provisioning controllers that supports Karpenter-core APIs, it should also support the new Karpenter CRDs for nodeClaims (machine CRD was deprecated in December). In best-case scenario Kaito (workspace-controller) should also run on AKS with NAP enabled and should be aware of the Karpenter version (or which CRD is available: machines vs nodeclaim).

Logs
Controller starts and prints this:

2024-04-04T10:07:49Z INFO Starting EventSource {"controller": "workspace", "controllerGroup": "kaito.sh", "controllerKind": "Workspace", "source": "kind source: *v1alpha5.Machine"}

Environment

  • Kubernetes version (use kubectl version): 1.28.5
  • Install tools: AKS NAP (Karpenter)
### Support Karpenter Tasks
- [x] Add NodeClaim API #362
- [x] Update workspace controller to support NodeClaim #366
- [x] Update pipelines #569
- [x] update chart #570
- [x] Add e2e tests  #375
- [ ] Add label to the nodeclaims to identify the node created by kaito
- [ ] Update doc
@philwelz philwelz added the bug Something isn't working label Apr 4, 2024
@Fei-Guo
Copy link
Collaborator

Fei-Guo commented Apr 4, 2024

Thanks for filing the issue. We are working towards supporting Karpenter new APIs.

@helayoty helayoty added enhancement New feature or request and removed bug Something isn't working labels Apr 30, 2024
@Fei-Guo Fei-Guo moved this to In Progress in Kaito Roadmap May 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants