AI starter kit chart #579
AI starter kit chart #579
Conversation
k8s-ci-robot
added
do-not-merge/work-in-progress
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: volatilemolotov The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
size/XXL
|
Here is the initial PR, currently in draft state. Think we should be able to send it for reviews |
Removed the example values and ci folder. Hope makefile can stay, it can be useful |
| --mount --mount-string="/tmp/models-cache:/tmp/models-cache" | ||
| ``` | ||
|
|
||
| 2. **Install the chart:** |
There was a problem hiding this comment.
better to clarify the path this command should be run within
|
|
||
| 2. **Install the chart:** | ||
| ```bash | ||
| helm install ai-starter-kit . \ |
There was a problem hiding this comment.
Looks like helm dependency build is needed before install
|
|
||
| 1. **Start minikube with persistent storage:** | ||
| ```bash | ||
| minikube start --cpus 4 --memory 15000 \ |
There was a problem hiding this comment.
Missing a step to create the models-cache folder? mkdir -p /tmp/models-cache
There was a problem hiding this comment.
Yes. Also decided to move to $HOME due to some linux distributions mount /tmp as tmpfs which is i think half of size of RAM
| @@ -0,0 +1,104 @@ | |||
| { | |||
There was a problem hiding this comment.
Just demonstrates that it can use preloaded model in memory. Just a sanity check if the both preload and jupyterhub work
| @@ -0,0 +1,798 @@ | |||
| { | |||
There was a problem hiding this comment.
Please remove all the "outputs" blob in this file
| -f values.yaml | ||
| ``` | ||
|
|
||
| 3. **Access JupyterHub:** |
There was a problem hiding this comment.
Which notebooks are runnable in the minikube environment? I assume the only one works is chat_bot? Can we add description in each notebook to detail in what environment it's runnable?
There was a problem hiding this comment.
I will check out which exactly can be run in Minikube and note accordingly
There was a problem hiding this comment.
All should work. Multi agent ray one needs ray enabled but we are not enabling it by default
| helm install ai-starter-kit . \ | ||
| --set huggingface.token="YOUR_HF_TOKEN" \ | ||
| -f values.yaml \ | ||
| -f values-gke.yaml |
There was a problem hiding this comment.
@janetkuo do you have concern to include the GKE specific setup in the example? Do you think we should remove this all?
There was a problem hiding this comment.
Yes, the example should be as generic as possible so that it's applicable to most Kubernetes clusters. It might be challenging at times for some platform-specific setup, and in that case we should call it out and mention alternatives.
There was a problem hiding this comment.
In our case, there are some platform specific setup such as specify the GPU in GKE cloud.google.com/gke-accelerator: nvidia-l4. What is your suggestion to handle this?
| helm install ai-starter-kit . \ | ||
| --set huggingface.token="YOUR_HF_TOKEN" \ | ||
| -f values.yaml \ | ||
| -f values-gke.yaml |
There was a problem hiding this comment.
For all the helm install on GKE autopilot instructions here I got error admission webhook "warden-validating.common-webhooks.networking.gke.io" denied the request: GKE Warden rejected the request because it violates one or more constraints. Violations details: {"[denied by autopilot-persistent-volume-limitation]":["Persistent Volume sources can only be of types [csi nfs gcePersistentDisk] within Autopilot."]}
There was a problem hiding this comment.
Ill check what is going on with GKE setup. Lets see if it even should be here
k8s-ci-robot
added
cncf-cla: no
5 participants
This PR adds a AI starter kit helm chart that aims to provide a out of the box development solution for AI workloads. Uses RayServe, Ollama or Ramalama to run the LLMs and JupyterHub for the development.