Serve Stable Diffusion using Ray

​

Step 1: Create a Kubernetes cluster with GPUs

1. In the left navigation, click Kubernetes.
2. Click Create K8s Cluster.
3. Select a region with GPU availability that fits your needs.
4. Create or select a project.
5. Configure the cluster (cluster name, nodes, image, SSH key, version, etc.).
6. Click Launch and wait until the cluster status is ready/active.
7. Copy KubeConfig from the cluster card/details and setup the kubeconfig file.

Recommended: Have at least one GPU-capable node available for Stable Diffusion serving.

​

Step 2: Install required Marketplace apps

​

2.1 Install NVIDIA GPU Operator

1. Open NVIDIA GPU Operator → View Details
2. Choose Existing Cluster = your cluster
3. Click Deploy Application

kubectl get pods -n gpu-operator

​

2.2 Install KubeRay Operator

1. Open KubeRay Operator → View Details
2. Choose Existing Cluster
3. Click Deploy Application

kubectl get crd | grep ray

​

2.3 Install Ray Cluster

1. Open Ray Cluster → View Details
2. Choose Existing Cluster
3. Click Deploy Application

kubectl get ray-cluster -A
kubectl get pods -A | grep -i ray

​

Step 3: Install the Stable Diffusion RayService

kubectl apply -f https://raw.githubusercontent.com/ray-project/kuberay/master/ray-operator/config/samples/ray-service.stable-diffusion.yaml

kubectl get rayservices -A
kubectl describe rayservices.ray.io stable-diffusion

​

Step 3: Forward the Serve port

kubectl get services

kubectl port-forward svc/stable-diffusion-serve-svc 8000

​

Step 4: Run the Gradio client

curl -L -o diffusion-run.py "https://gist.githubusercontent.com/rb-aion/f01e3abecaae3346fd3ef94122ea0caf/raw/5f46b75360c262963a535f6fbd954a56f2055592/diffusion-run.py"

python -m pip install --upgrade pip
python -m pip install gradio requests

python diffusion-run.py