Gradio App for Stable Diffusion on Ray Serve

This example demonstrates how to deploy a Gradio web application that generates images using Stable Diffusion, running on a Ray cluster managed by Ray Serve. The application allows users to input text prompts and receive generated images in response, leveraging GPU resources for efficient inference.

Prerequisites

Kubernetes cluster with GPUs (L40 or above recommended)
GPU Operators installed and configured (e.g., NVIDIA device plugin)
KubeRay operator installed and configured
kubectl configured to access the cluster
Hugging Face access token with read permissions for stabilityai/stable-diffusion-3-medium-diffusers

Step 1: Create the Hugging Face token secret

kubectl create secret generic hf-secret --from-literal=token=<YOUR_HF_TOKEN>

Step 2: Deploy

kubectl apply -f https://gist.github.com/karthik-aion/5260d2409b6fad31af60f5f2c8ec1235/raw

This creates:

ConfigMap (stable-diffusion-gradio-code): contains the Stable Diffusion Gradio app code
RayService (stable-diffusion-gradio): Ray cluster with a head node and GPU worker(s), running the app via Ray Serve

Step 3: Check status

Wait for the RayService to become ready:

kubectl get rayservice stable-diffusion-gradio

Check pod status:

kubectl get pods -l ray.io/cluster=$(kubectl get rayservice stable-diffusion-gradio -o jsonpath='{.status.activeServiceStatus.rayClusterName}')

Step 4: Access the Gradio UI

The service is exposed via NodePort on port 31770:

http://<cluster-id>.groundcontrol-aion.xyz:31770

To find the node port, if needed:

kubectl get svc stable-diffusion-gradio-serve-svc -o jsonpath='{.spec.ports[0].nodePort}'

To find the cluster ID: “Screenshot Placeholder” Alternatively, port-forward to access locally:

kubectl port-forward svc/stable-diffusion-gradio-serve-svc 8000:8000

Then open http://localhost:8000.

Cleanup

kubectl delete -f https://gist.github.com/karthik-aion/5260d2409b6fad31af60f5f2c8ec1235/raw
kubectl delete secret hf-secret

Getting started

Security

Cookbooks

Monitoring

FAQs

Guides

Others

Gradio App for Stable Diffusion on Ray Serve

Prerequisites

Step 1: Create the Hugging Face token secret

Step 2: Deploy

Step 3: Check status

Step 4: Access the Gradio UI

Cleanup

​Prerequisites

​Step 1: Create the Hugging Face token secret

​Step 2: Deploy

​Step 3: Check status

​Step 4: Access the Gradio UI

​Cleanup

Prerequisites

Step 1: Create the Hugging Face token secret

Step 2: Deploy

Step 3: Check status

Step 4: Access the Gradio UI

Cleanup