Create modelrepotest.mdx

docs.json

@@ -126,6 +126,7 @@
                   "serverless/development/dual-mode-worker"
                 ]
               },
+              "serverless/modelrepotest",
               {
                 "group": "Manage endpoints",
                 "pages": [

serverless/modelrepotest.mdx

@@ -0,0 +1,171 @@
+---
+title: "Model Repo testing"
+description: "Upload a model to Model Repo and deploy it to a Serverless endpoint."
+---
+
+<Note>
+Model Repo is currently in alpha and is available on Mac and Linux only. Windows support is coming soon.
+</Note>
+
+## Why use Model Repo
+
+Model Repo lets you upload your own models to private storage on Runpod and attach them directly to Serverless endpoints. Key benefits:
+
+- **Faster cold starts**: Models are pre-cached on the worker host rather than downloaded at runtime.
+- **No HuggingFace dependency**: Your models are stored in Runpod's infrastructure, so endpoints don't require an outbound download on every cold start.
+- **Private storage**: Models are stored in your account and are not accessible to other users.
+
+---
+
+## Manual testing
+
+### Prerequisites
+
+- Your email is feature-flagged for Model Repo access.
+- `jq` is installed for parsing JSON output.
+
+### Set environment variables
+
+Export the following before running any commands. **Make sure to set your actual API key — missing this is the most common source of auth errors later.**
+
+```bash
+export RUNPOD_API_URL="https://rest.runpod.io/v1"
+export RUNPOD_GRAPHQL_URL="https://api.runpod.io/graphql"
+export RUNPOD_API_KEY="your-api-key"   # replace with your actual API key
+
+export MODEL_NAME="model_name"  # unique name per test run — reusing the same name uploads a new version, not a new model
+export MODEL_PATH="/path/to/model"               # local path to the model files you want to upload
+```
+
+<Warning>
+`MODEL_NAME` must be unique for each test run. If you reuse the same name, the upload creates a new version of the existing model rather than a new model.
+</Warning>
+
+---
+
+### Step 1: Install runpodctl
+
+**Option A: Install via Homebrew (recommended)**
+
+```bash
+brew install runpod/runpodctl/runpodctl
+```
+
+**Option B: Build from source**
+
+```bash
+brew install go                              # install Go, required to build runpodctl
+git clone git@github.com:runpod/runpodctl.git
+cd runpodctl
+make                                         # builds the binary to ./bin/runpodctl
+```
+
+<Note>
+If you build from source, the binary is at `./bin/runpodctl`. Either run it with that path, or add `./bin` to your `PATH`. The steps below use `runpodctl` — adjust accordingly.
+</Note>
+
+---
+
+### Step 2: Upload the model
+
+```bash
+# --name: the name to register the model under in your repo
+# --model-path: local path to the model files
+# --create-upload: creates the upload session and transfers files
+runpodctl model add \
+  --name "$MODEL_NAME" \
+  --model-path "$MODEL_PATH" \
+  --create-upload
+```
+
+This outputs a JSON string listing all uploaded files.
+
+---
+
+### Step 3: Wait for the model to be hashed
+
+After upload, the model must be hashed by an asynchronous background process. This typically completes in a few minutes but can take up to 10–15 minutes.
+
+Poll until the `hash` field is non-null:
+
+```bash
+runpodctl model list --name "$MODEL_NAME" | jq -r '.[0].versions[0].hash'
+```
+
+While hashing is in progress, the command returns `null`:
+
+```
+% runpodctl model list --name "$MODEL_NAME" | jq -r '.[0].versions[0].hash'
+null
+```
+
+Once hashing is complete, it returns the hash value:
+
+```
+% runpodctl model list --name "$MODEL_NAME" | jq -r '.[0].versions[0].hash'
+71a311bdf0ca44119ed74dbef8cf573bc89b58cbc48a10fe508f756ebb1922dc
+```
+
+---
+
+### Step 4: Get your user ID and model hash
+
+```bash
+export USER_ID="$(runpodctl user | jq -r '.id')"        # your Runpod user ID
+export MODEL_HASH="$(runpodctl model list --name "$MODEL_NAME" | jq -r '.[0].versions[0].hash')"   # the hash from step 3
+```
+
+---
+
+### Step 5: Deploy a Serverless endpoint with the model attached
+
+```bash
+# --name: name for the endpoint
+# --hub-id: the Hub template to deploy
+# --gpu-id: GPU type
+# --workers-max: maximum number of active workers
+# --workers-min: minimum number of workers kept warm
+# --model-reference: attaches your model to the endpoint
+# --env: sets the model path on the worker
+# --min-cuda-version: works around a bug in runpodctl
+runpodctl serverless create \
+  --name "my_worker" \
+  --hub-id "cm8h09d9n000008jvh2rqdsmb" \
+  --gpu-id "AMPERE_24" \
+  --workers-max 3 \
+  --workers-min 1 \
+  --model-reference "https://local/$USER_ID/$MODEL_NAME:$MODEL_HASH" \
+  --env MODEL_NAME="/runpod/model-store/modelrepo-local/models/$USER_ID/$MODEL_NAME/$MODEL_HASH" \
+  --min-cuda-version "13.0"
+```
+
+<Note>
+`--model-reference` is only supported with `--hub-id` and GPU endpoints. It is repeatable if you need to attach multiple models to the same endpoint.
+</Note>
+
+---
+
+### Step 6: Verify the model is working
+
+Send a test request to confirm the endpoint is live and the model is accessible. Replace `ENDPOINT_ID` with the ID returned in the previous step:
+
+```bash
+curl -s -X POST "https://api.runpod.ai/v2/${ENDPOINT_ID}/runsync" \
+  -H "Authorization: Bearer $RUNPOD_API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{"input": {"prompt": "hello"}}' | jq
+```
+
+A successful response confirms the endpoint is running and the model is attached. If the request fails with an auth error, verify that `RUNPOD_API_KEY` is set correctly.
+
+If you prefer a graphical interface to curl, you can also send requests to the worker from the web UI.
+
+---
+
+### Step 7: Clean up
+
+Delete the endpoint after testing to stop accruing spend. Use the web UI or:
+
+```bash
+runpodctl serverless delete <endpoint-id>
+```