Deploy a model

• The model, which serves an endpoint: a URL you send requests to. It works with the OpenAI API format.
• A secret, which stores the access key that lets you call the endpoint.

​

What it does

• Dedicated hardware. The model runs on your environment cluster. No other customer shares it.
• OpenAI-compatible endpoint. OpenAI SDKs and tools work with it.
• Multiple deployments. You can deploy the same model to several environments, for example dev and prod.

​

Key words

• Balanced: a good mix of speed and cost
• Latency-optimized: faster replies for each request
• Throughput-optimized: handles more requests at once
• Custom: a special setup made for a specific need

​

Before you start

​

Deploy a model

1

Start a deployment

Go to AI Services → Models. Click + Deploy Model.You can also start from:

• A model’s page: click Deploy
• An environment: click ⋯, then Deploy Model

2

Name the deployment and pick a model

Under Deployment Details:

• Deployment Name: lowercase letters, numbers, and dashes. Up to 62 characters. Must be unique for this model.
• Model: search for and select the model.

3

Select an environment

Under Select Environment, pick where the model runs. Only Ready environments show in the list.No environment yet? Click + Create Environment. It opens in a new tab.

4

Select a profile

Under Select Profile, pick a profile.Each profile card shows the hardware it needs. The console checks this against your environment’s cluster.

If a card says No matching hardware in this environment, the cluster doesn’t have the right GPUs. Pick another profile, or use an environment on a different cluster.

5

Review and create

Check the Review your Deployment summary. Click Create Deployment.

6

Copy your access key

A Copy your Access Key window opens. It shows a new access key.

Copy the key now. You can’t see it again. Store it in a password manager or secrets store.

Click Copy Key & Create Deployment. This copies the key and starts the deployment. Nothing is created until you click it.

​

Deployment status

​

Call your model

1. Open the model’s page and go to Overview.
2. Find the Access endpoint card. If you have more than one deployment, pick one from the menu at the top of the card.
3. Copy the example in cURL, Python, or TypeScript.

export NSCALE_MODEL_ACCESS_KEY="<your-access-key>"

curl "<your-endpoint>/chat/completions" \
  -H "Authorization: Bearer $NSCALE_MODEL_ACCESS_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-name>",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

• <your-endpoint> with the Endpoint from the console
• <your-access-key> with the key you copied
• <model-name> with the model name shown on the model page

​

Lost your access key? Roll it

1. Open the model’s Deployments tab.
2. Click Roll Key on the deployment.
3. Copy the new key. Click Copy Key & Redeploy.

​

Delete a deployment

1. Open the model’s Deployments tab.
2. Click ⋯ on the deployment. Click Delete Deployment.
3. Confirm.