Skip to main content
Reuse your GPUs across multiple environments. You reserve GPU capacity once, as an environment cluster. Several environments, such as dev and staging, can then share that capacity. You deploy AI models into each environment, and each deployment has an OpenAI-compatible endpoint.
Beta: Environments is in beta for private cloud organizations. To join the waitlist, contact support.

​
What you get

  • Shared GPUs. Several environments can use the same cluster.
  • Network isolation. Each environment has its own network rules. By default, environments can’t reach each other over the network. Environments on the same cluster share its GPUs, so their compute isn’t isolated. To give an environment its own GPUs, put it on its own cluster.
  • Model deployment from the console. Select a model, an environment, and a profile. The console creates an OpenAI-compatible endpoint.
  • Managed setup. Nscale installs the GPU drivers, the model runtime, and the public gateway.

​
The three parts

There are three things to know. You create them in this order. How they connect:
  • One cluster can hold many environments.
  • One environment can hold many deployments.
  • A deployment always lives in one environment.
  • Deploying a model from the console sets up two things in the environment: the model, which serves the endpoint, and a secret that stores its access key. The console lists them together as one deployment.

​
Before you start

Check that you have:
  • A project
  • A VPC in the region you want
  • Enough GPU quota for the cluster you want, or a reservation with available hosts if you plan to use bare-metal node types
  • The right permissions. Creating clusters needs an organization-level role. Ask your organization owner if a button is missing. The console hides buttons you can’t use.

​
Get started

In the console, open your project and go to Platform Services → Environments. The page has two tabs:
  • Overview: your environments
  • Environment Clusters: the clusters your environments run on
When you open it for the first time, a Get started with Environments checklist walks you through three steps:
  1. Create an environment cluster. Provisioning takes up to 90 minutes.
  2. Create an environment on the cluster.
  3. Deploy a model into the environment, from AI Services → Models.
The Get started with Environments checklist, with three steps: Create Environment Cluster, Create Environment, and Deploy a Model.

The Get started checklist

​
Environment clusters

An environment cluster is a set of GPU nodes in one region, dedicated to your workloads. Nscale places the nodes to optimize the network topology between them. Your environments and models run on it. Nscale runs the cluster as a managed inference service. Nscale provisions, operates, and updates the nodes, the network, the GPU drivers, and the gateway that serves your model endpoints.

​
What a cluster does

  • Runs your environments. Every environment runs on a cluster. One cluster can run many environments.
  • Dedicated. No other customer runs on it.
  • Your choice of hardware. You pick the VPC, which sets the region, and the node types.

​
Cluster terms

​
Create an environment cluster

You need an organization-level role to create clusters. If you can’t see the button, ask your organization owner.
1

Open the Environment Clusters tab

Go to Platform Services → Environments and open the Environment Clusters tab.Click Create Environment Cluster. If you already have clusters, click + Add Cluster above the table.
Environment Clusters tab with the Create Environment Cluster button.
2

Name the cluster

Under Environment Cluster Details, enter a Name.Name rules:
  • 3 to 49 characters
  • Lowercase letters, numbers, and dashes only
  • Start and end with a letter or number
  • Can’t contain nscale
  • Must be unique in the project
The Environment Cluster Details step with the Name field.
3

Select a Regional VPC

Under Select Regional VPC, pick a VPC from Choose your Regional VPC. Each VPC shows its region. To create a VPC, click + Add new VPC.
You can’t change the VPC later. The VPC sets the cluster’s region, and the region decides which node types you can use.
The Select Regional VPC step with the VPC dropdown and the Add new VPC button.
4

Add node pools

Under Add Node Pools, set up at least one pool:
  • Pool Name: same rules as the cluster name. Must be unique in the cluster.
  • Node Type: pick a node type from the table. Check the GPUs and VRAM columns. The Type column shows VM or BM (bare metal).
The remaining fields depend on the node type:
  • VM node type: under Configure your Pool, set the Number of Replicas.
  • BM node type: the field label changes to Placement Pool Name. Under Set up your reservation, select the reservation to take hosts from. Under Configure your placement, set the Host Count. The maximum is the number of hosts available in the reservation. The pool uses the cluster’s VPC.
To add more pools, click + Add Another Pool.
Pick the node type with care. You can’t change it after the cluster is created. Choose hardware that fits the models you plan to run.
The Add Node Pools step with a B200 NVLink node type selected, a reservation selected, and the Host Count counter.
5

Review and create

Under Review Environment Cluster, check the cluster, VPC, and node pool summary.If the pools go over your GPU quota, the review shows a message and you can’t create the cluster. Remove a pool or lower its node or host count.Click Create Environment Cluster.
The Review Environment Cluster step with the cluster, VPC, and node pool summary.
What happens next: The console takes you back to the Environment Clusters tab. Your cluster shows Provisioning.
Provisioning can take up to 90 minutes. You can leave the page. When the status shows Ready, you can create an environment on it.

​
Cluster status

The Health column shows if a ready cluster is working well: Healthy, Degraded, Unhealthy, or Unknown.

​
View a cluster

Click a cluster in the list to open it. You’ll see three tabs:
  • Overview: node pools, assigned environments, status, health, cluster ID, and region
  • Node Pools: each pool, its node type, and node count
  • Assigned Environments: environments that run on this cluster
The environment cluster detail page showing the Overview tab.
You can’t edit a cluster or add pools after it is created. To change the hardware, create a new cluster.

​
Cancel a cluster that is still provisioning

If you made a mistake, you can stop it.
  1. In the cluster list, click ⋯ on the cluster.
  2. Click Cancel Current Request.
  3. Click Cancel request to confirm.
Any work done so far is lost. If you try again, it starts from the beginning.

​
Delete an environment cluster

Deleting a cluster is permanent. Environments that use it lose their capacity. Any model running on it stops.
1

Delete the environments first

Delete every environment listed under Assigned Environments. See Delete an environment.
2

Delete the cluster

In the cluster list, click ⋯ on the cluster, then click Delete Environment Cluster.
3

Confirm

Type the cluster name and confirm. The status changes to Deleting. The cluster disappears from the list when it is done.
You can’t delete a cluster while a request is still running. Wait for it to finish, or cancel it first.

​
Environments

An environment is a named, network-isolated space on an environment cluster. You deploy models into it. For example, you can have dev, staging, and prod environments on the same cluster.

​
What an environment does

  • Network isolation. By default, a model in dev can’t reach a model in prod over the network.
  • Grouped deletion. Deleting an environment also deletes every deployment in it.

​
Create an environment

You need an environment cluster with the status Ready.
1

Open Environments

Go to Platform Services → Environments. Click + Create Environment.
The Environments page with the Create Environment button.
2

Name the environment

Under Environment Details, enter a Name. For example, dev or prod.Name rules:
  • Up to 62 characters
  • Lowercase letters, numbers, and dashes only
  • Start and end with a letter or number
  • Must be unique in the project
Use a name that shows the purpose, like dev, test, or prod. You can’t rename it later.
3

Select a cluster

Under Select cluster, pick the environment cluster to run on.The table shows each cluster’s GPU Type, GPUs, Region, Status, and Health. Pick one that is Ready and has the GPUs your models need.
The Create Environment form with the Name field and the cluster table.
4

Review and create

Check the Review your Environment card. Click Create Environment.
What happens next: The console takes you back to the Environments list. Your environment shows Creating. When it shows Ready, you can deploy a model into it.

​
Environment status

You can only deploy models into a Ready environment.

​
View an environment

Click an environment in the list to open it. You’ll see three tabs:
  • Overview: its status, cluster, and latest deployments
  • Clusters: the environment clusters it runs on
  • Deployments: every deployment in it, with its service, model, profile, and status
The environment detail page showing the Overview tab.
To deploy a model from here, click ⋯ and choose Deploy Model.

​
Delete an environment

Deleting an environment is permanent. Every model deployed into it is removed. Their endpoints stop working.
1

Open the menu

In the Environments list, click ⋯ on the environment. Click Delete Environment.You can only delete an environment when it is Ready or Error.
2

Confirm

Type the environment name and click Delete.
The status changes to Deleting. The environment disappears from the list when it is done.

​
Next steps

Deploy a model

Run a model in your environment and call its OpenAI-compatible endpoint.