Managed Kubernetes

​

Summary

1. Create the cluster in the console. You pick a network, a Kubernetes version, and the machines to run on.
2. Wait for its status to change from Provisioning to Provisioned.
3. Run the command on the cluster’s Access your cluster card to connect kubectl.
4. Run kubectl get nodes to check the connection, then deploy your workloads.

​

Availability

​

Key concepts

​

Compute pools and reservation pools

​

Create a cluster

1

Set up your cluster

• Name: 3–50 lowercase letters, digits, and dashes. It must start with a letter or digit, can’t end with a dash, and can’t contain nscale. You can’t rename a cluster later.
• Choose your Regional VPC: the network the cluster joins, which also sets its region. You can’t change it later. To create a VPC without leaving the create flow, click Add new VPC.

If you come back and change the VPC, any node types you’ve picked are cleared, because each region offers different machines.

Node labels and initial taints can’t be set in the console. Use the API or the Nscale CLI instead.

2

Configure Kubernetes

Setting	Default	What it does	Change later?
NKS Version	Newest available	The Kubernetes version and add-ons the cluster runs. Only current versions offered in the VPC’s region are listed.	Upgrade only
Hardware Profile	On	Installs the drivers and plugins GPUs and the HPC network need. If you turn it off, you install and manage them yourself.	Yes, in Settings
Node Health Monitoring	On	Detects and reports problems with your nodes.	Yes, in Settings
Public API Endpoint	Off	Makes the cluster’s API reachable from the internet. Sign-in is still required. When it’s off, you can only reach the API from inside the cluster’s VPC.	Yes, with Manage Access
Allowed Address Ranges	Empty	Only shown when the public endpoint is on. Limits which IPv4 ranges can reach the API, for example 10.0.0.0/8. Leave it empty to allow any address.	Yes, with Manage Access

Nscale manages the control plane over its own network, so it never needs the public endpoint.

3

Add node pools

Name the pool and pick a node type from the table. The pool name must be unique in the cluster and follow the same rules as the cluster name. A node type is grayed out if the selected NKS version doesn’t support it.The rest of the step depends on the node type you picked:

• Compute pool
• Reservation pool

Under Configure your Pool, set Number of Replicas: how many nodes the pool runs.

1. Under Set up your reservation, pick a reservation in Select reservation for this node pool. Only provisioned reservations for the chosen node type are listed.
2. Under Configure your placement, set Host Count: how many of the reservation’s hosts the pool uses. The limit is the reservation’s free hosts, minus any that other pools in this cluster already use.
3. Pick a Placement Policy. Pack keeps hosts close together for fast communication. Spread spreads them out so one failure affects fewer of them. See Placement policy.

The Network row shows the cluster’s VPC, which the pool uses too. Change VPC takes you back to step 1, and changing the VPC clears the node types you’ve picked.

To add more pools, click Add Another Pool once the current one is complete. Only one pool is open at a time: Collapse folds it into a summary row, and Remove deletes it. A pool you haven’t changed from its defaults is ignored.

4

Review your cluster

Check the VPC and each node pool, then click Create Cluster. If your compute pools use GPUs, the review shows how much of your GPU quota they need.

​

Access your cluster

nscale k8s kubeconfig get \
  --org <organization-id> \
  --cluster <cluster-id> \
  --merge

nscale k8s kubeconfig get --org <organization-id> --cluster <cluster-id> -o my-cluster.yaml
export KUBECONFIG=$PWD/my-cluster.yaml

kubectl get nodes

​

View a cluster

​

Overview

​

Manage API access

​

Node Pools tab

​

Settings tab

• Cluster info: the cluster’s name and project. Names can’t be changed.
• Cluster configuration: click Edit to upgrade the NKS version or turn Hardware Profile and Node Health Monitoring on or off, then click Save Changes. The VPC is shown but can’t be changed.
• Delete: see Delete a cluster.

​

Upgrade a cluster

​

Manage node pools

​

Add a node pool

​

Edit a node pool

​

Delete a node pool

​

View a node pool

• Compute pool: the Nodes table lists each node with its private IP and Power Status. Use Stop or Start in the row, or Reboot Node from the row menu.
• Reservation pool: the table lists the servers of the pool’s placement, once NKS has taken its hosts from the reservation. Use Stop in the row, or Reboot Server (hard reboot) or Soft Reboot Server from the row menu. A stopped server shows Reboot to turn it back on.

​

Delete a cluster

​

Cluster and node pool status

​

Permissions

​

Quotas

​

How NKS runs your cluster

• Always available. The control plane runs as three copies on separate machines, so the cluster keeps working if one fails.
• Careful maintenance. When NKS replaces a machine, it starts the new one and waits for it to be healthy before removing the old one.
• Your workloads keep running. If the control plane is briefly unavailable, you can’t deploy or change things, but workloads already running on healthy nodes keep going. Applications that call the Kubernetes API all the time can still notice.
• Certificates. Nscale renews the control plane’s certificates without changing the cluster’s certificate authority (CA), so your clients keep trusting the cluster. Replacing the cluster CA isn’t supported.
• Secrets. NKS doesn’t encrypt Kubernetes Secrets at rest.

​

Who manages what

​

Advanced configuration

​

Use your own identity provider

"apiServer": {
  "authentication": {
    "externalIssuers": [
      {
        "issuerURL": "https://idp.example.com",
        "audiences": ["acme-tenant-api"],
        "usernameClaim": "sub",
        "usernamePrefix": "acme:",
        "groupsClaim": "groups",
        "groupsPrefix": "acme:"
      }
    ]
  },
  "authorization": {
    "clusterRoleBindings": [
      {
        "clusterRole": "cluster-admin",
        "subjects": [{ "kind": "Group", "name": "acme:platform-admins" }]
      }
    ]
  }
}

• It can’t be changed later. To change the provider or role bindings, create a new cluster. Pick an issuer CA that will outlast the cluster.
• Add at least one role binding. People only get the access you bind. Someone with a valid token but no binding is refused everything.
• Your provider affects the cluster. The cluster fetches your provider’s signing keys over HTTPS. Short outages are fine, because the keys are cached. A long outage, or a new key the cluster can’t fetch, stops people signing in and can make the API report as not ready.

​

Monitor the control plane

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: prometheus-control-plane-metrics
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nks-metrics-component-scraper
subjects:
  - kind: ServiceAccount
    name: prometheus
    namespace: monitoring

• Scrape from inside the cluster. Only addresses in the cluster’s subnet can reach the metrics, even over peered networks.
• Create your own binding. Don’t edit the existing nks-metrics-component-scraper binding. Nscale owns it and undoes changes.
• Ports. kube-apiserver on 6443, kube-controller-manager on 10257, and kube-scheduler on 10259. Each copy has its own name: <instance>.nks-control-plane-metrics.kube-system.svc.cluster.local.
• Scrape by name, not by IP address. A replacement machine can reuse an old IP address. Scraping by name shows a replacement as a gap, instead of silently mixing two machines’ data. Keep only endpoints marked ready, and give new copies time to start before alerting.
• Adding up metrics. Sum kube-apiserver counters across copies. Only one controller manager and one scheduler are active at a time, so take the maximum of their gauges, such as scheduler_pending_pods, instead of summing them.
• Not available. etcd, node, and host metrics.

​

Forward DNS zones to your resolvers

apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns-custom
  namespace: kube-system
data:
  corp.example.com.server: |
    corp.example.com:53 {
        errors
        cache 30
        forward . 10.0.0.53
    }

• Key names must end in .server. Other keys, such as the .override keys some providers use, are silently ignored.
• No restart needed. Changes apply within about two minutes.
• Your nodes must reach your DNS servers on port 53, over both UDP and TCP.
• Pods only. This doesn’t change DNS on the nodes themselves, or for Pods with dnsPolicy: Default.
• Leave the cluster’s own zones alone. Don’t add blocks for cluster.local, in-addr.arpa, ip6.arpa, or the root zone (.). The first three break Service and reverse lookups. A root block stops CoreDNS from starting the next time a CoreDNS Pod restarts.
• Check the logs after each change. If CoreDNS can’t read your file, it keeps using the last working one. Look for Corefile parse failed in kubectl logs -n kube-system -l k8s-app=kube-dns, and fix it before a CoreDNS Pod restarts.

​

Workload credentials

• Tokens last up to 24 hours. A request for longer is shortened to 24 hours without an error, so check the expiry you get back. Tokens from kubectl create token aren’t renewed, so automation outside the cluster must request a new one before it expires.
• Legacy token Secrets are empty. A kubernetes.io/service-account-token Secret is never filled in. Use the TokenRequest API or projected tokens instead.

​

Troubleshooting

​

Related resources