Fine Tuning

Documentation Index

Fetch the complete documentation index at: https://docs.nscale.com/llms.txt

Use this file to discover all available pages before exploring further.

​

What is Fine-tuning?

• Selecting a base model: Choose a pre-trained model as the starting point.
• Providing training and validation datasets: Supply data that represents your target use case.
• Launching a fine-tuning job: Start the process, which will train the model on your data.
• Monitoring job status: Track progress, review logs, and handle errors.
• Retrieving the tuned model: Once complete, access your custom model for inference.

​

Prerequisites

• Service Token (JWT): A valid JWT is required to authenticate your requests. For instructions, please see our guide on how to create a service token.
• Organization ID: You can find your Organization ID by navigating to Settings → Organisation in the Nscale platform.
• A Fine-Tuning Dataset: The dataset from which to train your new model. Please follow the Dataset Creation Guide.
• Sufficient Credit: Enough account credit to cover the job’s cost. Each fine-tuning job has a $2.00 minimum spend, so make sure you have enough credit before starting. You can estimate the cost using the Fine-tuning job cost estimator. We include $5 free credit when you sign up to help you get started.

​

Create a Fine-tuning Job

​

Step 1 - Select a Base Model

curl -X GET "https://fine-tuning.api.nscale.com/api/v1/organizations/$ORGANIZATION_ID/base-models" \
  -H "Authorization: Bearer $NSCALE_API_TOKEN" \
  -H 'Accept: application/json'

​

Step 2 - Create a Fine-tuning Job

curl -X POST "https://fine-tuning.api.nscale.com/api/v1/organizations/$ORGANIZATION_ID/jobs" \
  -H "Authorization: Bearer $NSCALE_API_TOKEN" \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json' \
  -d '{
    "name": "job name",
    "base_model_id": "<Base_Model_ID>",
    "dataset": {
      "id": "<Dataset_ID>",
      "prompt_column": "<Your-Prompt-column-name>",
      "answer_column": "Your-Answer-column-name"
    },
    "hyperparameters": {
      "n_epochs": 1,
      "evaluation_epochs": 1,
      "warmup_epochs": 0,
      "batch_size": 4,
      "learning_rate": 0.00001,
      "weight_decay": 0.0,
      "best_checkpoints": false,
      "mask_prompt_labels": true,
      "lora": {
        "enabled": true,
        "r": 8,
        "alpha": 16,
        "dropout": 0.01,
        "trainable_modules": []
      }
    }
  }'

​

Fine-tuning Job Configuration

​

Base Configuration

• name (string, required): The name of your fine-tuning job.
• base_model_id (string, required): The id of the foundation model to fine-tune. See the model id in the base model section for available options.
• dataset: Configuration for your training data. Details for creating a dataset can be found in the datasets
  • id: (string, required): The id of the dataset to use for training.
  • answer_column: (string, required): Column name containing expected outputs.
  • prompt_column: (string, optional): Column name containing input prompts.

​

Hyperparameters

• n_epochs (integer, optional): Number of complete passes through the training dataset. Default: 1, Min:1, Max:20.
• n_evals (integer, optional): Number of evaluation runs on validation data. Default: 0, Min:0, Max:100.
• batch_size (integer, optional): Number of training examples processed in one iteration. See the base model section for supported batch sizes for each model. Default: 4
• learning_rate (float, optional): Controls how much to adjust the model in response to errors. Default: 0.00001, Min: 0.00000001, Max: 0.01
• warmup_ratio (float, optional): The percent of steps at the start of training to linearly increase the learning rate. Default 0.0, Min: 0.0, Max: 1.0
• weight_decay (float, optional): Weight Decay parameter for the optimizer. Default: 0.0, Min: 0.0.
• best_checkpoints (boolean, optional): Determines if the checkpoint of the epoch exhibiting the best validation metric should be saved. When set to false, the checkpoint after the last epoch is saved. Default: false.
• mask_prompt_labels (boolean, optional): Whether to exclude prompt tokens from the training loss (mask labels for the prompt). When set to true, only answer tokens contribute to the loss. Recommended for instruction-style datasets. Default: false.
• lora (object, optional): Configuration for LoRA fine-tuning.

• enabled (boolean, optional): Whether to use LoRA instead of full fine-tuning. Default: true.
• r (integer, optional): Rank of LoRA adaptations. Default: 8, Min:1, Max:64.
• alpha (integer, optional): Scaling factor for LoRA updates. Default: 8, Min:1, Max:64.
• dropout (float, optional): The dropout probability for Lora layers. Default: 0.0, Min: 0.0, Max: 1.0.
• trainable_modules ([strings], optional): A list of LoRA trainable modules, separated by a comma. Default: all-linear(using all trainable modules). Trainable modules for each model are:
  • All models: k_proj, up_proj, o_proj, q_proj, down_proj, v_proj, gate_proj

​

Check the Status of your Jobs

curl -X GET "https://fine-tuning.api.nscale.com/api/v1/organizations/$ORGANIZATION_ID/jobs" \
  -H "Authorization: Bearer $NSCALE_API_TOKEN" \
  -H 'Accept: application/json'

​

Monitor Training Metrics for your Job

curl -X GET "https://fine-tuning.api.nscale.com/api/v1/organizations/$ORGANIZATION_ID/jobs/$JOB_ID/metrics" \
  -H "Authorization: Bearer $NSCALE_API_TOKEN" \
  -H 'Accept: application/json'

​

Export your Fine-tuned Model

​

From the Console

1. Open console.nscale.com and go to AI Services → Fine-Tuning.
2. Select the completed job you want to export. Export buttons are only enabled once the job status is completed.
3. On the Access your model card, choose one of:

​

Via the API

curl -X POST "https://fine-tuning.api.nscale.com/api/v1/organizations/$ORGANIZATION_ID/jobs/$JOB_ID/exports/hugging-face" \
  -H "Authorization: Bearer $NSCALE_API_TOKEN" \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json' \
  -d '{
    "huggingface_user_id": "your-hf-username",
    "huggingface_access_token": "hf_xxx",
    "huggingface_repository_id": "your-hf-username/your-model-repo"
  }'

curl -L -X GET "https://fine-tuning.api.nscale.com/api/v1/organizations/$ORGANIZATION_ID/jobs/$JOB_ID/download" \
  -H "Authorization: Bearer $NSCALE_API_TOKEN" \
  -o fine_tuned_model.tar.gz

​

With the Python client

from nscale import FineTuningClient

with FineTuningClient(api_token=token, organization_id=org_id) as client:
    # 1) Download the fine-tuned model archive (.tar.gz)
    client.jobs.download_model(
        job_id="your-job-id",
        output_path="exports/fine_tuned_model.tar.gz",
    )

    # 2) Download the dataset source files used to train the model
    dataset = client.datasets.get("your-dataset-id")
    client.files.download_file(
        file_id=dataset["training_file_id"],
        output_path="exports/train.csv",
    )
    if dataset.get("validation_file_id"):
        client.files.download_file(
            file_id=dataset["validation_file_id"],
            output_path="exports/validation.csv",
        )

​

Fine-tuning Job Cost Estimator

• The number of token in your training and validation files.
• The number of traning and evaluation epochs.
• The base model you are using.

​

Example: PubMedQA cookbook