Tuning dataset for reinforcement learning fine-tuning

A reinforcement learning fine-tuning job is driven by a training dataset and an optional validation dataset. Each dataset is a JSON file stored in Cloud Storage, where every line is a single tuning example.

This page describes the dataset schema, how to use the references field for reward computation, and the per-modality dataset limits enforced by the service.

Dataset schema

Each line of the JSONL dataset file follows this schema:

{
  "systemInstruction": {
    "role": string,
    "parts": [
      {
        "text": string
      }
    ]
  },
  "contents": [
    {
      "role": string,
      "parts": [
        {
          // Union field data: text or fileData.
          "text": string,
          "fileData": {
            "mimeType": string,
            "fileUri": string
          }
        }
      ]
    }
  ],
  "references": {
    string: string,
    ...
  }
}

The contents and systemInstruction fields use the same schema as the Gemini Enterprise Agent Platform generateContent API. systemInstruction accepts text only. contents can include text or file data (image, audio, or video), subject to the dataset limits.

The references field

The references field is specific to reinforcement learning fine-tuning. It stores any metadata that your reward function needs to score the model's response. It accepts a string-to-string dictionary.

For example, a string-matching reward that compares the model's answer to a known ground truth could use a dataset line like the following:

{
  "systemInstruction": {
    "role": "system",
    "parts": [{ "text": "You are a helpful math tutor." }]
  },
  "contents": [
    {
      "role": "user",
      "parts": [{ "text": "What is 17 * 23? Put your final answer in $\\text{your answer}$." }]
    }
  ],
  "references": {
    "ground_truth_answer": "391"
  }
}

Your reward function may read references["ground_truth_answer"] and compares it to the value the model produces. For details on how each reward type consumes references, see the Reward functions page.

Dataset limits

The following limits apply to reinforcement learning fine-tuning datasets:

Multimodal limits

The following per-modality limits apply when your dataset includes image, audio, or video data:

What's next

• Configure hyperparameters for your tuning job.
• Define reward functions.
• Monitor job status and metrics.
• Follow the Quick start to create your first reinforcement learning fine-tuning job.