Evaluation.GoldenExpectation

Represents a single, checkable requirement.

JSON representation 
{
  "note": string,
  "skipEvaluation": boolean,
  "expectationLevelMetricsThresholdsOverride": {
    object (EvaluationMetricsThresholds.GoldenEvaluationMetricsThresholds.ExpectationLevelMetricsThresholds)
  },
  "agentResponseSemanticSimilarityMetricsConfigOverride": {
    object (EvaluationMetricsConfig.SemanticSimilarityMetricsConfig)
  },
  "agentResponseHallucinationMetricsConfigOverride": {
    object (EvaluationMetricsConfig.HallucinationMetricsConfig)
  },
  "comparisonType": enum (EvaluationMetricsConfig.ComparisonType),

  // Union field condition can be only one of the following:
  "toolCall": {
    object (ToolCall)
  },
  "toolResponse": {
    object (ToolResponse)
  },
  "agentResponse": {
    object (Message)
  },
  "agentTransfer": {
    object (AgentTransfer)
  },
  "updatedVariables": {
    object
  },
  "mockToolResponse": {
    object (ToolResponse)
  },
  "noToolCalls": boolean
  // End of list of possible types for union field condition.
}
Fields
note

string

Optional. A note for this requirement, useful in reporting when specific checks fail. E.g., "Check_Payment_Tool_Called".
skipEvaluation

boolean

Optional. If set to true, this specific expectation will not be evaluated.
expectationLevelMetricsThresholdsOverride

object (EvaluationMetricsThresholds.GoldenEvaluationMetricsThresholds.ExpectationLevelMetricsThresholds)

Optional. Overrides metrics at the step level.
agentResponseSemanticSimilarityMetricsConfigOverride

object (EvaluationMetricsConfig.SemanticSimilarityMetricsConfig)

Optional. Overrides for agentResponse semantic similarity metrics.
agentResponseHallucinationMetricsConfigOverride

object (EvaluationMetricsConfig.HallucinationMetricsConfig)

Optional. Overrides for agentResponse hallucination metrics.
comparisonType

enum (EvaluationMetricsConfig.ComparisonType)

Optional. The comparison type to use for the expectation check.
Union field condition. The actual check to perform. condition can be only one of the following:
toolCall

object (ToolCall)

Optional. Check that a specific tool was called with the parameters.
toolResponse

object (ToolResponse)

Optional. Check that a specific tool had the expected response.
agentResponse

object (Message)

Optional. Check that the agent responded with the correct response. The role "agent" is implied.
agentTransfer

object (AgentTransfer)

Optional. Check that the agent transferred the conversation to a different agent.
updatedVariables

object (Struct format)

Optional. Check that the agent updated the session variables to the expected values. Used to also capture agent variable updates for golden evals.
mockToolResponse

object (ToolResponse)

Optional. The tool response to mock, with the parameters of interest specified. Any parameters not specified will be hallucinated by the LLM.
noToolCalls

boolean

Optional. Check that no tools were called during this turn.

EvaluationMetricsConfig.HallucinationMetricsConfig

Configuration for the hallucination metrics for the evaluation. To disable the metric, set the message but do not set the enableHallucinationMetrics field to true (or explicitly set it to false). To unset the configuration and fallback to the default behavior, omit the message entirely.

JSON representation 
{
  "enableHallucinationMetrics": boolean
}
Fields
enableHallucinationMetrics

boolean

Optional. Whether to calculate hallucination metrics for the evaluation.

EvaluationMetricsConfig.ComparisonType

Supported comparison types for checking the agent's response.

Enums
COMPARISON_TYPE_UNSPECIFIED Unspecified comparison type. Behavior defaults to SEMANTIC_SIMILARITY for agent responses and tool calls.
EQUALS Exact string match.
CONTAINS Substring match (checks if the expected string is contained in the actual response).
SEMANTIC_SIMILARITY Semantic similarity match (evaluates meaning similarity using an LLM).