evaluation_metrics

Creates, updates, deletes, gets or lists an evaluation_metrics resource.

Overview

Name	evaluation_metrics
Type	Resource
Id	digitalocean.genai.evaluation_metrics

Fields

The following fields are returned by SELECT queries:

genai_list_evaluation_metrics

A successful response.

Name	Datatype	Description
metric_name	string	(example: example name)
associated_presets	array	Saved model evaluation presets that reference this metric. Populated for custom metrics when listing metrics so the dashboard can warn that deleting the metric will also delete these presets. Empty for built-in metrics.
category	string	(METRIC_CATEGORY_UNSPECIFIED, METRIC_CATEGORY_CORRECTNESS, METRIC_CATEGORY_USER_OUTCOMES, METRIC_CATEGORY_SAFETY_AND_SECURITY, METRIC_CATEGORY_CONTEXT_QUALITY, METRIC_CATEGORY_MODEL_FIT) (default: METRIC_CATEGORY_UNSPECIFIED, example: METRIC_CATEGORY_UNSPECIFIED)
custom_eval_config	object	Configuration for a custom model-evaluation metric scored by an LLM judge. Prompt and model response are always included in the judge context.
description	string	(example: example string)
evaluation_scope	string	Scope that determines whether a metric belongs to agent evaluation or model evaluation. For backwards compatibility, UNSPECIFIED defaults to agent metrics only in list operations. (EVALUATION_SCOPE_UNSPECIFIED, EVALUATION_SCOPE_AGENT, EVALUATION_SCOPE_MODEL) (default: EVALUATION_SCOPE_UNSPECIFIED, example: EVALUATION_SCOPE_UNSPECIFIED)
inverted	boolean	If true, the metric is inverted, meaning that a lower value is better.
is_metric_goal	boolean
metric_rank	integer (int64)
metric_type	string	(METRIC_TYPE_UNSPECIFIED, METRIC_TYPE_GENERAL_QUALITY, METRIC_TYPE_RAG_AND_TOOL, METRIC_TYPE_MODEL_QUALITY, METRIC_TYPE_MODEL_SAFETY) (default: METRIC_TYPE_UNSPECIFIED, example: METRIC_TYPE_UNSPECIFIED)
metric_uuid	string	(example: 123e4567-e89b-12d3-a456-426614174000)
metric_value_type	string	(METRIC_VALUE_TYPE_UNSPECIFIED, METRIC_VALUE_TYPE_NUMBER, METRIC_VALUE_TYPE_STRING, METRIC_VALUE_TYPE_PERCENTAGE) (default: METRIC_VALUE_TYPE_UNSPECIFIED, example: METRIC_VALUE_TYPE_UNSPECIFIED)
range_max	number (float)	The maximum value for the metric.
range_min	number (float)	The minimum value for the metric.
source	string	Distinguishes platform catalog metrics from user-defined LLM-as-judge metrics. (EVALUATION_METRIC_SOURCE_UNSPECIFIED, EVALUATION_METRIC_SOURCE_BUILTIN, EVALUATION_METRIC_SOURCE_CUSTOM) (default: EVALUATION_METRIC_SOURCE_UNSPECIFIED, example: EVALUATION_METRIC_SOURCE_UNSPECIFIED)

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
genai_list_evaluation_metrics	select			To list all evaluation metrics, send a GET request to /v2/gen-ai/evaluation_metrics.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description

SELECT examples

genai_list_evaluation_metrics

To list all evaluation metrics, send a GET request to /v2/gen-ai/evaluation_metrics.

SELECT
metric_name,
associated_presets,
category,
custom_eval_config,
description,
evaluation_scope,
inverted,
is_metric_goal,
metric_rank,
metric_type,
metric_uuid,
metric_value_type,
range_max,
range_min,
source
FROM digitalocean.genai.evaluation_metrics
;