model_evaluation_presets

Creates, updates, deletes, gets or lists a model_evaluation_presets resource.

Overview

Name	model_evaluation_presets
Type	Resource
Id	digitalocean.genai.model_evaluation_presets

Fields

The following fields are returned by SELECT queries:

genai_get_model_evaluation_preset

A successful response.

Name	Datatype	Description
name	string	Name of the evaluation preset. (example: example name)
candidate_model_name	string	Model slug used to call the candidate model API. Empty when the CANDIDATE section was not saved. (example: example name)
dataset_name	string	Display name of the dataset stored on this preset. Empty when the DATASET section was not saved or the dataset no longer exists. (example: example name)
judge_model_name	string	Display name of the judge model stored on this preset. Empty when the JUDGE section was not saved or the model no longer exists. (example: example name)
candidate_inference_config	object	Inference configuration for the candidate model during evaluation.
candidate_model_source	string	Whether inference runs against the serverless platform, a dedicated deployment, or a model router. (CANDIDATE_MODEL_SOURCE_SERVERLESS, CANDIDATE_MODEL_SOURCE_DEDICATED, CANDIDATE_MODEL_SOURCE_ROUTER) (default: CANDIDATE_MODEL_SOURCE_SERVERLESS, example: CANDIDATE_MODEL_SOURCE_SERVERLESS)
candidate_model_uuid	string	UUID of the candidate model stored on this preset. Empty when the CANDIDATE section was not saved. For DEDICATED candidates this is the dedicated inference deployment UUID. (example: 123e4567-e89b-12d3-a456-426614174000)
candidate_system_prompt	string	System prompt / instructions to send to the candidate model. Empty when the SYSTEM_PROMPT section was not saved (check saved_sections). (example: 2023-01-01)
created_at	string (date-time)	Timestamp when the preset was created. (example: 2023-01-01T00:00:00Z)
dataset_uuid	string	UUID of the dataset stored on this preset. Empty when the DATASET section was not saved (check saved_sections). (example: 123e4567-e89b-12d3-a456-426614174000)
eval_preset_uuid	string	UUID of the evaluation preset. (example: 123e4567-e89b-12d3-a456-426614174000)
judge_model_uuid	string	UUID of the judge model stored on this preset. Empty when the JUDGE section was not saved (check saved_sections). (example: 123e4567-e89b-12d3-a456-426614174000)
metrics	array	Metrics selected for this preset. Empty when the METRICS section was not saved.
saved_sections	array	Sections of the inline evaluation config that were persisted when this preset was created. Use this to tell "section was saved with an empty value" apart from "section was not saved" — scalar fields like dataset_uuid or candidate_system_prompt are always emitted as the empty string when the section was not saved.
star_metric	object

genai_list_model_evaluation_presets

A successful response.

Name	Datatype	Description
name	string	Name of the evaluation preset. (example: example name)
candidate_model_name	string	Model slug used to call the candidate model API. Empty when the CANDIDATE section was not saved. (example: example name)
dataset_name	string	Display name of the dataset stored on this preset. Empty when the DATASET section was not saved or the dataset no longer exists. (example: example name)
judge_model_name	string	Display name of the judge model stored on this preset. Empty when the JUDGE section was not saved or the model no longer exists. (example: example name)
candidate_inference_config	object	Inference configuration for the candidate model during evaluation.
candidate_model_source	string	Whether inference runs against the serverless platform, a dedicated deployment, or a model router. (CANDIDATE_MODEL_SOURCE_SERVERLESS, CANDIDATE_MODEL_SOURCE_DEDICATED, CANDIDATE_MODEL_SOURCE_ROUTER) (default: CANDIDATE_MODEL_SOURCE_SERVERLESS, example: CANDIDATE_MODEL_SOURCE_SERVERLESS)
candidate_model_uuid	string	UUID of the candidate model stored on this preset. Empty when the CANDIDATE section was not saved. For DEDICATED candidates this is the dedicated inference deployment UUID. (example: 123e4567-e89b-12d3-a456-426614174000)
candidate_system_prompt	string	System prompt / instructions to send to the candidate model. Empty when the SYSTEM_PROMPT section was not saved (check saved_sections). (example: 2023-01-01)
created_at	string (date-time)	Timestamp when the preset was created. (example: 2023-01-01T00:00:00Z)
dataset_uuid	string	UUID of the dataset stored on this preset. Empty when the DATASET section was not saved (check saved_sections). (example: 123e4567-e89b-12d3-a456-426614174000)
eval_preset_uuid	string	UUID of the evaluation preset. (example: 123e4567-e89b-12d3-a456-426614174000)
judge_model_uuid	string	UUID of the judge model stored on this preset. Empty when the JUDGE section was not saved (check saved_sections). (example: 123e4567-e89b-12d3-a456-426614174000)
metrics	array	Metrics selected for this preset. Empty when the METRICS section was not saved.
saved_sections	array	Sections of the inline evaluation config that were persisted when this preset was created. Use this to tell "section was saved with an empty value" apart from "section was not saved" — scalar fields like dataset_uuid or candidate_system_prompt are always emitted as the empty string when the section was not saved.
star_metric	object

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
genai_get_model_evaluation_preset	select	eval_preset_uuid		To retrieve a saved model evaluation preset, send a GET request to /v2/genai/model_evaluation_presets/{eval_preset_uuid}.
genai_list_model_evaluation_presets	select			To list all saved model evaluation presets, send a GET request to /v2/genai/model_evaluation_presets.
genai_delete_model_evaluation_preset	delete	eval_preset_uuid		To delete a saved model evaluation preset, send a DELETE request to /v2/gen-ai/model_evaluation_presets/{eval_preset_uuid}.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
eval_preset_uuid	string	UUID of the evaluation preset to delete. (example: "123e4567-e89b-12d3-a456-426614174000")

SELECT examples

genai_get_model_evaluation_preset

To retrieve a saved model evaluation preset, send a GET request to /v2/genai/model_evaluation_presets/{eval_preset_uuid}.

SELECT
name,
candidate_model_name,
dataset_name,
judge_model_name,
candidate_inference_config,
candidate_model_source,
candidate_model_uuid,
candidate_system_prompt,
created_at,
dataset_uuid,
eval_preset_uuid,
judge_model_uuid,
metrics,
saved_sections,
star_metric
FROM digitalocean.genai.model_evaluation_presets
WHERE eval_preset_uuid = '{{ eval_preset_uuid }}' -- required
;

genai_list_model_evaluation_presets

To list all saved model evaluation presets, send a GET request to /v2/genai/model_evaluation_presets.

SELECT
name,
candidate_model_name,
dataset_name,
judge_model_name,
candidate_inference_config,
candidate_model_source,
candidate_model_uuid,
candidate_system_prompt,
created_at,
dataset_uuid,
eval_preset_uuid,
judge_model_uuid,
metrics,
saved_sections,
star_metric
FROM digitalocean.genai.model_evaluation_presets
;

DELETE examples

genai_delete_model_evaluation_preset

To delete a saved model evaluation preset, send a DELETE request to /v2/gen-ai/model_evaluation_presets/{eval_preset_uuid}.

DELETE FROM digitalocean.genai.model_evaluation_presets
WHERE eval_preset_uuid = '{{ eval_preset_uuid }}' --required
;