custom_evaluation_metrics

Creates, updates, deletes, gets or lists a custom_evaluation_metrics resource.

Overview

Name	custom_evaluation_metrics
Type	Resource
Id	digitalocean.genai.custom_evaluation_metrics

Fields

The following fields are returned by SELECT queries:

SELECT not supported for this resource, use SHOW METHODS to view available operations for the resource.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
genai_create_custom_evaluation_metric	insert			To create a custom LLM-as-judge metric for model evaluation, send a POST request to /v2/gen-ai/custom_evaluation_metrics.
genai_update_custom_evaluation_metric	replace	metric_uuid		To update a custom metric (issuing a new metric UUID), send a PUT request to /v2/gen-ai/custom_evaluation_metrics/{metric_uuid}.
genai_delete_custom_evaluation_metric	delete	metric_uuid		To soft-delete a custom model evaluation metric, send a DELETE request to /v2/gen-ai/custom_evaluation_metrics/{metric_uuid}.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
metric_uuid	string	UUID of the custom metric to delete. (example: "123e4567-e89b-12d3-a456-426614174000")

INSERT examples

genai_create_custom_evaluation_metric

To create a custom LLM-as-judge metric for model evaluation, send a POST request to /v2/gen-ai/custom_evaluation_metrics.

INSERT INTO digitalocean.genai.custom_evaluation_metrics (
config,
description,
metric_name
)
SELECT 
'{{ config }}',
'{{ description }}',
'{{ metric_name }}'
RETURNING
metric
;

Manifest

# Description fields are for documentation purposes
- name: custom_evaluation_metrics
props:
  - name: config
    description: |
      Configuration for a custom model-evaluation metric scored by an LLM judge.
      Prompt and model response are always included in the judge context.
    value:
      created_at: "{{ created_at }}"
      deleted_at: "{{ deleted_at }}"
      requires_ground_truth: {{ requires_ground_truth }}
      scoring_prompt: "{{ scoring_prompt }}"
      updated_at: "{{ updated_at }}"
  - name: description
    value: "{{ description }}"
  - name: metric_name
    value: "{{ metric_name }}"

REPLACE examples

genai_update_custom_evaluation_metric

To update a custom metric (issuing a new metric UUID), send a PUT request to /v2/gen-ai/custom_evaluation_metrics/{metric_uuid}.

REPLACE digitalocean.genai.custom_evaluation_metrics
SET 
config = '{{ config }}',
description = '{{ description }}',
metric_name = '{{ metric_name }}',
metric_uuid = '{{ metric_uuid }}'
WHERE 
metric_uuid = '{{ metric_uuid }}' --required
RETURNING
metric;

DELETE examples

genai_delete_custom_evaluation_metric

To soft-delete a custom model evaluation metric, send a DELETE request to /v2/gen-ai/custom_evaluation_metrics/{metric_uuid}.

DELETE FROM digitalocean.genai.custom_evaluation_metrics
WHERE metric_uuid = '{{ metric_uuid }}' --required
;