evaluation_datasets

Creates, updates, deletes, gets or lists an evaluation_datasets resource.

Overview

Name	evaluation_datasets
Type	Resource
Id	digitalocean.genai.evaluation_datasets

Fields

The following fields are returned by SELECT queries:

genai_list_evaluation_datasets

A successful response.

Name	Datatype	Description
dataset_name	string	Name of the dataset. (example: example name)
created_at	string (date-time)	Time created at. (example: 2023-01-01T00:00:00Z)
dataset_type	string	(EVALUATION_DATASET_TYPE_UNKNOWN, EVALUATION_DATASET_TYPE_ADK, EVALUATION_DATASET_TYPE_NON_ADK, EVALUATION_DATASET_TYPE_MODEL) (default: EVALUATION_DATASET_TYPE_UNKNOWN, example: EVALUATION_DATASET_TYPE_UNKNOWN)
dataset_uuid	string	UUID of the dataset. (example: 123e4567-e89b-12d3-a456-426614174000)
file_size	string (uint64)	The size of the dataset uploaded file in bytes. (example: 12345)
has_ground_truth	boolean	Does the dataset have a ground truth column?
row_count	integer (int64)	Number of rows in the dataset.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
genai_list_evaluation_datasets	select		dataset_type	To list evaluation datasets, send a GET request to /v2/gen-ai/evaluation_datasets.
genai_create_evaluation_dataset	insert			To create an evaluation dataset, send a POST request to /v2/gen-ai/evaluation_datasets.
genai_delete_evaluation_dataset	delete	dataset_uuid		To delete an evaluation dataset, send a DELETE request to /v2/genai/evaluation_datasets/{dataset_uuid}. This works for both model and agent evaluation datasets.
genai_create_evaluation_dataset_file_upload_presigned_urls	exec			To create presigned URLs for evaluation dataset file upload, send a POST request to /v2/gen-ai/evaluation_datasets/file_upload_presigned_urls.
genai_create_model_eval_dataset_upload_presigned_urls	exec			To create presigned URLs for model evaluation dataset file upload, send a POST request to /v2/genai/model_evaluation/datasets/file_upload_presigned_urls.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
dataset_uuid	string	UUID of the evaluation dataset to delete. (example: "123e4567-e89b-12d3-a456-426614174000")
dataset_type	string	Filter by evaluation dataset type. (example: EVALUATION_DATASET_TYPE_UNKNOWN)

SELECT examples

genai_list_evaluation_datasets

To list evaluation datasets, send a GET request to /v2/gen-ai/evaluation_datasets.

SELECT
dataset_name,
created_at,
dataset_type,
dataset_uuid,
file_size,
has_ground_truth,
row_count
FROM digitalocean.genai.evaluation_datasets
WHERE dataset_type = '{{ dataset_type }}'
;

INSERT examples

genai_create_evaluation_dataset

To create an evaluation dataset, send a POST request to /v2/gen-ai/evaluation_datasets.

INSERT INTO digitalocean.genai.evaluation_datasets (
dataset_type,
file_upload_dataset,
name
)
SELECT 
'{{ dataset_type }}',
'{{ file_upload_dataset }}',
'{{ name }}'
RETURNING
evaluation_dataset_uuid
;

Manifest

# Description fields are for documentation purposes
- name: evaluation_datasets
props:
  - name: dataset_type
    value: "{{ dataset_type }}"
    valid_values: ['EVALUATION_DATASET_TYPE_UNKNOWN', 'EVALUATION_DATASET_TYPE_ADK', 'EVALUATION_DATASET_TYPE_NON_ADK', 'EVALUATION_DATASET_TYPE_MODEL']
    default: EVALUATION_DATASET_TYPE_UNKNOWN
  - name: file_upload_dataset
    description: |
      File to upload as data source for knowledge base.
    value:
      original_file_name: "{{ original_file_name }}"
      size_in_bytes: "{{ size_in_bytes }}"
      stored_object_key: "{{ stored_object_key }}"
  - name: name
    value: "{{ name }}"
    description: |
      The name of the agent evaluation dataset.

DELETE examples

genai_delete_evaluation_dataset

To delete an evaluation dataset, send a DELETE request to /v2/genai/evaluation_datasets/{dataset_uuid}. This works for both model and agent evaluation datasets.

DELETE FROM digitalocean.genai.evaluation_datasets
WHERE dataset_uuid = '{{ dataset_uuid }}' --required
;

Lifecycle Methods

genai_create_evaluation_dataset_file_upload_presigned_urls

To create presigned URLs for evaluation dataset file upload, send a POST request to /v2/gen-ai/evaluation_datasets/file_upload_presigned_urls.

EXEC digitalocean.genai.evaluation_datasets.genai_create_evaluation_dataset_file_upload_presigned_urls 
@@json=
'{
"files": "{{ files }}"
}'
;

genai_create_model_eval_dataset_upload_presigned_urls

To create presigned URLs for model evaluation dataset file upload, send a POST request to /v2/genai/model_evaluation/datasets/file_upload_presigned_urls.

EXEC digitalocean.genai.evaluation_datasets.genai_create_model_eval_dataset_upload_presigned_urls 
@@json=
'{
"files": "{{ files }}"
}'
;