Version v0.6 of the documentation is no longer actively maintained. The site that you are currently viewing is an archived snapshot. For up-to-date documentation, see the latest version.

Metadata API Reference

Reference documentation for the Kubeflow Metadata API
api/service.proto

Kubeflow Metadata Service REST API

Version: v1alpha1

Default request content-types: application/json
Default response content-types: application/json
Schemes: http, https

Summary

Path Operation Description
/api/v1alpha1/artifact_types GET
POST
/api/v1alpha1/artifact_types/{name} DELETE
GET
/api/v1alpha1/artifact_types/{name}/artifacts GET
/api/v1alpha1/artifact_types/{name}/artifacts/{id} DELETE
GET
/api/v1alpha1/artifact_types/{parent}/artifacts POST

/api/v1alpha1/artifacts GET
/api/v1alpha1/events POST
/api/v1alpha1/events/artifacts/{name} GET

List events based on an artifact or execution id.

/api/v1alpha1/events/executions/{name} GET

List events based on an artifact or execution id.

/api/v1alpha1/execution_types GET
POST
/api/v1alpha1/execution_types/{name} DELETE
GET
/api/v1alpha1/execution_types/{name}/executions GET
/api/v1alpha1/execution_types/{name}/executions/{id} DELETE
GET
/api/v1alpha1/execution_types/{parent}/executions POST
/api/v1alpha1/executions GET

Paths

GET /api/v1alpha1/artifact_types

Tags: MetadataService

Uses default content-types: application/json

200 OK

A successful response.

POST /api/v1alpha1/artifact_types

Tags: MetadataService

DELETE /api/v1alpha1/artifact_types/{name}

Tags: MetadataService
name path string

Uses default content-types: application/json

200 OK

A successful response.

GET /api/v1alpha1/artifact_types/{name}

Tags: MetadataService
name

ArtifactType names are of the form artifact_types/{namespace}/{name}.

path string

Uses default content-types: application/json

200 OK

A successful response.

GET /api/v1alpha1/artifact_types/{name}/artifacts

Tags: MetadataService
name path string

Uses default content-types: application/json

200 OK

A successful response.

DELETE /api/v1alpha1/artifact_types/{name}/artifacts/{id}

Tags: MetadataService
id path string
name path string

Uses default content-types: application/json

200 OK

A successful response.

GET /api/v1alpha1/artifact_types/{name}/artifacts/{id}

Tags: MetadataService
id path string
name

Artifact name is like artifact_types/{namespace}/{typename}/artifact/{id}.

path string

Uses default content-types: application/json

200 OK

A successful response.

NOTE: The order of the following RPC methods affects the order of matching a particular HTTP path. So put a more specific path pattern before a generic one. For example, GET /api/v1alpha1/artifact_types/{parent}/artifacts should appear before GET /api/v1alpha1/artifact_types/{name} to be possibly matched.

POST /api/v1alpha1/artifact_types/{parent}/artifacts

Tags: MetadataService

Uses default content-types: application/json

The Artifact to create. Note that Artifact.type_id is ignored.

parent

Creates the specified artifact as an instance of ArtifactType with this fully qualified name. |parent| takes the form artifact_types/{namespace}/{name>}.

path string

Uses default content-types: application/json

200 OK

A successful response.

GET /api/v1alpha1/artifacts

Tags: MetadataService
name query string

Uses default content-types: application/json

200 OK

A successful response.

POST /api/v1alpha1/events

Tags: MetadataService

Uses default content-types: application/json

Uses default content-types: application/json

200 OK

A successful response.

List events based on an artifact or execution id.

GET /api/v1alpha1/events/artifacts/{name}

Tags: MetadataService
name path string

Uses default content-types: application/json

200 OK

A successful response.

List events based on an artifact or execution id.

GET /api/v1alpha1/events/executions/{name}

Tags: MetadataService
name path string

Uses default content-types: application/json

200 OK

A successful response.

GET /api/v1alpha1/execution_types

Tags: MetadataService

Uses default content-types: application/json

200 OK

A successful response.

POST /api/v1alpha1/execution_types

Tags: MetadataService

DELETE /api/v1alpha1/execution_types/{name}

Tags: MetadataService
name path string

Uses default content-types: application/json

200 OK

A successful response.

GET /api/v1alpha1/execution_types/{name}

Tags: MetadataService
name path string

Uses default content-types: application/json

200 OK

A successful response.

GET /api/v1alpha1/execution_types/{name}/executions

Tags: MetadataService
name path string

Uses default content-types: application/json

200 OK

A successful response.

DELETE /api/v1alpha1/execution_types/{name}/executions/{id}

Tags: MetadataService
id path string
name path string

Uses default content-types: application/json

200 OK

A successful response.

GET /api/v1alpha1/execution_types/{name}/executions/{id}

Tags: MetadataService
id path string
name

Execution name is like execution_types/{namespace}/{typename}/execution/{id}.

path string

Uses default content-types: application/json

200 OK

A successful response.

POST /api/v1alpha1/execution_types/{parent}/executions

Tags: MetadataService

Uses default content-types: application/json

The Execution to create. Note that Execution.type_id is ignored.

parent

Creates the specified artifact as an instance of ExecutionType with this fully qualified name. |parent| takes the form execution_types/{namespace}/{name>}.

path string

Uses default content-types: application/json

200 OK

A successful response.

GET /api/v1alpha1/executions

Tags: MetadataService
name query string

Uses default content-types: application/json

200 OK

A successful response.

Schema definitions

apiCreateArtifactResponse: object

artifact: ml_metadataArtifact

Newly created artifact with id.

apiCreateArtifactTypeResponse: object

artifact_type: ml_metadataArtifactType

Newly created artifact type with id.

apiCreateExecutionResponse: object

execution: ml_metadataExecution

Newly created execution with id.

apiCreateExecutionTypeResponse: object

execution_type: ml_metadataExecutionType

Newly created execution type with id.

apiGetArtifactResponse: object

apiGetArtifactTypeResponse: object

artifact_type: ml_metadataArtifactType

apiGetExecutionResponse: object

apiGetExecutionTypeResponse: object

execution_type: ml_metadataExecutionType

apiListArtifactsResponse: object

artifacts: object[]

apiListArtifactTypesResponse: object

artifact_types: object[]

apiListEventsResponse: object

events: object[]
artifacts: object
executions: object

apiListExecutionsResponse: object

executions: object[]

apiListExecutionTypesResponse: object

execution_types: object[]

EventPath: object

A simple path (e.g. {step{key:"foo"}}) can name an artifact in the context of an execution.

steps: object[]

A simple path (e.g. {step{key:"foo"}}) can name an artifact in the context of an execution.

ml_metadataAnyArtifactStructType: object

Every ArtifactStruct is a member of this type.

ml_metadataArtifact: object

id: string (int64)

The id of the artifact.

type_id: string (int64)

The id of an ArtifactType. Type must be specified when an artifact is created, and it cannot be changed.

uri: string

The uniform resource identifier of the physical artifact. May be empty if there is no physical artifact.

properties: object

Properties of the artifact. Properties must be specified in the ArtifactType.

custom_properties: object

User provided custom properties which are not defined by its type.

ml_metadataArtifactStructType: object

The type of an ArtifactStruct. An artifact struct type represents an infinite set of artifact structs. It can specify the input or output type of an ExecutionType. See the more specific types referenced in the message for more details.

simple: ml_metadataArtifactType
union_type: ml_metadataUnionArtifactStructType
intersection: ml_metadataIntersectionArtifactStructType
list: ml_metadataListArtifactStructType
none: ml_metadataNoneArtifactStructType
any: ml_metadataAnyArtifactStructType
tuple: ml_metadataTupleArtifactStructType
dict: ml_metadataDictArtifactStructType

ml_metadataArtifactType: object

id: string (int64)

The id of the type. 1-1 relationship between type names and IDs.

name: string

The name of the type.

properties: object

The schema of the type. Properties are always optional in the artifact. Properties of an artifact type can be expanded but not contracted (i.e., you can add columns but not remove them).

ml_metadataDictArtifactStructType: object

properties: object

Underlying properties for the type.

none_type_not_required: boolean (boolean)

If true, then if properties["foo"] can be None, then that key is not required.

extra_properties_type: ml_metadataArtifactStructType

Extra keys are allowed that are not specified in properties. These keys must have the type specified below. If this is not specified, then extra properties are not allowed.

ml_metadataEvent: object

An event represents a relationship between an artifact ID and an execution. There are four kinds of events, relating to both input and output, as well as declared versus undeclared. For example, the DECLARED_INPUT events are part of the signature of an execution. For example, consider: my_execution({"data":[3,7],"schema":8}) Where 3, 7, and 8 are artifact_ids. Assuming execution_id is 12, this becomes: {artifact_id:3, execution_id: 12, type:DECLARED_INPUT, path:{step:[{"key":"data"},{"index":0}]}} {artifact_id:7, execution_id: 12, type:DECLARED_INPUT, path:{step:[{"key":"data"},{"index":1}]}} {artifact_id:8, execution_id: 12, type:DECLARED_INPUT, path:{step:[{"key":"schema"}]}} The INPUT is an artifact actually read by the execution. The OUTPUT is an artifact actually written by the execution. The DECLARED_OUTPUT are the artifacts that are the "official" output. For example, the trainer may output multiple caches of the parameters (as OUTPUT objects), but then finally write the SavedModel as a DECLARED_OUTPUT. TODO(martinz): add a type for Event, similar to ArtifactType.

artifact_id: string (int64)

The artifact id is required for an event, and should refer to an existing artifact.

execution_id: string (int64)

The execution_id is required for an event, and should refer to an existing execution.

path: EventPath

The path in an artifact struct, or the name of an artifact.

type: ml_metadataEventType

The type of an event.

milliseconds_since_epoch: string (int64)

ml_metadataEventType: string , x ∈ { UNKNOWN (default) , DECLARED_OUTPUT , DECLARED_INPUT , INPUT , OUTPUT }

Events distinguish between an artifact that is written by the execution (possibly as a cache), versus artifacts that are part of the declared output of the Execution. For more information on what DECLARED_ means, see the comment on the message.

ml_metadataExecution: object

id: string (int64)

The id of the execution.

type_id: string (int64)

The id of an ExecutionType. The ExecutionType must be specified and cannot be changed.

last_known_state: ml_metadataExecutionState

The last known state of an execution.

properties: object

Properties of the Execution. Properties must be specified in the ExecutionType.

custom_properties: object

User provided custom properties which are not defined by its type.

ml_metadataExecutionState: string , x ∈ { UNKNOWN (default) , NEW , RUNNING , COMPLETE , FAILED }

TODO(martinz): consider adding INVALIDATED as a state.

ml_metadataExecutionType: object

id: string (int64)

The id of the type. 1-1 relationship between type names and IDs.

name: string

The name of the type.

properties: object

The schema of the type. Properties are always optional in the execution.

input_type: ml_metadataArtifactStructType

The ArtifactStructType of the input. For example: { "dict":{ "properties":{ "schema":{ "union_type":{ "none":{}, "simple":{...schema type...} }, }, "data":{ "simple":{...data_type...} } } } } That would be an optional schema field with a required data field.

output_type: ml_metadataArtifactStructType

ml_metadataIntersectionArtifactStructType: object

A member of this type must satisfy all constraints. This primarily useful not as an end-user type, but something calculated as an intermediate type in the system.

For example, suppose you have a method: def infer_my_input_type(a): # try to infer the input type of this method. use_in_method_x(a) # with input type x_input use_in_method_y(a) # with input type y_input

Given this information, you know that infer_my_input_type has type {"intersection":{"constraints":[x_input, y_input]}}.

IntersectionArtifactStructType intersection_type = {"constraints":[ {"dict":{"properties":{"schema":{"any":{}}}, "extra_properties":{"any":{}}}}, {"dict":{"properties":{"data":{"any":{}}}, "extra_properties":{"any":{}}}}]} Since the first constraint requires the dictionary to have a schema property, and the second constraint requires it to have a data property, this is equivalent to: ArtifactStructType other_type = {"dict":{"properties":{"schema":{"any":{}},"data":{"any":{}}}}, "extra_properties":{"any":{}}}

constraints: object[]

ml_metadataListArtifactStructType: object

Represents an ArtifactStruct list type with homogeneous elements.

element: ml_metadataArtifactStructType

Every entry in the list must be of this type. Note: if this type is Any, then the list can have arbitrary elements.

ml_metadataNoneArtifactStructType: object

ml_metadataPropertyType: string , x ∈ { UNKNOWN (default) , INT , DOUBLE , STRING }

TODO(martinz): consider moving this inside some message, to avoid having literals directly in apo package.

ml_metadataTupleArtifactStructType: object

An ordered list of heterogenous artifact structs. The length of the list is fixed. Each position in the list can have a different type.

elements: object[]

ml_metadataUnionArtifactStructType: object

Represents a union of types.

candidates: object[]

An artifact struct matches this type if it matches any of the candidates. If candidates is empty, this is a bottom type (matches no artifacts).

ml_metadataValue: object

A value in properties.

int_value: string (int64)
double_value: number (double)
string_value: string

PathStep: object

index: string (int64)
key: string