Download OpenAPI specification:Download
Marquez is an open source metadata service for the collection, aggregation, and visualization of a data ecosystem's metadata.
Creates a new namespace object. A namespace enables the contextual grouping of related jobs and datasets. Namespaces must contain only letters (a-z
, A-Z
), numbers (0-9
), underscores (_
), dashes (-
), colons (:
), slashes (/
), or dots (.
). A namespace is case-insensitive with a maximum length of 1024
characters. Note jobs and datasets will be unique within a namespace, but not across namespaces.
namespace required | string <= 1024 characters Example: my-namespace The name of the namespace. |
ownerName required | string The owner of the namespace. |
description | string The description of the namespace. |
{- "ownerName": "me",
- "description": "My first namespace!"
}
{- "name": "my-namespace",
- "createdAt": "2019-05-09T19:49:24.201361Z",
- "updatedAt": "2019-05-09T19:49:24.201361Z",
- "ownerName": "me",
- "description": "My first namespace!"
}
Returns a namespace.
namespace required | string <= 1024 characters Example: my-namespace The name of the namespace. |
{- "name": "my-namespace",
- "createdAt": "2019-05-09T19:49:24.201361Z",
- "updatedAt": "2019-05-09T19:49:24.201361Z",
- "ownerName": "me",
- "description": "My first namespace!"
}
Soft deletes a namespace, and every job and dataset inside. On next event containing this namespace, the namespace will be undeleted.
namespace required | string <= 1024 characters Example: my-namespace The name of the namespace. |
{- "name": "my-namespace",
- "createdAt": "2019-05-09T19:49:24.201361Z",
- "updatedAt": "2019-05-09T19:49:24.201361Z",
- "ownerName": "me",
- "description": "My first namespace!"
}
Returns a list of namespaces.
limit | integer Default: 100 Example: limit=25 The number of results to return from offset. |
offset | integer Default: 0 The initial position from which to return results. |
{- "namespaces": [
- {
- "name": "my-namespace",
- "createdAt": "2019-05-09T19:49:24.201361Z",
- "updatedAt": "2019-05-09T19:49:24.201361Z",
- "ownerName": "me",
- "description": "My first namespace!"
}
]
}
Returns a list of OpenLineage events, sorted in direction of passed sort parameter. By default it is desc.
sortDirection | string Example: sortDirection=name Sorts the results of your query by indicated direction |
before | string <date-time> Example: before=2022-09-15T07:47:19Z Returns events before passed date. |
after | string <date-time> Example: after=2022-09-15T07:47:19Z Returns events after passed date. |
limit | integer Default: 100 Example: limit=25 The number of results to return from offset. |
offset | integer Default: 0 The initial position from which to return results. |
{- "events": [
- {
- "eventType": "COMPLETE",
- "eventTime": "2019-05-09T19:49:24.201361Z",
- "run": {
- "runId": "d46e465b-d358-4d32-83d4-df660ff614dd"
}, - "job": {
- "namespace": "my-namespace",
- "name": "my-job"
}, - "outputs": [
- {
- "namespace": "my-namespace",
- "name": "my-output",
- "facets": {
- "schema": {
- "fields": [
- {
- "name": "a",
- "type": "INTEGER"
}, - {
- "name": "b",
- "type": "TIMESTAMP"
}, - {
- "name": "c",
- "type": "INTEGER"
}, - {
- "name": "d",
- "type": "INTEGER"
}
]
}
}
}
],
}
], - "totalCount": 0
}
Creates a new source object. A source is the physical location of a dataset such as a table in PostgreSQL, or topic in Kafka. A source enables the grouping of physical datasets to their physical source.
source required | string <= 1024 characters Example: my-source The name of the source. |
type required | string The type of the source. |
connectionUrl required | string <URL> The URL to the location of the source. |
description | string The description of the source. |
{- "type": "POSTGRESQL",
- "connectionUrl": "jdbc:postgresql://db.example.com/mydb",
- "description": "My first source!"
}
{- "type": "POSTGRESQL",
- "name": "my-source",
- "createdAt": "2019-05-09T19:49:24.201361Z",
- "updatedAt": "2019-05-09T19:49:24.201361Z",
- "connectionUrl": "jdbc:postgresql://db.example.com/mydb",
- "description": "My first source!"
}
Returns a source.
source required | string <= 1024 characters Example: my-source The name of the source. |
{- "type": "POSTGRESQL",
- "name": "my-source",
- "createdAt": "2019-05-09T19:49:24.201361Z",
- "updatedAt": "2019-05-09T19:49:24.201361Z",
- "connectionUrl": "jdbc:postgresql://db.example.com/mydb",
- "description": "My first source!"
}
Returns a list of sources.
limit | integer Default: 100 Example: limit=25 The number of results to return from offset. |
offset | integer Default: 0 The initial position from which to return results. |
{- "sources": [
- {
- "type": "POSTGRESQL",
- "name": "my-source",
- "createdAt": "2019-05-09T19:49:24.201361Z",
- "updatedAt": "2019-05-09T19:49:24.201361Z",
- "connectionUrl": "jdbc:postgresql://db.example.com/mydb",
- "description": "My first source!"
}
]
}
Creates a new dataset.
namespace required | string <= 1024 characters Example: my-namespace The name of the namespace. |
dataset required | string <= 1024 characters Example: my-dataset The name of the dataset. |
type required | string Value: "DB_TABLE" The type of the dataset. |
physicalName required | string The physical name of the table. |
sourceName required | string The name of the source associated with the table. |
required | Array of objects[ items ] The fields of the table. |
tags | Array of strings List of tags. |
description | string The description of the table. |
runId | string The ID associated with the run modifying the table. |
{- "type": "DB_TABLE",
- "physicalName": "public.mytable",
- "sourceName": "my-source",
- "fields": [
- {
- "name": "a",
- "type": "INTEGER"
}, - {
- "name": "b",
- "type": "TIMESTAMP"
}, - {
- "name": "c",
- "type": "INTEGER"
}, - {
- "name": "d",
- "type": "INTEGER"
}
], - "description": "My first dataset!"
}
{- "id": {
- "namespace": "my-namespace",
- "name": "my-dataset"
}, - "type": "DB_TABLE",
- "name": "my-dataset",
- "physicalName": "public.mytable",
- "createdAt": "2019-05-09T19:49:24.201361Z",
- "upodatedAt": "2019-05-09T19:49:24.201361Z",
- "namespace": "my-namespace",
- "sourceName": "my-source",
- "fields": [
- {
- "name'": "a",
- "type": "INTEGER",
- "tags": [ ]
}, - {
- "name'": "b",
- "type": "TIMESTAMP",
- "tags": [ ]
}, - {
- "name'": "c",
- "type": "INTEGER",
- "tags": [ ]
}, - {
- "name'": "d",
- "type": "INTEGER",
- "tags": [ ]
}
], - "tags": [ ],
- "lastModifiedAt": null,
- "description": "My first dataset!",
- "facets": { },
- "currentVersion": "b1d626a2-6d3a-475e-9ecf-943176d4a8c6"
}
Returns a dataset.
namespace required | string <= 1024 characters Example: my-namespace The name of the namespace. |
dataset required | string <= 1024 characters Example: my-dataset The name of the dataset. |
{- "id": {
- "namespace": "my-namespace",
- "name": "my-dataset"
}, - "type": "DB_TABLE",
- "name": "my-dataset",
- "physicalName": "public.mytable",
- "createdAt": "2019-05-09T19:49:24.201361Z",
- "upodatedAt": "2019-05-09T19:49:24.201361Z",
- "namespace": "my-namespace",
- "sourceName": "my-source",
- "fields": [
- {
- "name'": "a",
- "type": "INTEGER",
- "tags": [ ]
}, - {
- "name'": "b",
- "type": "TIMESTAMP",
- "tags": [ ]
}, - {
- "name'": "c",
- "type": "INTEGER",
- "tags": [ ]
}, - {
- "name'": "d",
- "type": "INTEGER",
- "tags": [ ]
}
], - "tags": [ ],
- "lastModifiedAt": null,
- "description": "My first dataset!",
- "facets": { },
- "currentVersion": "b1d626a2-6d3a-475e-9ecf-943176d4a8c6"
}
Soft deletes dataset. It will be un-deleted if new OpenLineage event containing this dataset comes.
namespace required | string <= 1024 characters Example: my-namespace The name of the namespace. |
dataset required | string <= 1024 characters Example: my-dataset The name of the dataset. |
{- "id": {
- "namespace": "my-namespace",
- "name": "my-dataset"
}, - "type": "DB_TABLE",
- "name": "my-dataset",
- "physicalName": "public.mytable",
- "createdAt": "2019-05-09T19:49:24.201361Z",
- "upodatedAt": "2019-05-09T19:49:24.201361Z",
- "namespace": "my-namespace",
- "sourceName": "my-source",
- "fields": [
- {
- "name'": "a",
- "type": "INTEGER",
- "tags": [ ]
}, - {
- "name'": "b",
- "type": "TIMESTAMP",
- "tags": [ ]
}, - {
- "name'": "c",
- "type": "INTEGER",
- "tags": [ ]
}, - {
- "name'": "d",
- "type": "INTEGER",
- "tags": [ ]
}
], - "tags": [ ],
- "lastModifiedAt": null,
- "description": "My first dataset!",
- "facets": { },
- "currentVersion": "b1d626a2-6d3a-475e-9ecf-943176d4a8c6"
}
Returns a version for a dataset.
namespace required | string <= 1024 characters Example: my-namespace The name of the namespace. |
dataset required | string <= 1024 characters Example: my-dataset The name of the dataset. |
version required | string <uuid> Example: ea9badc5-7cb2-49af-9a9f-155771d3a797 The ID of the job or dataset version. |
{- "id": {
- "namespace": "my-namespace",
- "name": "my-dataset"
}, - "type": "DB_TABLE",
- "name": "my-dataset",
- "physicalName": "public.mytable",
- "createdAt": "2019-05-09T19:49:24.201361Z",
- "version": "d224dac0-35d7-4d9b-bbbe-6fff1a8485ad",
- "namespace": "my-namespace",
- "sourceName": "my-source",
- "fields": [
- {
- "name": "a",
- "type": "INTEGER",
- "tags": [ ]
}, - {
- "name": "b",
- "type": "TIMESTAMP",
- "tags": [ ]
}, - {
- "name": "c",
- "type": "INTEGER",
- "tags": [ ]
}, - {
- "name": "d",
- "type": "INTEGER",
- "tags": [ ]
}
], - "tags": [ ],
- "description": "My first dataset!",
- "createdByRun": {
- "id": "0e89b60a-d80d-42ae-a12b-bd79ae1fb576",
- "createdAt": "2019-05-09T18:52:35.620763Z",
- "updatedAt": "2019-05-09T18:53:41.461122Z",
- "nominalStartTime": null,
- "nominalEndTime": null,
- "state": "COMPLETED",
- "startedAt": "2021-02-05T18:53:33.245492Z",
- "endedAt": "2021-02-05T18:53:41.461122Z",
- "durationMs": 8216,
- "args": {
- "email": "me@example.com",
- "emailOnFailure": false,
- "emailOnRetry": true,
- "retries": 1
}, - "jobVersion": {
- "namespace": "my-namespace",
- "name": "my_processing_job",
- "version": "0ba5b13b-350e-40f3-ba3f-cc00a4e92d11"
}, - "inputVersions": [
- {
- "namespace": "my-namespace",
- "name": "an_input_dataset",
- "version": "d224dac0-35d7-4d9b-bbbe-6fff1a8485ad"
}
], - "outputVersions":