DataDios Lineage

Overview

Lineage is the live map of how data flows through your stack. DataDios discovers it automatically from the warehouse, the orchestrator, and your code, then keeps it current as those sources change — so you can trace any column back to its origin and forward to every consumer without maintaining a manual catalog.

Key Capabilities

The graph is complete and live

Every dataset, every job, and every column-level dependency lives in one graph. It updates as your warehouse, dbt manifest, and orchestrator change — there is no separate catalog to keep current. Stale lineage is worse than none, so the graph is refreshed continuously rather than snapshotted once.

• Granularity: column-level
• Refresh: continuous
• Curation: the catalog is built from continuous discovery, not hand-entered — but the Entity Catalog below also lets you mint entity types and instances, rename, edit attributes, delete, and group by hand where discovery doesn't cover it

Workflows carve the graph into ownership

A workflow is a named slice of the graph that a team owns — its sources, its outputs, and the intermediate nodes between them. Two teams can have overlapping workflows; ownership here means responsibility, not exclusivity, so a shared table can belong to more than one team's view.

• Sources: per workflow
• Outputs: owned
• Overlap: allowed

Propose connections discovery can't see

Some dependencies automated discovery can't observe — a Lambda that reads from S3, a manual CSV upload. In free-form mode you can drag a connector between two nodes, and the edit goes into an approval queue (author is not the approver) rather than silently rewriting the global graph. The map stays trustworthy because every manual edge is reviewed.

Impact: what breaks if this changes?

Select any node and the Impact view lists every downstream consumer. Before you ship a schema change, Impact tells you exactly who depends on the column you're about to drop or rename — so a breaking change becomes a conversation with the right teams instead of an outage.

Pairs with drift and quality

Lineage is most useful next to the rest of the platform. Cross-reference it with the Metadata Timeline to see when a column changed, and with data quality checks to see whether a failing rule's root cause is an upstream table that shifted.

Entity Catalog

Overview

The Catalog tab on the Lineage page is a flat, searchable list of the structural objects DataDios has catalogued — sources, databases, schemas, objects and columns in one table, sorted by name, with each row's full path beneath it.

Those five levels are the whole table. Tags, domains, environments and classifications are entities too, but they are curation vocabulary rather than data objects, so they appear as values you pick in an attribute cell rather than as rows of their own. Domain, Env, Classification and Tags are also the filter rail's four dimensions, once a structural row actually carries the value. The same is true of any entity type you declare yourself: its instances are stored and usable, but the table does not list them.

Where Workflows shows how data moves, the Catalog answers "what do we have, and how is it classified". Within those five levels it is not scoped to a workflow: it spans everything in the tenant.

Before you start

The catalog is populated by the entity sync, which publishes through the tenant's default queue. If no queue is configured the catalog stays empty and nothing reports an error — see Queue Configuration.

Navigation

• Open Lineage and select the Catalog tab, or go straight to /lineage?tab=catalog.

Reading the table

Column	What it shows
Entity	The object's name, with its full path beneath it (source/database/schema/object/column)
Level	Source, Database, Schema, Object or Column
Domain · Env · Classification · Tags	Curated attributes, where they have been set. Click a cell to edit it
Groups	The groups this entity belongs to. Click to add or remove

Columns, in the toolbar, turns any column off and on again except the entity name and the row actions. The choice is remembered in the browser you made it in, so it does not follow you to another machine or to a colleague.

A path shown with a leading …/ is incomplete — one of its ancestors has been retired or removed. An attribute cell reading not shown means the row carries more values than the table resolves at once; it does not mean the row has none.

Finding things

• Search matches the start of an entity's name and ignores case, so ord finds both orders and ORDERS. It does not match the middle of a name, and it does not search the path.
• Level narrows the table to the levels you pick.
• The filter rail on the left filters by classification, domain, environment and tags. Its counts cover every entity in the tenant. Selecting a Level narrows the table but not these counts; applying a rail filter or a search does narrow them.

Paging

The catalog uses Back and Next rather than page numbers. The count above the pager shows 500+ once a result set passes five hundred — the exact total is not computed, because counting every match costs more than showing the page.

Managing entities

Create covers four kinds, chosen from one dropdown:

Kind	What it makes	Appears in the table?
Structural entity	A source, database, schema, object or column instance — the same kind discovery produces	Yes
Vocabulary value	A tag, domain, environment or classification value to curate with	No — it becomes a value you can pick in an attribute cell. It also becomes a filter-rail entry once applied to a row
Instance of a declared type	A row of a type you've minted below	No — it is created and stored, but nothing in the UI lists it
Entity type	A new type, with its own attribute definitions, to hold instances	No — the type itself is not a row

Creating a name that already exists never duplicates it, and what happens next depends on the kind. A structural entity or a custom-type instance is refused: nothing is created, the dialog stays open and says so, and anything you typed alongside the name — a parent, attribute values — is not applied to the existing row. Edit that row to change its values. A vocabulary value is not refused, because the request carries nothing but the name: you are told it already exists, and the dialog closes, since the value you wanted is there and ready to pick.

Only the first kind produces a row you will see here. The table is always scoped to the five structural levels, and the Level filter offers only those five, so a vocabulary value or a custom-type instance cannot be found by searching or filtering the Catalog. Create those when something else needs them — a tag you are about to apply, a type you are modelling — not as a way to add a row to this table.

Edit renames an entity you created by hand (origin: user_defined). On a discovered row the name belongs to the source system, so the field is shown read-only with the reason underneath — most rows here are discovered, so for those Edit is for attributes only. A rename changes only the display name — existing filters, curation and counts keep pointing at the same row.

Attributes are edited from the table, not from a form. Click any Domain, Env, Classification, Tags or Groups cell and a picker opens on the spot:

• Click a value to add it; click it again to take it off. Domain and Env hold one value, so picking a second replaces the first. Classification, Tags and Groups hold as many as you like.
• Type in the box to narrow the list, then press Enter to attach a value that isn't there yet — it is created and applied in one step, so there's no separate "add a tag first".
• A cell shows one value plus a count (+2) when there are more; hover the count to see the rest.
• A cell reading not shown carries more values than the table resolves at once. It can't be edited from here, because the picker would have to replace a list it never fully loaded.

The list of existing values shows the first 200. Past that, type the value's exact name and press Enter — it resolves to the existing one rather than making a duplicate.

Delete runs in the background — you'll see the row disappear once it finishes, not immediately. It's permanent. There's no audit trail and no undo. It's only offered on rows you created by hand (origin: user_defined) — discovered rows can't be deleted from here. A row with children can't be deleted until they are.

Groups are a lightweight, user-defined way to cluster entities across levels — unlike Domain, Env, Classification or Tags, a group isn't seeded by DataDios. Add a row to a group, or take it out again, from the Groups cell.

On a deployment that used the catalog's earlier "Add to group" action, the Groups cell opens with a message saying groups can't be edited there. Group membership on those installs was recorded in a way the table cannot read back, and it can't be corrected from the UI — ask your DataDios contact. Everywhere else the column behaves like the others.