Alation
A data catalogue that records what data exists and observes how it is actually queried to inform what it shows.
Alation is a data catalogue. It connects to the systems an organisation holds data in, records what tables and columns exist, and lets people document them. It also reads query logs, so what it shows about a table is informed by how that table is actually used rather than only by what somebody wrote about it.
The problem a catalogue addresses
In an organisation of any size, nobody knows what data exists.
There are hundreds of tables across several systems. Some are authoritative, some are abandoned copies, some looked promising and were never finished. Names are unhelpful. The people who built them have moved on.
So an analyst needing a figure asks colleagues, gets pointed at a table, and hopes it is the right one. That is how the same question ends up answered three different ways.
A data catalogue is a record of what exists and what it means.
Discovery and documentation
The catalogue connects to source systems and records what is there: tables, columns, types, where they live.
That is the mechanical part, and it is necessary rather than sufficient. Knowing a table has a column named cust_status does not tell you what the values mean or whether the column is still maintained.
So people add documentation: descriptions, context, warnings, which of two similar tables is the one to use. That is where the value is, and it is also the part that requires effort to sustain.
What query logs add
Alation's distinguishing capability is reading query history.
This produces information nobody has to write down, and which stays current on its own:
Which tables matter. A table queried three hundred times a week is important. One untouched in two years is probably abandoned. That distinction is invisible from the schema and obvious from usage.
Who the experts are. The people querying a table most are the ones who understand it. When somebody needs to ask, usage shows who to ask.
How tables are joined. Common join patterns reveal relationships that no documentation records.
Usage information is honest in a way documentation is not. Documentation describes what somebody believed at the time they wrote it. Usage describes what is happening.
Search
Because the catalogue records everything, finding data becomes searching rather than asking.
Results informed by usage matter here. A search returning forty tables with similar names is not helpful. The same search, with the heavily used ones surfaced, usually points at the right answer immediately.
Warnings before use
A useful catalogue feature is attaching warnings to assets.
A table with a known problem, a column that has been unreliable since a migration, a data set that should not be used for a particular purpose. That knowledge normally circulates informally and reaches people after they have already used the data.
Recorded against the asset, it appears when somebody looks it up.
Who uses Alation
Alation is used by large organisations with data spread across many systems, by analysts trying to find and understand data, and by governance teams recording what exists and who is responsible for it.
Points to consider
A catalogue is only as good as its adoption. One nobody contributes to becomes an inventory of stale descriptions that people learn to distrust.
The automated parts, particularly usage information, stay current without effort. The human parts need ownership, and that ownership has to be assigned rather than assumed.
Connecting to source systems also requires access, and what the catalogue can read is what it can record.
Getting started
The documentation covers connecting data sources, configuring query log ingestion, documenting assets and using search. Connecting one well used system and letting usage information accumulate demonstrates what the catalogue adds before broader rollout.
Key features of Alation
Capabilities described in the official documentation.
Automatic discovery of data assets
Connections to source systems record the tables, columns and types that exist without them being entered by hand.
Query log analysis
Reading query history shows which tables are actually used, by whom and how they are joined.
Documentation attached to assets
People add descriptions, warnings and context to tables and columns so knowledge is recorded rather than remembered.
Search across the catalogue
Assets are found by searching, with results informed by how heavily each is used.
Advantages of Alation
Factual advantages that follow from the features above.
Usage separates the important from the abandoned
Knowing which tables are queried constantly distinguishes them from the many that nobody has touched in years.
Knowledge stops living in people
Recorded documentation means what one analyst knows about a table survives them leaving.
Finding data becomes possible
Search across systems means an analyst locates the right table rather than asking colleagues.
Warnings reach people before use
A note about a table's known problem appears when somebody looks it up rather than after they have used it.
Common use cases for Alation
Situations the official documentation describes this tool as being used for.
Finding the right table
An analyst searches the catalogue rather than asking colleagues which of several similar tables to use.
Recording institutional knowledge
What experienced staff know about a data set is documented against it rather than held informally.
Identifying what can be retired
Usage information shows which tables nobody queries, which informs decommissioning decisions.
Flagging data with known issues
A warning attached to a table appears to anyone who looks it up before they build on it.
Official website
Everything on this page is based on the official documentation for Alation. You can read the source here.
Alation official documentation