Apache Ranger

Access control in an environment of many services

A Hadoop environment is not one system. It is several services that happen to share storage and a cluster: a query engine, a store for records, an ingestion service, a processing framework.

Each has some notion of permissions of its own. Left alone, that means access rules configured separately in every service, in different formats, by whoever set that service up.

The consequences are predictable. The same rule is expressed differently in different places. A person removed from a team loses access in three services and keeps it in a fourth. Nobody can state with confidence who can reach a particular table, because answering requires checking several systems.

Apache Ranger addresses this by defining policies centrally and enforcing them within each service.

Two parts

The design has two components, and understanding the split explains most of its behaviour.

Administration is central. There is one place where policies are defined, reviewed and changed, with an interface for managing them.

Enforcement is distributed. Each supported service has a plugin installed into it. The plugin retrieves policies and evaluates each incoming request against them.

The reason for enforcing inside the service rather than in front of it is worth stating, because it is the part that matters for whether the control is real.

A control placed in front of a service protects requests going through the front. Anything reaching the service by another route is unaffected. In an environment where services are accessed by several tools, through several interfaces, that is a meaningful gap.

A plugin running within the service evaluates requests arriving by any supported path. The control is at the point where access actually happens.

What a policy can say

Policies grant permissions to users and groups, and they can be specific about what.

Access is expressible at database, table and column level. That granularity matters because sensitivity is rarely uniform across a table. A table of customer records may be broadly useful while three columns in it are not, and without column level control the choice is between exposing everything and exposing nothing.

The usual result of that binary choice is that people copy the data, remove the sensitive columns and share the copy. That copy then has no governance at all, which is worse than the problem it solved.

Auditing

Access decisions are recorded.

The reason this is a core feature rather than an addition is that governance obligations are usually about demonstrating something rather than only doing it. The question is not only whether access is controlled but whether you can show who could reach particular data and who did.

Records of decisions provide that basis. Without them, the answer to an auditor's question is an assertion about how the system is configured, which is a considerably weaker position.

Alongside Atlas

Ranger is frequently deployed with Apache Atlas, and they address different things.

Atlas is about metadata and classification: what data exists, what it means, how it is labelled.

Ranger is about access: who may reach it.

The connection between them is that classification is useful input to policy. A rule referring to data classified in a particular way is more durable than one naming specific tables, because new data carrying that classification falls under it without anyone writing a new rule.

Who uses it

Apache Ranger is used by platform and security teams operating Hadoop and related data environments. It is packaged with several Hadoop distributions, so many organisations encounter it as part of a platform rather than adopting it separately.

Points to consider

Ranger is an Apache project under an open source licence, and the official project site is the reference for supported services, versions and configuration.

Coverage is defined by which plugins exist and are installed. A service without a plugin is not governed by these policies, and assuming otherwise leaves a gap that is invisible until someone looks for it.

Its natural setting is a Hadoop and related services environment. Estates spanning cloud data platforms may need it alongside other controls, or a platform designed to span both, which is the space commercial products in this area occupy.

Policies still require design and review. Central definition removes duplication rather than the need to decide what the rules should be, and rules that go unexamined drift away from what the organisation actually intends.

Getting started

The official project documentation covers the architecture, installing plugins for supported services, defining policies and the auditing configuration. Setting a column level policy on a single table and querying as two different users demonstrates the enforcement model directly.