Apache Zeppelin
A notebook where different paragraphs can use different languages, sharing results between them in one note.
Apache Zeppelin is a web based notebook for data work. What distinguishes it is that each paragraph declares which interpreter it uses, so one note can hold SQL, Scala, Python and shell paragraphs together. Results can be passed between interpreters, and query output is rendered as charts without additional code.
What makes Zeppelin different
Zeppelin is a web based notebook, and its distinguishing property is that a single note is not tied to one language.
Most notebook environments run one language per notebook. A Python notebook runs Python. Working in a second language means a second notebook.
In Zeppelin, each paragraph declares its interpreter. One paragraph runs SQL. The next runs Scala. The next runs a shell command. All within one note.
Why mixing languages is useful
Real data work is not naturally one language.
Getting the data is a query, and SQL expresses that best. Processing it may need code, because the operation is awkward in SQL. Preparing a file might be a shell command. Presenting the result is a chart.
Forced into one language, some of those steps become contorted. Complex procedural logic written in SQL is painful; simple aggregation written as procedural code is verbose.
Choosing per paragraph means each step is expressed in whatever suits it, in one document.
Passing results between interpreters
Mixing languages is only useful if the results connect.
Output from one interpreter can be made available to another, so a table produced by a SQL paragraph is processed by a code paragraph. Without that, a mixed language note would be several unrelated analyses sharing a page.
How the sharing works depends on the interpreters involved, and the documentation covers it per interpreter.
Charts without plotting code
Tabular results are rendered as charts through the interface.
During exploration this removes a real detour. You run a query, want to see the shape of the result, and would otherwise write plotting code to look at it. Being able to switch the output to a chart means the examination is immediate.
For finished work, deliberate plotting still produces better results. For the exploratory phase, where you are looking at dozens of results and keeping few, it is the right trade.
Input controls
Values in a paragraph can be bound to form controls in the note.
That turns a note from something only its author can run into something colleagues use. They change a date or a region in a control and rerun, without editing code they may not understand.
Where it is found
Zeppelin appears most often in environments built around distributed processing engines, because interpreters exist for those engines and a note can run work on a cluster directly.
That is its natural setting: a data platform where the processing happens on a cluster and analysts need an interface to it.
Who uses it
Zeppelin is used by data engineers and analysts working with cluster based processing, particularly where several languages are involved in the same platform.
Points to consider
It is a server application, so it is deployed and operated rather than installed on a laptop. Interpreters also need configuring for the engines they connect to.
Its ecosystem is smaller than that of the most widely used notebook environment, which affects available material and community answers.
Mixing languages is a strength and it needs discipline. A note where the language changes every paragraph without reason is harder to follow than one that stayed in a single language.
Getting started
The documentation covers installation, configuring interpreters, writing paragraphs in different languages and using input controls. Running a SQL paragraph, charting its result and then processing that result in a code paragraph demonstrates what the environment is for.
Key features of Apache Zeppelin
Capabilities described in the official documentation.
Interpreters chosen per paragraph
Each paragraph declares its language, so one note can mix SQL, Scala, Python and shell commands.
Results shared between languages
Output from one interpreter can be made available to another, so a query result is used by code.
Charts without plotting code
Tabular results are displayed as charts through the interface rather than requiring a plotting library.
Input controls in a note
Form elements bound to values let a reader change parameters and rerun without editing the code.
Advantages of Apache Zeppelin
Factual advantages that follow from the features above.
Each step uses a suitable language
Querying in SQL and processing in code within one note avoids forcing everything through one language.
Charting a query takes no code
Because results render as charts directly, examining a query result does not require plotting to be written.
Notes can be used by others
Input controls let colleagues change parameters and rerun rather than needing to edit anything.
Close to cluster processing engines
Interpreters for distributed engines mean a note runs work on a cluster rather than only locally.
Common use cases for Apache Zeppelin
Situations the official documentation describes this tool as being used for.
Exploring data on a cluster
A note queries and processes data through a distributed engine without leaving the browser.
Combining SQL and code in one analysis
A SQL paragraph produces a result that a code paragraph then processes further.
Sharing a parameterised analysis
Input controls let colleagues run the same note with their own values.
Presenting results from a query
Query output is charted directly so findings are shown without a separate visualisation step.
Official website
Everything on this page is based on the official documentation for Apache Zeppelin. You can read the source here.
Apache Zeppelin official documentation