Skip to main content

Episode 12: SQL + YAML = Dashboards

Lars Kamp
Michael Driscoll

Creating an analytics dashboard is a time-consuming process that involves stitching together many components: ELT pipelines, cloud warehouses, transformation and semantic layers, data catalogs, and a dashboard tool. The flexibility of the Modern Data Stack (MDS) also means a great deal of complexity and many design decisions.

Rill Data is on a mission to radically simplify how developers create operational dashboards. Rill offers blazing fast dashboards that come bundled with a real-time analytical database and a modeling layer.

Michael Driscoll is the co-founder and CEO of Rill Data. In this episode, Mike demos the latest 0.16 release of Rill Developer.

There are three pieces of infrastructure that form a Rill dashboard application:

  • Sources: Rill ships with a CLI you can use to import data from an object store like AWS S3 or Google Cloud Storage. Rill treats the object store as the source of truth and imports data for the "last-mile ETL." As data in the object store changes, Rill orchestrates incremental updates.
  • Runtime: The runtime itself consists of a database (DuckDB), a web UI for rendering the dashboards (SvelteKit), and a middleware written in Go. Rill Enterprise replaces DuckDB with Apache Druid to process large data sets.
  • Models: Configuration code that parameterizes the dashboards, using YAML and SQL.

Bringing these things together in one application is an opinionated way to transform data to dashboards that Mike says covers "80%+ of the use cases that [they've] come across when building operational dashboards." Rill customers create dashboards to build analytics for their advertising, marketplace, and infrastructure operations.

Rill's stack is a departure from point-and-click interfaces, moving towards what Mike calls "BI-as-code." Source definitions and metrics are implemented in YAML, and models create a SQL query. The combination of SQL and YAML creates a BI layer that can be checked into a Git repository, which can then be managed automatically by CI workflows.

We also cover broader trends in our discussion, including the convergence of engineering and analytics cultures as engineers adopt practices from analytics to work with infrastructure data. Watch this episode to learn more about building data infrastructure for engineering teams with SQL and YAML with Rill.