Snappy Dashboards with Redis

It seems that almost every application I’ve worked on has some sort of dashboard component. It’s not usually the first feature but it often becomes the most useful.

But dashboards are hard to build.

They break resource-oriented design patterns and often rely on complicated one-off SQL queries. Dashboards also usually feature graphs and activity streams, both of which are not trivial to implement well.

At Sqoot, a local deal API, we have a simple dashboard for our customers. Given a time frame, it tells developers how many deals they’ve served (impressions), how many clicks they’ve driven, and how much affiliate revenue they’ve earned (earnings). It might also be interesting to know how many API calls they’ve made.

Sqoot Dashboard Screenshot

All of these stats follow a pretty standard query pattern:


This works great for small collections or well-indexed relational tables. For example, earnings are reported exactly like this. But enormous collections, especially when stored in document-centric databases, become slow to query. This is especially true if you have to join disparate data sources.

So we developed another way to store the counts we need. It’s a thin Ruby wrapper around Redis and we call it The Count (ah ah ah ah ah!). Here’s how it works.

For every stat we’re interested in, we set a few keys:


The first three keys tell us totals for all users by year, month, and day. The last three scope the same totals to one user. For example, if you want to know how many clicks user 5 drove in September:


You can also query across date ranges with the help of some Ruby:

]).inject { |sum, i| sum + i.to_i }

Every time an API call, impression, or click happens, we update all the relevant keys at once:

REDIS.multi { keys.each { |key| REDIS.incr key } }

If Redis isn’t available, for whatever reason, we could rebuild the gaps from the canonical data in Mongo. We’ve never had to do this.

Now, when our customers load their dashboard, we don’t need to go slogging through millions of records. This makes our dashboards snappy. It’s also worth noting that this approach takes up very little space. If we segment by day, each year for each stat only uses 375 keys per user and another 375 for the aggregate data. The values are almost inconsequential since their just numbers. All our dashboard data is stored in this way with our friends at Redis To Go in a 20 MB instance.

Thanks for reading! I'm Avand.

I’ve been working on the web for over a decade and am passionate about building great products.

My last job was with Airbnb, where I focused on internal products that helped teams measure the quality of the software they were building. I also built internal tools for employees to stay more connected, especially after the COVID-19 pandemic. Before that, I was lead engineer at Mystery Science, the #1 way in which science is taught in U.S. elementary school classroms. For a while, I also taught with General Assembly, teaching aspiring developers the basics of front-end web development.

I was born in Boston, grew up in Salt Lake City, and spent many years living in Chicago. Now, I call San Francisco my home and Mariposa my home away from home.

I enjoy the great outdoors and absolutely love music and dance. Cars have been an lifelong obsession of mine, especially vintage BMWs and Volkswagens. I’m the proud owner of a 2002 E-250 Sportsmobile van, and he and I have enjoyed many trips to beautiful and remote parts of the West Coast to create good vibes.

What can I do for you?

Read my other posts or get in touch: