Hackathons are where a lot of your favorite features are born.

A Statsig tradition, Hackathons are quarterly, two-day parties wherein employees have free reign to work on whatever they want.

To some, this means fixing and improving existing stuff. To others, it means building new features that might otherwise be outside of scope.

🏍️ Fun fact: Statsig Sidecar began as a hackathon project.

Vijaye is a Hackathon fan

Let’s take a look back on some of the projects and hacks that Statsig built during our Q3 2024 Hackathon which ended this week.

Note: Unless otherwise stated, please assume all Hackathon projects are simply concepts and have not yet been shipped to end users, and may not be shipped at all.

If you really want us to prioritize the development of a certain project listed in this article, please let us know!

Finally, this is also not a comprehensive list, as it’s only comprised of the hacks we’re allowed to talk about. 😉

Skip ahead:

AI projects

Experiment ideas generator

Presenters: Brock Lumbard, Sedona Duggal, Minhye Kim, Matt Garnes

Summary: Experiment ideas generator is an aptly-named tool that generates ideas for experiments for your website.

It is currently open for all users to try, and was shipped during our hackathon.

To get started, simply enter any website into the search bar. The AI will analyze the website, including scraping source code and taking screenshots to garner a better “understanding” of the site, its purpose, its CTAs and prospective user flows, and so on.

Its output includes a summary, a hypothesis, and a control and test for each experiment.

You can open these experiments directly within your Statsig account, or (coming soon) use Statsig Sidecar to make them happen in real time..

Automatically generate Statsig projects

Presenters: Pablo Beltran

Summary: Pablo’s hack allows for Statsig users to automatically generate Statsig projects and data using AI.

For Pablo’s demonstration, he prompted to create “a tech company made by evil villains whose clients are also villains.”

On the backend, this calls out to OpenAI’s SDKs and describes Statsig console API, and ensures that all response conforms to the console API.

From there, it creates some “dummy data” to experiment with including experiments, metrics, and other sample data.

The example data was hilarious. Some highlights:

• Blackmail metrics

• Disaster simulations

• Lair upgrades

AI console search

Presenters: Lior Barak, Jairo Garciga

Summary: AI console search enhances the console’s existing search functionality with the power of AI. ✨

You now have the option to ask AI to generate page results to navigate to based on whatever you typed into the search bar.

It will generate a few results that it “thinks” are appropriate, rather than requiring users to know the exact name of what they’re searching for.

Codemonkey

Presenters: Chong Xie

Summary: “Everyone wants extra coding help so they can build their own kingdom,” jokes Chong.

Codemonkey is meant to be a coding AI that takes open-ended queries and translates them into impactful changes, leveraging the power of LLMs to perform ultimately simple tasks.

In his example, Chong says “Make our codebase better.” Codemonkey then goes through the Statsig directory and feeds each file to the LLM, presenting it with options on what to do with the code.

This is an iterative process. After each iteration, the changes are compiled and run in a test environment. If the changes don’t work, they are sent back to the LLM for a redo.

Experiment knowledge bank

Presenters: Kenny Yi

Summary: A lot of information is gained along a user’s experimentation journey that can be helpful to others.

To address this, Kenny created an experiment knowledge bank that sends experiment data to OpenAI with an assistant behind it, which allows users to query for very specific criteria using regular language.

Questions like “show me an experiment where we did a certain thing” will now yield intelligent answers.

What would Craig say?

Presenters: Yuzheng Sun

Summary: Do you ever wish you had more Craig? Same.

“What would Craig say” is an AI chatbot trained on our knowledge base, Statsig docs, and anonymized sales call transcripts.

“The ideal situation is that we’d have a Craig in every conversation,” muses YZ.

For now, YZ’s Craigbot is only available for Statsig employees to communicate with on a 1:1 basis, but who knows what integration opportunities will await it in the future?

MEx assistant

Presenters: Andrew Huang

Summary: MEx assistant is probably what you’d imagine; Statbot wrapped into Metrix Explorer.

This allows users to ask questions like “What exactly does this funnel show me?” directly in the console and receive clear answers.

As of writing, this has not been shipped.

Features and updates

CUPAC

Presenters: Craig Sexauer, Daniel West

Summary: CUPAC is a way to use pre-experiment data to predict what a user will do, but instead of using one variable, it can use N variables.

Still running in SQL, this new calculation method cuts off ~20% of variance, and allegedly uses “really complicated math,” as evidenced by Craig and Daniel’s really complicated math slides.

In the console, this will show live data about how much time was saved on an experiment using CUPAC when users hover over CUPED.

“Oh no, what happened?”

Presenters: Liz Obermaier and Nicole Smith

Summary: “Oh no, what happened” is a new feature that will allow users to generate tables as a starting point when investigating something that went wrong.

Especially useful for looking at different properties where a metric drilldown is required. If a value changed, you can look at how it changed for specific percentages of your sample population.

User attributes, experiments, and cohorts can also be plugged in for more accurate tracking.

As of right now, Oh no, what happened? works for count, sum, and mean metrics.

Sigma

Presenters: Brent Echols

Summary: In true Sigma fashion, Brent Echols created a unique solution to a common infra problem.

“Right now, Statsig gets results from Mongo and ‘shoves them’ into Redis, which creates a situation where Redis has to be checked,” Brent explains. “This isn’t good for caching efficiency, as you can only cache things per query. Without Mongo connections, we would have to do a huge offline cycle of caching results into Redis.”

Sigma is a read/write cache that acts as a reimplementation of MongoDB, but is done in memory which grants it greater speed.

The way this workflow works is that Mongo acts as storage, but CRUD operations and listings can be done in Sigma without the need to validate queries.

Statsig Kubernetes operator

Presenters: Jay Kodeswaran, Jason Wang

Summary: Say hello to one-shot infra rollouts.

The way the current model works is app owners interface with SDKs, which use Statsig to get results.

App > SDK > Statsig > Results

The Kubernetes operator is a new workflow that adds a second pathway: Kubernetes operator > Statsig > Results.

Jay demonstrated this by using it to change the color of a shape. Directly from Statsig, he added which parameters needed to be changed and started a simple rollout.

Kubernetes operator takes the current deployment, copies it, changes the fields, and then rolls it out.

Jay reckons it will be a “delight” for infra people.

Experiments into Slack

Presenters: Cooper Reid

Summary: Cooper’s Slack integration is a way to post experiments into customer conversations via Slack Connect.

Essentially, each experiment can have a webhook URL, which, when dropped into a project, can serve as a changelog which pings Slack every time the project is updated.

Analytics projects

MEx forecasting

Presenters: Alex Coleman

Summary: Using Prophet, an open-source forecasting library, Alex has brought forecasting capabilities directly to Metrics Explorer.

With a little bit of manual tuning, Metrics Explorer can now create forecasts with built-in holiday detection and anomaly detection, and can even be used with a group-by applied, like “DAU by country.”

MEx maps

Presenters: Pierre Estephan

Summary: Pierre’s hack, MEx maps, brings a new visualization type to Metrics Explorer: Maps.

Have you ever wanted to see pageloads grouped by country? With MEx maps, you can view this data as a geomap, pan, zoom, and all the other things you’d expect to be able to do with a map visualization.

CPU profiling for Statsig experiments

Presenters: Eric Liu

Summary: CPU profiling will allow users to see which of their processes are taking the most time.

Traditionally, this is a tiring process for engineers and involves digging around through snapshots, hoping to find one that helps.

With Eric’s changes, Statsig SDKs can now capture profiles. A Statsig agent that runs within our clusters parses and processes and sends the output to Statsig as events, logging all stat traces.

To demonstrate, Eric created a feature called “do a lot of work,” which was meant to use tons of computing time.

Eric presenting “do a lot of work”

Warehouse native performance observability

Presenters: Nick Jiang

Summary: Statsig Warehouse Native runs a lot of queries a day. Naturally, some are slower than others—and then this happens, users want to know why.

Nick added a post-processing step that uploads all job data to SWN, shows some visualizations, and can identify which experiments are the most performance expensive or which ones cost the most time per job, for instance.

This also enables users to see which metrics are contributing to overall loading times, and even es a sample view for all the jobs running in a warehouse native environment.

Culture hacks

Statsig dog calendar

Presenters: Cat Lee

Summary: The aptly named Statsig Dog Calendar is a calendar that features a different Statsig dog each month.

Designed by our internal brand design team, the calendar will hopefully be a wonderful and welcomed addition to just about every household in the world.

Would you buy one?

Internal directory

Presenters: Jess Barkley

Summary: Remember Myspace? Jess created an internal tool that looks eerily similar.

On our internal tools interface, users now have their own accounts which include their anniversary date, accomplishments, badges, and other internal tracking information.

First feature request: Let me add emo music to my profile.

Statsig Wrapped

Presenters: Katie Braden, Jessie Ong, Maria McCulley

Summary: Remember Spotify Wrapped? Imagine that but for experimentation!

Statsig Wrapped is a way to entertain users by showing them historical experimentation wins and metrics, all beautifully wrapped up into a shareable post. (If you’re okay with sharing your internal experimentation metrics.)

Statsig Wrapped includes different types of awards that users can earn, and has been tested on a handful of beta customers to much delight.

Gaming PC

Presenters: Yuzheng Sun

Summary: When asked to recap this hackathon project, YZ told me:

• There is an office gaming PC

• It is by the window

The office gaming PC, by the window.

Thanks, YZ!

Internal stuff

Costs by jobs

Presenters: Pushpenda Nagtode

Summary: Costs by jobs is a handy way of being able to see how expensive specific jobs are.

This is internal-only, but it allows us to more easily track by time and by job all in one dashboard.

Brand images in codebase

Presenters: Brian Do

Summary: Brian made it easier for Statsig developers to make Statsig look great!

Our new design system introduced lots of new visual treats from colors and icons to layouts and themes.

With Brian’s hack, Figma files in a certain format are automatically added to our codebase for use by developers. It also works on images, and is filterable, bringing more pleasing aesthetics to our end users!

That’s a wrap

This article was meant to give a peek behind the scenes at what our Hackathons actually entail. If you’re a Statsig fan, you probably see the end result when we ship features or post product updates.

This is a special glimpse at how many such features are born.

Thanks for reading!

Tell us your favorite!

Which Hackathon project is your favorite? Are there any you really want to try? Reach out and let us know!

Get in touch