How a Series B SaaS Built Experimentation Infrastructure from Zero

A Series B SaaS platform making product decisions on gut feeling transformed into a data-driven experimentation machine — running 15 experiments per month within 8 weeks.

The Challenge: Building Products in the Dark

The product team was shipping features fast — two-week sprint cycles, 30+ releases per quarter. But they had a fundamental problem: nobody could measure whether any of it actually worked.

The company had grown to $12M ARR on the strength of a great founding insight and aggressive sales execution. But as they moved from founder-led sales to product-led growth, the cracks started to show:

• Zero event tracking. The product had basic page-view analytics (Google Analytics) but no behavioral event tracking. Nobody knew what users actually did inside the application.
• No activation metrics. The team couldn't define — let alone measure — what "activation" meant. New user onboarding was a black box.
• Feature decisions based on loudest customer. The product roadmap was driven by enterprise customer requests and internal opinions, not usage data or experimentation.
• No A/B testing capability. The engineering team had discussed experimentation for 18 months but never implemented it. Every feature shipped to 100% of users on day one.
• Churn analysis was impossible. When a customer churned, the post-mortem was qualitative only — "they said they didn't find value." But nobody could pinpoint where in the product experience the value gap existed.

The VP of Product knew this had to change. The board was asking for product-led growth metrics, the CPO wanted experiment velocity, and the team was tired of debating features without data. They needed someone who could design and implement the entire analytics and experimentation stack — fast.

Our Approach: Instrument, Then Experiment

We designed an 8-week engagement with two distinct phases: first build the measurement foundation, then layer experimentation on top. You can't experiment if you can't measure.

Weeks 1–2: Tracking Plan & Event Architecture

We started by mapping the entire user journey — from first landing page visit through activation, engagement, and expansion. Working with the PM team, we identified 47 critical user events across 6 product areas.

We designed a structured tracking plan following the "Object-Action" naming convention (e.g., workspace_created, report_exported, team_member_invited) to ensure consistency and prevent the tracking debt that plagues most SaaS products.

Key decisions at this stage:

• Mixpanel as the primary product analytics platform — chosen over Amplitude for its superior funnel analysis and the team's existing familiarity
• Segment as the CDP layer — future-proofing the data pipeline so switching analytics tools later wouldn't require re-instrumentation
• Server-side tracking for critical business events (subscription changes, feature usage milestones) to ensure data accuracy regardless of ad blockers

Weeks 3–4: Implementation & Activation Funnel

We worked directly with two frontend engineers and one backend engineer to implement the tracking plan. Rather than trying to instrument everything at once, we focused on the activation funnel first — the highest-leverage area for a product-led growth motion.

The activation analysis revealed something the team had suspected but never proven: 68% of new users who didn't complete their first workflow within 48 hours never came back. This single insight reshaped the entire onboarding strategy.

We also built the company's first real-time product dashboard, showing daily active users, activation rate, feature adoption, and retention curves — metrics the leadership team had never had visibility into.

Weeks 5–6: Experimentation Framework

With measurement in place, we built the experimentation infrastructure:

• Feature flagging system using LaunchDarkly, integrated with Mixpanel for automatic experiment analysis
• Experiment design templates — standardized documents for hypothesis, metrics, sample size calculation, and decision criteria
• Statistical rigor guidelines — minimum detectable effect sizes, required confidence levels, and guardrail metrics to prevent shipping changes that improve one metric while degrading another
• Experiment review process — weekly 30-minute sessions where the team reviews results and decides ship/iterate/kill

Weeks 7–8: First Experiments & Team Enablement

We launched 8 experiments in the final two weeks, focusing on the activation funnel:

• Onboarding flow variations (3 experiments)
• First-run experience simplification (2 experiments)
• Activation email timing and content (2 experiments)
• In-app guidance triggers (1 experiment)

Simultaneously, we ran hands-on training sessions for the PM team — teaching them to design experiments, interpret Mixpanel funnels, and make statistically sound decisions. The goal was full team autonomy within 30 days of our engagement ending.

Key Deliverables

• Tracking Plan — 47 events across 6 product areas, with naming conventions and governance process
• Segment + Mixpanel Implementation — full instrumentation, server-side and client-side
• Activation Funnel Analysis — quantified the exact drop-off points and behavioral patterns of retained vs. churned users
• Experimentation Playbook — templates, statistical guidelines, and review process documentation
• Feature Flagging Infrastructure — LaunchDarkly setup integrated with analytics for automated experiment reporting
• Real-time Product Dashboard — DAU, activation rate, feature adoption, retention curves
• Team Training — 3 workshops on experiment design, analytics interpretation, and data-driven prioritization

Results

• 0 → 15 experiments per month — within 60 days of launch, the product team was running and analyzing experiments independently
• 23% improvement in activation rate — the onboarding experiments alone moved the needle from 31% to 38% activation within the first week
• 40% faster feature validation — features that previously took 6–8 weeks to evaluate (ship → wait for NPS feedback) were now validated in 2–3 weeks through controlled experiments
• First data-driven roadmap — Q3 planning was the first quarter where every major initiative had a measurable hypothesis and success metric
• 3x reduction in "ship and pray" releases — major features now go through staged rollout with measurement gates

"Before this engagement, our product reviews were 90% opinion and 10% data. Now it's the opposite. We killed two features we were sure would work, and doubled down on one we almost didn't build. The experimentation culture has fundamentally changed how we make product decisions."

— VP of Product, Series B SaaS Platform

Why This Approach Works

• Measurement before experimentation. Many teams try to run A/B tests before they have reliable tracking. We built the measurement foundation first, so every experiment produces trustworthy results.
• Activation focus. Rather than trying to optimize everything at once, we focused on the highest-leverage metric — activation — where small improvements compound into significant revenue impact.
• Team enablement, not dependency. We designed everything for handoff. The PM team owns experimentation now — they don't need us to run tests.
• Speed over perfection. We shipped a working experimentation system in 8 weeks, not 8 months. Perfect tracking coverage can come later; the cultural shift of data-driven decision-making needed to happen immediately.