StayLab

Problem statement

Product teams need credible experiment readouts, but marketplace listing data alone cannot support randomised A/B analysis without a transparent simulated assignment layer.

Engineering objective

1. Identified which listing attributes could be treated as observed marketplace data.
2. Designed an explicit simulated user, session and booking layer on top of real listings.
3. Defined primary, secondary and guardrail metrics with SQL-first aggregation.
4. Scoped decision rules beyond p-values alone.

Solution

StayLab combines real Inside Airbnb London listing data with a transparent simulated experiment layer, PostgreSQL analytics, Python statistical inference, a FastAPI service and a Next.js readout for experiment EXP-024.

• Python ETL from Inside Airbnb London snapshot into PostgreSQL.
• Simulated user assignment, sessions, funnel events and bookings with fixed seed reproducibility.
• Named SQL views and queries consumed by the API without caller-supplied SQL.
• Z-tests, confidence intervals, sample-ratio mismatch checks and guardrail metrics.

How it works

Python ETL and inference, PostgreSQL, FastAPI.

Next.js dashboard for experiment readout.

Docker Compose for local Postgres.

pytest coverage for assignment, uplift, z-test and API schemas.

Controls and constraints

• API exposes only named, read-only SQL catalogue queries.
• Raw Inside Airbnb files are not committed to the repository.
• Simulated experiment layer is labelled throughout README, API and UI.

Key capabilities

• Data pipeline from marketplace snapshot to warehouse tables.
• Experiment simulator with reproducible assignment.
• FastAPI endpoints for summary, timeseries, segments and statistics.
• Next.js experiment dashboard.

Product and system views

A full experimentation stack for analysing product changes with reproducible data pipelines and statistical decision rules.

What the system enables

• Shows how to separate observed marketplace data from simulated experiment events.
• Demonstrates statistical decision discipline beyond headline significance.
• Provides a portfolio-grade experimentation analytics reference implementation.

Validation and testing

• pytest coverage for random assignment, one variant per user and decision policy.
• Integration tests available when a loaded database is present.

Limitations

• Booking conversion in the simulated layer is not observed marketplace behaviour.
• Calendar aggregation is the slowest ETL step on full snapshots.
• Requires Docker for local PostgreSQL in the documented quick start.