Problem Validation Experiments SOP

A problem validation experiments SOP replaces "people told me it's a problem" with "people did something about it", and it takes two to four weeks. Interviews and surveys collect words. Experiments collect behaviour: booked calls, downloads, replies and small payments, all aimed at the problem before any product exists. People who won't spend 20 minutes on a problem won't spend money on its solution.

What is the problem validation experiments SOP and when should you use it?

A problem validation experiment is a small, time-boxed test with a written hypothesis, a fixed metric and a pass threshold you set before it starts. This SOP uses the Test Card and Learning Card from Testing Business Ideas by David J. Bland and Alexander Osterwalder, and Rob Fitzpatrick's idea from The Mom Test that real interest costs people something: time, reputation or money.

Every experiment here tests the problem only. You offer help, information or a diagnosis, never a product. Tests of a specific solution belong to the landing page test, fake door testing and concierge testing SOPs.

Use this SOP:

• After interviews and a validated problem statement, when you need evidence beyond what people say.
• When investors or co-founders doubt the problem and you want numbers that survive a hard question.
• Before you spend on design or code, since a failed problem test costs a few hundred dollars and a failed product costs months.

What do you need before you start?

• One problem statement that passed recognition tests with your target group.
• A list of 50 to 200 reachable people in that group: LinkedIn connections, community members, past interviewees' referrals.
• A simple page builder (Carrd, Notion, Typeform) and a booking link.
• $100 to $500 for ads or incentives. The marketing budget calculator helps you split it across channels.
• A test log in a spreadsheet: one row per experiment, filled in before launch.

How do you run it, step by step?

1. Write a test card for each hypothesis

Use Bland and Osterwalder's four lines: We believe that [customer] has [problem]. To verify that, we will [experiment]. And measure [metric]. We are right if [threshold]. Write the threshold as a number and a deadline. If two co-founders can read the card and disagree on whether a result passes, rewrite it.

2. Pick three experiments of rising cost to the customer

Choose one test per currency. Time: offer a free 20-minute call to help with the problem. Reputation: ask people to share a problem checklist with a colleague or join a waitlist with their work email. Money: sell a small paid diagnostic or audit, done by hand. Running all three gives you weak, medium and strong evidence side by side.

3. Build each test in a day

A help-call offer needs one paragraph and a booking link. A content test needs a one-page guide or checklist about the problem, with no product mentioned, and a signup form. A paid diagnostic needs a description, a price and a payment link. If a build takes longer than a day, cut scope.

4. Send traffic from one channel per test

Use direct messages for the help-call offer, a community post or small ad budget for the content test, and warm contacts for the paid diagnostic. Keep one channel per test so you know where the result came from. Log every send and every response.

5. Run each test for a fixed window

Give each test 7 to 14 days and do not move the threshold while it runs. If a test gets fewer than 30 people through the door, the sample is too small to judge: extend it once, then call it.

6. Fill in a learning card

For each test write: We believed that, We observed, From that we learned, and So we will. Record the raw counts next to the percentages.

7. Decide: continue, narrow or drop

Two of three tests passing means people act on the problem, and you can move to solution tests. One pass means narrow the customer group to the people who responded and rerun. Zero passes means drop or rewrite the problem.

Template you can copy

TEST CARD
Test name: ______________________   Deadline: __________
We believe that: [customer] has [problem] at least [frequency]
To verify that, we will: [experiment, channel, audience size]
And measure: [metric]
We are right if: [number] by [date]

LEARNING CARD
We believed that: ...
We observed: [raw counts and %]
From that we learned: ...
So we will: continue / narrow to [segment] / drop

HELP-CALL OFFER (direct message)
"Hi [name], I'm researching how [role] handle [problem].
I've put together a short checklist and I'm offering free 20-minute
calls to work through it with your own case. Want a slot this week?
[booking link]"

TEST LOG COLUMNS
Test | Currency (time/reputation/money) | Channel | Reached |
Responded | Committed | Rate | Threshold | Pass/Fail | Cost

Worked example: HR managers and expiring work visas

This is an illustrative case. A founder believes HR managers at companies with 50 to 300 staff lose track of employee visa and permit expiry dates. She writes three test cards.

Time test: she messages 70 HR managers on LinkedIn offering a free 20-minute "permit expiry check". Threshold: 15% book. Result: 13 book, which is 18.6%. Pass.

Reputation test: a one-page checklist on expiry tracking, promoted with a $250 LinkedIn ad budget, asks for a work email. Threshold: 30 work-email signups. Result: 41 signups, 9 of them replying to the follow-up email with their own story. Pass.

Money test: she offers a manual audit of permit dates for €99 to the 13 people who booked calls. Threshold: 3 paid. Result: 2 paid. Fail, by one.

Two of three pass. Her learning card notes that both paying customers had more than 40 international staff, so she narrows her customer group and moves to a concierge test with that segment.

What should you measure, and when do you stop?

The pass numbers above are starting points; set your own on the test card before launch and do not change them afterwards. Stop when all three cards have a learning card. More tests on the same problem rarely change a clear two-out-of-three result.

Which mistakes should you avoid?

• Testing the solution by accident. A page that says "our app does X" tests the app. Keep every asset about the problem.
• Moving the threshold after the result. If 9% booked and your bar was 15%, the test failed. Write down why, then decide.
• Counting clicks as commitment. A click costs nothing. Bookings, work emails, shares and payments cost something.
• Mixing channels in one test. You will not know whether the problem or the channel produced the result.
• Running tests with friends. Warm is fine; personal is not. Friends book calls to be kind.

How can AI speed this up?

An AI assistant can draft test cards from your problem statement, write three versions of the outreach message and produce the one-page checklist for the content test. Ask it to review each test card for loopholes: vague metrics, missing deadlines, thresholds that any result would pass. The judgment calls stay with you: which segment to narrow to and whether a near miss is worth a rerun. To set up the page for the content test, the F/MS landing page module covers layout and copy.

Where does this SOP fit in your validation plan?

Run it after problem statement validation and alongside a problem monetization assessment, which digs deeper into budgets. Once two of three tests pass, move into Solution Validation, starting with a landing page test or concierge testing. After launch, the same test-card habit carries over to the growth experiment SOP. The full stage is on the Problem Validation page.

FAQ

What is a problem validation experiment?

A small, time-boxed test that asks people to spend time, reputation or money on a problem before any solution exists. It has a written hypothesis, one metric and a pass threshold set in advance.

How is this different from a fake door test?

A fake door test shows a button or offer for a specific product and counts clicks. A problem experiment offers help, information or a diagnosis about the problem itself, so it works before you have chosen a solution.

How many experiments should I run?

Three, one for each currency: time, reputation and money. Two passes out of three is enough evidence to start testing solutions.

Do I need statistics for problem experiments?

At this stage, no. You are looking for clear behaviour from 30 to 200 people, not a 2% lift. Save statistical tests for later A/B tests with larger traffic.