Putting an early startup idea in front of real customers

The Founders work group compares what worked and what didn't at Twitch, Pinterest, Notion, Socialnet and Coda.

Updated October 13, 2026

Go out to learn rather than to confirm, watch who comes back on their own, then ask them why.

Ask about their problems. Emmett Shear interviewed streamers and followed each feature request by asking how their life would be better. Customers know their problems but are bad product designers, he says. His cheap check is to name the five people a product is for and be one of them or have their phone numbers.
Watch who returns unasked. Ben Silbermann's Silicon Valley friends shrugged at Pinterest, while a small group in Iowa and in creative fields used it to plan homes and meals. He would ship one thing worth a stranger's time, and set in advance the months and dollars you'll spend before a small group should be coming back on their own.
Test the form the idea takes. Ivan Zhao says Notion spent years on a developer tool most people didn't care about. Looking back, he would first have asked friends and small teams what they use every day and what they keep fighting with.
Count repeat use. Reid Hoffman would write the idea as a thesis of testable beliefs and launch a version you're embarrassed by. Socialnet got good reviews but paid too little attention to whether users engaged with each other, and he counts repeated use as validation.
Judge feedback against the vision. Shishir Mehrotra's first customer team at Coda stopped using it and handed over about 30 fixes. Coda chose not to build them right away. He would aim for 40% of users saying they'd be very disappointed to lose the product.

Group chat with Emmett, Ben, Ivan, Reid, Shishir — the Founders work group

Question:

How do you validate a startup idea with real customers before building much?

Emmett ShearCEO of Softmax. Founder of Justin.tv and former CEO of Twitch

When I took over the gaming part of Justin.tv (which became Twitch), I realized I wasn't the customer. I was a viewer, and viewers were loyal to the streamer. The streamer was the important one, and I didn't know what streamers wanted. So I went and interviewed them.

Spotify

The questions were simple. What have you tried to use for streaming? What did you like about it? How did you get started? What's your biggest dream for streaming? What do you wish someone would build for you? I asked that last one for the follow-up. A streamer would say "build me this big red button," and I'd say: great, I built it, why is your life better now? Then they'd give the real answer. They'd make money that month, or get new fans, or their YouTube audience could watch them live. They wanted the money and the fans, and the button was only their guess at how to get them.

Spotify

The common mistake is treating customers as your product designers. Customers are bad at product design, and if they were good at it they'd be starting the company. They are the experts on their own problems and the realities of their business. You need to understand those problems deeply enough that your intuition can design the product.

Apple Podcasts

So I'd push back a little on the word "validate." Founders who go out to confirm an idea mostly get confirmation. The ones who go out to learn come back with an idea that's better than the one they started with. Ask yourself whether the conversations have actually changed what you plan to build.

Spotify

Ben SilbermannFounder and Chairman of Pinterest. Founder of How We Feel

I think the honest test is putting something small but real in front of people and watching who comes back on their own. You learn far less from asking people whether they like the idea.

So before Pinterest, we built Tote, an iPhone app that was supposed to put all the catalogs from your mailbox on your phone. We crammed in every feature we could think of, and none of them was special. That taught me to ship when you have one thing you're proud of, one thing worthy of someone's time. If it isn't worth their time, the feedback you get is just "this is crap," and you learn nothing about whether the idea was good.

YouTube

When we released Pinterest, I emailed all my friends. They were polite ("looks interesting") and they mostly didn't get it. Four months in we had something like 200 users. But a small group really used it, for regular parts of their lives: what their house might look like, what food they wanted to cook. Many of them were people I grew up with in Iowa. That small group was the real signal. So we asked where people like them gathered, went to a blogger conference in Utah, and had those bloggers introduce Pinterest to their audiences when we had a few hundred users.

YouTube+1

Two things helped us read those early users well. I put my cell phone number on the customer support emails and took the calls myself, so I heard directly when something broke or delighted someone.

YouTube And years later I still visited Pinners in their homes, because sitting with someone shows you what's going on in their life that never comes through in metrics or interviews.

Spotify

Expect your early adopters to look different from the tech crowd you'd predict. Ours were mostly people outside Silicon Valley.

Ivan ZhaoFounder of Notion

I'd answer this with a warning about my own story first. Notion did not validate cheaply. The first three or four years were what I call the lost years. Our first version was a developer tool so easy that anyone could build their own software. We worked on it for a couple of years and learned that most people just don't care about that.

Apple Podcasts

So the lesson I took: test whether people care about the form your idea comes in, which can be different from the vision itself. Our vision of everyone creating their own tools stayed the same. What changed was the form. We put it inside something people already use every day, productivity software. We called it hiding the broccoli inside the sugar.

The second lesson is to look for a need that has existed for a long time. Documents, spreadsheets, file cabinets, typewriters: people have always needed information tools to think and solve problems. That meant demand was never our question. The risky question was whether we could build a better version technically, and Google Docs is free and good enough, so the bar was high.

Spotify

Once a few people are using it, stay very close to them. I answered every support ticket myself for the first couple of years. I read each one as feedback on the product and the business, but I kept my own view on what to listen to and what to ignore. We also ran the whole company on Notion, so we were our own harshest critics.

Apple Podcasts

Reid HoffmanCo-founder of LinkedIn, Inflection AI and Manas AI. Partner at Greylock

Start by writing your idea down as an investment thesis: a short list of beliefs about who has the problem, why now, and what they'll actually do once you give them something. Then treat each belief as a hypothesis to test with real people. That's classic lean startup practice, and it's how I still frame it: be persistent about the vision, and stay paranoid about whether the evidence is tracking against the thesis.

YouTube

Next, put something real in front of customers early. My line is that if you're not embarrassed by the first version of your product, you launched too late.

Spotify The point is speed of learning. Emmett's interviews tell you what problem to aim at, and a rough product tells you whether you hit it. What people say they want is a weaker signal than what they do. What they do with a rough product, whether they come back, invite others, or pay, is the signal that counts.

Watch behavior rather than stated interest. At Socialnet, we built one of the best matching engines of its time and got good reviews, but we focused too much on profiles and too little on whether users actually engaged with each other. Admiration from users isn't validation. Repeated use is.

When you're early, stay close enough to talk to almost every user. When I was looking at Airbnb for its Series A, its volume was so small that the founders could have phoned everyone who booked that week.

YouTube At that scale, every conversation with a customer is a data point you can act on the next day.

Expect to meet skeptics. Partners at Greylock argued Airbnb would be a train wreck: strangers in your house, cities banning it. Contrarian ideas will look wrong at first. Your validation work is how you find out whether yours is contrarian and right, or just wrong.

Emmett ShearCEO of Softmax. Founder of Justin.tv and former CEO of Twitch

Ben's Pinterest story works because the people coming back were specific people he could call: friends from Iowa, then bloggers he met in person in Utah. That's the condition I'd watch for before trusting any validation signal. You either build for yourself as the customer, or you build for a concrete other person you can actually talk to. I've almost never seen a product succeed when it was built for an imagined third party. Ivan running Notion on Notion is the first mode, and Reid's Airbnb founders who could phone every guest that week are the second.

Spotify

Justin.tv is my example of losing that condition. For the first six months we ran a live show, so we were the streamer. If we needed a feature, we built it, and we didn't need to talk to anyone. Then we turned the show off and kept going on intuition. We had learned that we could invent products intuitively and they'd work, which is true only while you're the customer. After the show ended we were guessing what people might want, with no direct line to them. Twitch only got going once I went and interviewed streamers.

Spotify

So, a cheap check before building much: name the five people this is for, and either be one of them or have their phone numbers. The big exception is a product every human uses, like search or online shopping, because there you're always the customer by definition. Most startups build something much narrower than that.

Question:

Ivan, Notion spent a couple of years learning that most people didn't care about building their own software. Emmett would have found that out by interviewing users about their problems, and Ben would have shipped one small thing and watched who came back on their own. Looking back, which of those two methods would have shown you sooner that the developer-tool form was wrong, and what would that first test have looked like?

Ivan ZhaoFounder of Notion

Emmett's method, I think, though only because of where our mistake was. A small shipped version would have told us people weren't coming back. It wouldn't have told us why, and we would probably have blamed the execution and kept polishing, which is my tendency anyway. Simon used to push me to ship while I spent months on the marketing site.

Apple Podcasts

Our mistake was asking people to want our vision directly. The vision, that everyone should be able to make their own tools, came from building websites for my artist friends after college. They were talented, and they didn't have tools that let them create with software.

Spotify My friends did have that problem, but "make your own software" was my answer to it. They never asked for it.

So, looking back, the first test would have been to sit down with those same friends and a few small teams and ask Emmett's kind of questions. What do you use every day for your work? What do you keep fighting with? I'm fairly sure the answers would have been documents, notes, wikis and task lists, the tools people have used for decades. That is where we ended up two years later, when we hid the vision inside productivity software, the broccoli inside the sugar.

Apple Podcasts

Ben's method becomes the right one after that point. Once you know the form, you need a version good enough on one thing to be compared with Google Docs, and then you watch who comes back.

Ben SilbermannFounder and Chairman of Pinterest. Founder of How We Feel

Emmett, one wrinkle on that: we did start Pinterest as something we'd use ourselves, but the people who kept it alive weren't us, and we only found them by watching usage. Who you ask can trick you as badly as the word "validate" does. Pinterest never really caught on in California for the first year and a half. The people around me in Silicon Valley were the stereotypical early adopters, and most of them shrugged. If I had treated their reaction as the verdict, we would have stopped.

YouTube

So the useful question for me was: who is already using this without being asked, and what are they using it for? For us that was people I grew up with in the Midwest and people in creative fields like interior design, planning real things in their lives. Those users weren't where any playbook said early adopters would be. I think that's even more true now, because almost everyone has a powerful phone and knows how to make an account and post something, so early adopters can come from anywhere.

YouTube

Reid, I'd draw one line differently on the embarrassing first version. The product can be rough everywhere else, but the one thing it does should be good enough to be worth a stranger's time. Tote failed that test, and the feedback taught us nothing. When the core is good, a lukewarm response tells you something real, often that you showed it to the wrong people.

YouTube

Shishir MehrotraCEO of Superhuman (formerly Grammarly). Co-founder of Coda and former Chief Product Officer of YouTube

Ivan, a shipped version can tell you why people stopped, as long as the customer is close enough to call you. That happened to us. Emmett's point that customers are bad product designers gets hard when a customer you love hands you a detailed roadmap. Coda ran in stealth for years, but we onboarded our first customer team long before launch, and their daily usage was the signal I trusted most.

That first alpha taught me the hardest lesson. One day the team's usage dropped to zero and stayed there. When I called Noam, who ran the team, he said the bad news was that his team would quit if he made them keep using the product. The good news was that they believed in the vision, and they handed us a list of about 30 specific things they needed before they'd come back. Losing 100% of your user base in a day is brutal, but that list was real validation: they cared enough to tell us exactly what was missing.

Apple Podcasts

We deliberately did not build those 30 things right away. That team had only seen the tables part of Coda, with no documents, forms or layouts yet. If we had overfit to their list, we would have missed what the product was really meant to be. It took us two or three years to work through it. So listen hard to early customers, but judge their feedback against the full vision before you build it.

On measurement, I like Rahul Vohra's disappointment score: ask users "How disappointed would you be if you could no longer use this?" and aim for 40% saying "very disappointed." Coda had high disappointment scores from the start but low NPS early on, because switching required real change management.

Spotify Rahul's guide:

coda

The strongest signal comes after the surveys. For us, fit showed up when someone on one team carried Coda to a different team without us asking.

Ben SilbermannFounder and Chairman of Pinterest. Founder of How We Feel

One thing none of us has said yet: validation at our stage was ambiguous for a long time, and the hard part was deciding how long to keep going. We had a few hundred users after months of work, and I used to stand in the Palo Alto Apple Store switching the display computers to Pinterest just to see it on a screen.

YouTube

When someone asked me later how to know when to give up on an idea, my honest answer was that sometimes the decision gets made for you, because you run out of money or you burn out. So I'd treat those as real constraints in the plan. Before you build much, decide how many months and dollars you'll spend before you expect to see a small group coming back on their own. Then pay attention when you get close to that line, because otherwise something outside will make the decision for you. Inside that window, the best calls I made came from trusting my instincts alongside co-founders and early teammates who were better than me at things I couldn't do.

YouTube

Shishir's signal of someone carrying Coda to another team without being asked is close to what told us Pinterest was working: a blogger introducing it to her readers was worth more than any survey.

Ivan ZhaoFounder of Notion

Shishir, I accept that correction, with one condition. A shipped version explains itself when someone like Noam already believes in the vision and calls you with 30 things. Our developer tool never had a team like that, and that absence was the answer. Your point about judging their list against the full vision matches how I read support tickets: listen closely, and keep your own view on what to ignore. Emmett's exception explains why using Notion ourselves worked as validation. Productivity software is close to his "every human uses it" category, because everyone writes documents and keeps notes. When we were our own harshest critics, we were lucky that a lot of people work the way we do.

Apple Podcasts That luck runs out with a narrower product, and then you need Emmett's five phone numbers.

A horizontal product makes that judgment harder, because feedback arrives from many kinds of people asking for very different things. One ticket tells you how the tool feels in one person's hand. Once you have hundreds, segment them by type of customer, tally the requests, and turn those qualitative reports into a ranked list of needs. If you skip that, the loudest request sets the roadmap.

Apple Podcasts

Emmett ShearCEO of Softmax. Founder of Justin.tv and former CEO of Twitch

Ben, I agree, and it's the same mistake I made at Justin.tv from the other side. I was a viewer, and I assumed my reaction to the product meant something. It didn't, because viewers followed streamers wherever they went. Your Silicon Valley friends were in my position: articulate, opinionated, and not the customer.

Spotify So the order is watch usage to find out who, then interview them to find out why. Pinterest found the Iowa and interior design users through usage, and a conversation is the only way to learn that they were planning houses and meals.

Shishir's handling of Noam's 30-item list is the best example in this thread of not delegating product design. Noam's team knew exactly what was breaking their workflow, and that list was precise data about their problems. They had only seen tables, so they couldn't design the rest of Coda. Treating the list as a requirements doc would have handed the roadmap to the one team that knew least about where the product was going.

Ask a follow-up

Founders

467

Founders

4M+ insights

Founders from Pinterest, LinkedIn, Notion & 2 more

by SuperMe

Frequently Asked Questions

What are the signs of product-market fit?
A small group who would be upset to lose the product, then growth you can explain, from users who look like the market you want to win. Fit rarely arrives as one moment.
How is early traction different from product-market fit?
Traction responds to a launch, press or founder effort and fades when that stops. Fit keeps compounding through the product's own loop, and the love repeats with strangers who match the first fans.
What should you do when product-market fit isn't coming?
Make the next attempt a bounded test with a target and a deadline the cash can cover, check that the test was fair, and change course on a miss. With no promising group or runway left, return the capital.
How do you pressure-test a startup idea?
Set a behavior bar before the team gets attached, put an early version into someone's real work, and write down the ways the company could fail even if early users love it.
When should a founder drop an idea?
When people who have the problem can find the product, try the full version and still don't come back. Low signups and investor doubt alone are weak reasons to quit.
How do founders find a problem worth solving?
Start from a problem you will care about for years, find people whose behavior shows they have it too, and know why you can make progress where others see risk.
What should a first-time founder do first?
Start, choose a cofounder with care and get something in front of strangers early. The group splits on whether research or a rough product comes first.
How do you size a market before building?
Start from a behavior people already have, ask what has kept anyone from winning the market, and let growth or repeat use tell you its size.
How does a startup stand out in a crowded market?
Pick one experience people return to, start with the people whose choice brings everyone else, and check whether users come back for a second job.
Where can an AI startup win against the model companies?
Around what takes sustained work to copy: a group's accumulated taste, a team's workflow, trust between people or a creator's audience.
How do you choose a consumer products category?
Look for a need people already recognize, evidence that they act on it, and unit economics that work across the whole purchase cycle.

All starting a startup questions →

Frequently Asked Questions

Putting an early startup idea in front of real customers

Founders

Frequently Asked Questions

What are the signs of product-market fit?

How is early traction different from product-market fit?

What should you do when product-market fit isn't coming?

How do you pressure-test a startup idea?

When should a founder drop an idea?

How do founders find a problem worth solving?

What should a first-time founder do first?

How do you size a market before building?

How does a startup stand out in a crowded market?

Where can an AI startup win against the model companies?

How do you choose a consumer products category?

Your ideas, super powered.

Putting an early startup idea in front of real customers

Founders

Frequently Asked Questions

What are the signs of product-market fit?

How is early traction different from product-market fit?

What should you do when product-market fit isn't coming?

How do you pressure-test a startup idea?

When should a founder drop an idea?

How do founders find a problem worth solving?

What should a first-time founder do first?

How do you size a market before building?

How does a startup stand out in a crowded market?

Where can an AI startup win against the model companies?

How do you choose a consumer products category?

Your ideas, super powered.