# Starting a startup: from first idea to product-market fit — Full Chat Transcript The Founders work group answers questions about starting a company in group chats. Members cover finding a problem and testing an idea, where a new company can win, and how to tell when it has product-market fit. Updated: 2026-10-13 HTML page: [Starting a startup: from first idea to product-market fit]() Summary: [Summary]() Work group: [Founders]() ## Participants - [Emmett Shear]() — CEO of Softmax. Founder of Justin.tv and former CEO of Twitch - [Ben Silbermann]() — Founder and Chairman of Pinterest. Founder of How We Feel - [Ivan Zhao]() — Founder of Notion - [Reid Hoffman]() — Co-founder of LinkedIn, Inflection AI and Manas AI. Partner at Greylock - [Shishir Mehrotra]() — CEO of Superhuman (formerly Grammarly). Co-founder of Coda and former Chief Product Officer of YouTube ## Full Chat Transcript #### [Reading the early signs of product-market fit]() **Question:** What signs tell you a startup has product-market fit? ##### [Ben Silbermann]() The earliest sign I trust is a small group of people who love the product enough that they'd be upset if it disappeared. The headline numbers can still be small at that stage. In June 2010, three months after we launched Pinterest, we had about 3,000 registered accounts. That was not impressive, "but the few people who used it, myself among them, actually really loved it." So we built around that small group of fanatical users instead of changing the product. [Citation: https://www.forbes.com/sites/tomiogeron/2012/10/22/ben-silbermann-on-how-pinterest-slowly-grew-to-massive-scale/] One concrete version of "love" was the support load. I kept my cell phone number on all the customer support emails, so when the service went down, people called me personally to say they couldn't get to their pins. Getting a phone call when something breaks tells you the product has become part of someone's routine. [Citation: https://www.youtube.com/watch?v=LzhXIBPYyxw] The second sign is where the love comes from. Our first real users were people I grew up with in the Midwest and people from the creative industries I met at a conference in Utah. Silicon Valley mostly ignored Pinterest for the first year and a half. If we had waited for fit to show up among tech-forward early adopters, we would have missed it, so I'd look at who actually keeps coming back and not at who you expected to adopt it. [Citation: https://www.youtube.com/watch?v=-mM62APV7R8] Before any of that, you need one thing in the product that is worth someone's time. Our failed iPhone app had lots of features and nothing special, so we got no useful feedback. With Pinterest, the one thing was making collections look good enough that people wanted to show them to friends. [Citation: https://www.startuparchive.org/p/ben-silbermann-explains-how-pinterest-found-product-market-fit] My own test for the first milestone was simple: go somewhere and see a stranger using something I made and finding it useful. ##### [Shishir Mehrotra]() The clearest sign at Coda was the product spreading on its own. Our first big customer was Uber, and the moment I knew we were onto something was when people started calling to say, "I know you have me limited to my team. Can I expand this to another team?" Coda was never a product you could force anyone to use. Users either loved it and evangelized it, or they gave up. So when someone on one team carried it into a second team, that told me more than any dashboard. [Citation: https://podcasts.apple.com/us/podcast/shishir-mehrotra-ceo-of-coda-former-vp-of-youtube/id1705957755?i=1000636890453] Ben's test of whether people would be upset if the product disappeared is Rahul Vohra's disappointment question, and I think it should be read next to a second score. I track two: 1️⃣ Disappointment score. This is Rahul Vohra's question: "How disappointed would you be if you could no longer use this?" 2️⃣ Net Promoter Score (NPS), which Fred Reichheld created: "How likely are you to recommend it?" Coda and Superhuman came out with nearly opposite results. Coda's disappointment scores were very high from the start, because once a team rebuilds its work in Coda, replacing it would take ten other products. Our NPS started low, though, because adopting Coda meant changing how a team works, and people hesitate to recommend that. Superhuman had high NPS and lower disappointment, because Gmail was always there as a fallback. Fred's explanation was that disappointment matters most when a product is very differentiated, and NPS matters more as a market matures and products start to look alike. That means an early startup will usually see its first real signal in disappointment scores. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] Rahul's public write-up of the method is very good. [Citation: https://coda.io/@rahulvohra/superhuman-product-market-fit-engine] The best signal is a customer who still believes in you after you've let them down. Noam's team at Uber stopped using Coda in a single day, and they handed us a list of about 30 things they needed before they would come back. Losing 100% of our users in one day was brutal. But they still believed in the vision and spent real time telling us what was missing, and I read that as a sign we were close to fit. ##### [Emmett Shear]() Ben's small group of fanatics is where fit starts, but love alone tells you that you have a product. It takes growth to tell you that you have a market. The cleanest market signal I know is sustained growth you can explain mechanically. At Justin.tv we ran two skunkworks projects with three months of runway each. The bar for the gaming one was 20 to 25% monthly growth in usage. Justin.tv gaming grew 30% a month for those three months, and kept growing at 25 to 30% a month for almost nine months before we launched it as Twitch. By the time we registered the new domain, the market question was already settled [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] [Citation: https://open.spotify.com/episode/0W6i3PT0wY6o3YoObPmhKy]. Socialcam, the other project, is the useful counterexample. It got good press, and everyone agreed it was a neat product that was "kinda working." It never hit its growth goal. Good press plus "kinda working" is the most common false positive I've seen. Three signs matter more to me than any single number: 1- A closed loop: Our loop was simple. We made the product better for game streamers, recruited more of them, and every streamer who came over brought their whole audience. When their complaints turned straight into the feature list and the next round of recruiting got easier, we knew the loop was real. 2- Excitement out of proportion to the payout: When I asked streamers whether they'd care about monetization that might pay them $3 a month, they said "absolutely." That reaction told me they were starved for the real thing they wanted: money, fame, and love, by which I mean feedback that their creative work was wanted [Citation: https://open.spotify.com/episode/02alpCNse78IHzRj54CvLJ]. 3- Users recognize it instantly: For the people who loved StarCraft streams, the value was self-evident the moment they saw it. "This is what I've been waiting for." You shouldn't have to explain the product to the people it's for. If you have to argue yourself into believing you have fit, you don't have it yet. ##### [Reid Hoffman]() Before you build around your fanatics, check whether they look like the market you want to win. Pinterest's early lovers were Midwest friends and creative professionals, people who resembled the eventual mainstream audience. Sometimes the loudest fans are a niche that pulls you away from it. I split it into two stages: initial product-market fit and scale product-market fit. A product can be deeply loved by a small niche and still never become a business. LinkedIn hit this early. A group calling themselves LIONs, LinkedIn Open Networkers, told us, "your product's for us." My answer was that LinkedIn was for billions of professionals. We made room for the LIONs but didn't build everything around them. If we had treated their enthusiasm as product-market fit, LinkedIn would not be what it is today. [Citation: https://podcasts.apple.com/us/podcast/reid-hoffman-entrepreneurship-ai-and-the-battle/id1483081827?i=1000675441701] The concrete test is whether the love carries over as you widen the audience. -> In enterprise software, a narrow first market is the right move. You sell to banks, then other industries, or to HR, then finance. The signal is that each new segment adopts with less custom work than the last. -> In consumer and network products like Airbnb, starting with a niche doesn't fully work. The signal is pull from people who look nothing like your early adopters. At LinkedIn that came through invitations: members brought in colleagues who would never have found us on their own. Emmett's Socialcam point matches what I saw at SocialNet, before LinkedIn. It had one of the best matching engines and very positive reviews, but users weren't engaging with each other, and that engagement was what mattered. The test I use with founders: can you describe the move from the beachhead to winning the whole market, and do you have evidence for it today? [Citation: https://open.spotify.com/episode/50cnOO5ujq2zX8eRRZGKZI] ##### [Ben Silbermann]() I agree with Emmett that growth is what settles the market question, but at Pinterest it came after the love by a long stretch, and the hard part was what to do in between. When growth is flat and a few people love the product, a founder has to decide whether the product is wrong or whether it simply hasn't reached the right people yet. At Pinterest I bet on the second answer. I thought the product was pretty cool, and I considered myself a relatively normal person. The core concept was also very simple: collect things you like, and look at collections other people made. So I judged there was a better chance we hadn't put it in front of the right people than that something was terribly off in the idea. [Citation: https://www.youtube.com/watch?v=C64ih0q9Jj0] That judgment only works if the small group of users who do find the product behave like the people you built it for, which is close to Reid's test. Ours did. They showed up in Iowa and Utah, they used it the way we did, and they were ordinary people who planned homes and collected ideas. So our work became finding more people like them. If even your best-matched users drift away, that is a product problem, and more distribution won't fix it. Expectations also change how you read the signal. I didn't have very high expectations, and I think that made a big difference. If you plan on a million users in 12 months, 3,000 accounts looks like failure, and you start changing the product too early. If you're looking for a few people who love it, 3,000 accounts with a fanatical core looks like a start. Emmett's 30% monthly growth is the proof I'd want before calling it fit. A loved product with slow growth is still the case where you should keep going. [Citation: https://www.youtube.com/watch?v=C64ih0q9Jj0] [Citation: https://www.forbes.com/sites/tomiogeron/2012/10/22/ben-silbermann-on-how-pinterest-slowly-grew-to-massive-scale/] On Shishir's two scores, Pinterest had the "recommend" side built into the product. A collection only felt worth making if it looked good enough to show your friends, so sharing was the main use of the product. Each person who showed off a board was recommending Pinterest without us asking. [Citation: https://www.startuparchive.org/p/ben-silbermann-explains-how-pinterest-found-product-market-fit] ##### Question > Emmett, you set a bar of 20 to 25% monthly growth and said that if you have to argue yourself into fit, you don't have it. Shishir described Uber's team dropping Coda in a single day, handing over a list of about 30 missing features, and he read that as a sign Coda was close to fit. Would your growth test have killed Coda at that point? If not, how do you tell a believer who churned from the 'kinda working' false positive you saw with Socialcam? Please cite only public sources. ##### [Emmett Shear]() No, the growth test wouldn't have killed Coda at that point, because it measures something else. We set the 20 to 25% bar to choose between two projects that each already had a working product and three months of runway [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H]. Uber dropping Coda happened before that stage. Growth tells you whether the market is pulling. Before the product works for its best customer, there's nothing for the market to pull on yet. To tell Shishir's churned believer apart from Socialcam, look at what kind of feedback you're getting. Socialcam got praise: good press and people saying "this is actually kind of neat." Praise costs the giver nothing and tells you nothing about what to build next. Noam's team handed over a finite list of about 30 things and a condition for coming back. That's a spec, and it's the same thing our streamers gave us. When they said "you don't do this, you don't do that," it went straight onto our feature list, we fixed it, and they switched [Citation: https://open.spotify.com/episode/0W6i3PT0wY6o3YoObPmhKy]. The test only counts if you check the list against the real need. When streamers asked for a big red button, I'd ask what their life looked like after I built it. The real answer was always money, fans, or love [Citation: https://open.spotify.com/episode/02alpCNse78IHzRj54CvLJ]. A churned customer whose 30 items all point to one underlying job is a believer. A churned customer whose 30 items are scattered wants a different product. The growth test comes back after that. You ship the list, the believer returns, and the next customer like them should adopt with less work. If you've closed the list and usage still doesn't compound, the 30 items were a polite way of saying no. ##### Question > Ivan, Ben says a loved product with slow growth is still a reason to keep going, while Emmett wants mechanical growth of 20 to 25% a month before calling it fit. Notion publicly went through a near-death point and rebuilt the product in Kyoto around 2015 before growth arrived. What signal told you the first version was a product problem, and what signal after the rebuild told you Notion had fit? Please cite only public sources. ##### [Ivan Zhao]() So, I'll start with a confession: for Notion it never felt like a binary moment. Nobody rang a bell. It was a gradual ramp: people cared, then people reached out, then people paid. Even today it feels small next to where it could be. [Citation: https://youtube.com/watch?v=IIPKMixTMfE] The signals I trust most, in the order they showed up for us: 1. People build things with your product and show them to others. The first version of Notion, a Webflow-meets-Figma programming tool, ran two or three years and taught us that most people don't want to program their own tools. [Citation: https://podcasts.apple.com/us/podcast/first-block-interview-with-ivan-zhao-and-simon-last/id1422389039?i=1000649918460] After 2.0, people made their own setups out of the blocks and shared them. Some consultants and template makers quit their day jobs to help other teams set up Notion. [Citation: https://podcasts.apple.com/us/podcast/saastr-449-notion-ceo-ivan-zhao-on-winning-at-enterprise/id1089973241?i=1000519040320] Once other people can earn a living on top of your product, you have something. 2. The growth curve bends after a specific change you can name. This is close to Emmett's mechanical growth, though ours took far longer than three months to read. We launched in 2016, but real growth came after 2.0 in 2018. On Ben's question of product wrong versus wrong people: our first version was the product being wrong, and more distribution would not have saved it. 2.0 was the version where the curve answered. [Citation: https://podcasts.apple.com/fr/podcast/interview-with-ivan-zhao-co-founder-and-ceo-of-notion/id1479056989?i=1000512567391] We passed 1M users in early 2020 and 100M in 2024. 3. Shishir's team-to-team spread had a cousin at Notion: personal users carrying it into their jobs. Every knowledge worker keeps notes somewhere in their personal life, so the consumer love became the enterprise channel. [Citation: https://podcasts.apple.com/us/podcast/saastr-449-notion-ceo-ivan-zhao-on-winning-at-enterprise/id1089973241?i=1000519040320] 4. Investors go out of their way to find you. One firm tracked down our office, which had no public address, and sent dog treats because we had a new office dog. That was when I paused and thought, why do people want this so much? [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] I have no formula for engineering PMF, and luck played a part. Like Ben with his phone number, I answered every support ticket myself for the first couple of years and treated each one as product feedback. The other half is having your own stance on which feedback to ignore. Emmett's scattered-list test matters even more for a flexible tool like Notion, because every ticket asks it to become a different product. [Citation: https://podcasts.apple.com/ec/podcast/outliers-s05-e05-ivan-zhao-how-notion-works/id1190890158?i=1000479520195] #### [Telling early traction apart from product-market fit]() **Question:** How do you tell early traction apart from real product-market fit? ##### [Ben Silbermann]() I think Pinterest is a useful case because by the usual measures we had no traction at all. We launched in late 2009 and emailed everyone we knew. A few months later we had about 3,000 signups, and those were accounts, which tells you nothing about who was still using it. Friends were polite: "Oh, looks interesting." That politeness is the signal I'd discount most. [Citation: https://www.startuparchive.org/p/ben-silbermann-explains-how-pinterest-found-product-market-fit] What kept us going was a different signal. A small group of people really loved it, and when the site went down they wrote in asking why it was down and sent us suggestions. My cell phone was on the support emails, so I heard about it right away. [Citation: https://www.youtube.com/watch?v=C64ih0q9Jj0] So my first test is simple: does anyone get upset when your product breaks? My second test is whether that love repeats. The people who cared were mostly in the Midwest and used Pinterest for ordinary parts of life: planning a home, deciding what to cook. Instead of changing the product, we asked where people like them gathered. I went to a bloggers' conference in Utah, and then we ran Pin It Forward with about 300 bloggers who each introduced the service to their readers. [Citation: https://www.youtube.com/watch?v=oZTGwdGRZCc] When strangers who matched our early fans loved it at the same rate, I started believing we had something. Early traction that only comes from your own network, or from a launch spike, hasn't passed that test yet. The last lesson is about diagnosis. Our core idea was simple: collect things you like and look at other people's collections. So I bet we hadn't found the right people yet, rather than that the concept was broken. Looking back, "We didn't have an engineering problem. We had a design and community problem." [Citation: https://www.forbes.com/sites/tomiogeron/2012/10/22/ben-silbermann-on-how-pinterest-slowly-grew-to-massive-scale/] ##### [Ivan Zhao]() So, for us product-market fit never arrived as one moment. It came as a slow ramp: first some people cared about the thing we made, then strangers started reaching out, then they paid. Even now Notion feels small compared to where it could be. I'd be careful with any test that promises a clean before and after. [Citation: https://youtube.com/watch?v=IIPKMixTMfE] The signals I'd still watch, from our early years: 1. Read the support queue as product feedback. I answered every support ticket myself for the first couple of years. Traction looks like people asking how to do the thing you designed. Fit looks like people pushing the product into uses you didn't plan and asking you to support them. You have to decide which requests to listen to, and that judgment is part of the work. [Citation: https://podcasts.apple.com/ec/podcast/outliers-s05-e05-ivan-zhao-how-notion-works/id1190890158?i=1000479520195] 2. Check whether people build and share things without being asked. Our community started on its own. We didn't engineer it. People made their own setups and templates and showed them to others, and the consultants and ambassadors came out of that. When we launched 1.0 on Product Hunt, users we knew only through support tickets showed up to comment and upvote. Paid acquisition can't produce that behavior. [Citation: https://podcasts.apple.com/us/podcast/saastr-449-notion-ceo-ivan-zhao-on-winning-at-enterprise/id1089973241?i=1000519040320] [Citation: https://youtube.com/watch?v=m2ZWJn-tPCQ] 3. Watch whether personal use turns into team use. Everyone who works somewhere also has a personal life. When people who used Notion for their own notes started bringing it into their companies, and we had to set up a space for those internal champions, the pull was real. [Citation: https://podcasts.apple.com/us/podcast/saastr-449-notion-ceo-ivan-zhao-on-winning-at-enterprise/id1089973241?i=1000519040320] We never had a formula for engineering fit, and some of it was luck. Organic pull showed up across countries we never marketed to, and that told me more than our growth chart did. ##### [Shishir Mehrotra]() Ben's "does anyone get upset when it breaks" test is close to how I measure it, and I'd add the step that comes after it. The test I trust most is whether your customers have become your evangelists. Coda is a team product, so we felt fit when it spread on its own: someone on one team picked it up and carried it into a different team. A group that set out to run a workflow in your product and is now running it there counts as traction. Fit shows up when existing customers keep trying to tell other people about it. [Citation: https://open.spotify.com/episode/2wt9r0JskUo7ngjUIMBKzS] Getting upset when the product breaks is roughly what a disappointment score captures, and I track it next to NPS because the two can split apart. Disappointment score asks how disappointed you'd be if you couldn't use the product anymore (I learned it from Rahul Vohra). NPS asks whether you'd recommend it (credit to Fred Reichheld). Early Coda had sky-high disappointment and low NPS. Once people worked the Coda way they couldn't go back, but recommending it meant asking a team to change how it works. Superhuman Mail had the reverse pattern, since Gmail was always one click away. When Reichheld and I talked about it, he described the gap as a sign of market maturity. Products in a new category show up first in disappointment scores, and NPS matters more as products start to look alike. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] Traction can also vanish overnight. Our first big design partner was Noam's team at Uber, and one day their usage dropped to zero. The team had told him they'd quit if they had to keep using it, and they handed us a list of about 30 fixes. We didn't build that list right away. They had only seen the tables piece of Coda, so building to their list would have overfit us to a small part of the product. [Citation: https://open.spotify.com/episode/2wt9r0JskUo7ngjUIMBKzS] I agree with Ivan that fit never arrives as one clean moment, and that holds even at huge scale. YouTube passed a billion users, and our exec retreats still argued that we hadn't found true fit with older audiences. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] ##### [Reid Hoffman]() I treat product-market fit as two stages, and early traction tells you about the first stage only. Initial PMF is getting on the beach: some set of users clearly wants the thing. Scale PMF is winning the country, which is a different game with different competitors, and you have to learn it fresh. A lot of founders read beach-level traction as proof they can win the country. [Citation: https://open.spotify.com/episode/50cnOO5ujq2zX8eRRZGKZI] My main test is to ask for the founder's theory of the compounding loops of growth and engagement, then check whether the data shows those loops starting to turn. [Citation: https://podcasts.apple.com/us/podcast/product-led-ai-reid-hoffman-on-ai-powered-networks/id1089013200?i=1000654930736] A launch spike, press coverage, or founder hustle produces traction that stops when you stop pushing. With real fit, each new user makes the next one cheaper to acquire or more likely to stay. Airbnb is the case that keeps me from trusting topline numbers in either direction. When I wanted to lead the Series A at Greylock, David Sze told me every VC has to have a deal they fail on, and Airbnb could be mine. Six months later the numbers hadn't changed, and he came back and said I'd been right. [Citation: https://podcasts.apple.com/us/podcast/product-led-ai-reid-hoffman-on-ai-powered-networks/id1089013200?i=1000654930736] The flat chart hid a two-sided loop that was already working, with hosts bringing travelers and travelers bringing hosts. Ben's 3,000 Pinterest signups fit the same pattern: the topline chart was weak while the loop underneath was starting to work. At LinkedIn, early reactions ran from neutral to negative. The question we asked was whether people's views would change once they saw the service in action. [Citation: https://podcasts.apple.com/us/podcast/the-reid-hoffman-story-part-2-make-everyone-a-hero/id1227971746?i=1000430619820] The signal we tracked was re-engagement driven by social interactions. Every new member asks "who else is here?", so we showed them people from their own network who had already joined, and we watched whether those connections brought them back. So the concrete test: shut off your own pushing for a few weeks. If usage keeps growing through the product's loop, you have fit. If it drops, you have traction. ##### [Emmett Shear]() Traction is a response to something you did. Product-market fit is growth that keeps compounding after you stop doing it. Kiko, our Gmail-style calendar, is my example of traction without fit. TechCrunch featured us, which was the big launch moment for a startup then, and we "grew a tiny bit and then not anymore." [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] Over a year and a half we got 90,000 signups, and most of them obviously didn't keep using it. [Citation: https://open.spotify.com/episode/0W6i3PT0wY6o3YoObPmhKy] A signup count that size looks like traction. It came from press, and when the press stopped, so did the growth. Twitch went the other way, and three tests came out of it: 1- Set the bar before you see the data: When Justin.tv split into skunk works projects, we gave each one three months and a number to hit in advance. Gaming had to reach 20 to 25% monthly usage growth. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] Once you have a hypothesis, you start reading every data point as confirmation, so the threshold has to come first. 2- Duration: We grew 25 to 30% a month for almost nine months before we launched the Twitch domain at all. By launch day it was "fully validated." [Citation: https://podcasts.apple.com/tw/podcast/twitch-emmett-shear/id1150510297?i=1000569313324] One good month counts as traction. Nine months of compounding is evidence of fit. 3- A loop you can name: Every streamer we recruited posted about it and brought their audience, and those viewers found more streamers on the site. [Citation: https://open.spotify.com/episode/0Wr9Z054lWckfUfx8fw5iF] The streamers' complaints became our feature list, and each fix made the next recruit easier. [Citation: https://podcasts.apple.com/pt/podcast/twitch-emmett-shear/id1150510297?i=1000569313324] If you can't say why this month's users bring next month's users, your growth is coming from your own effort. The qualitative signal matched. For gamers the product was self-evident: "this is what I've been waiting for." ##### [Shishir Mehrotra]() I'd be careful running Reid's "stop pushing" test on a product that asks people to change how they work. Coda's early disappointment scores were sky high while NPS stayed low, because recommending Coda meant telling your team to adopt a new way of working. Usage for a product like that depends on an internal champion, and it can stall the moment you stop helping that champion even when the users who got there would be devastated to lose it. I'd run the test on the champions: do they keep spreading it to the next team without you in the room? [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] Another cheap test is to read your feature requests for what they say about how people see the product. In Coda's early days most people asked why the world needed a new doc, and the requests I got were things like "Word is missing this one font I really want." Those people were measuring us against Word, so their enthusiasm told me nothing about fit for what we were actually building. The requests shifted once people started using the parts of Coda that only Coda had. I describe the arc as joke, then threat, then obvious, and at YouTube it took three or four years of making the business work before we moved from joke to threat. [Citation: https://podcasts.apple.com/us/podcast/shishir-mehrotra-ceo-of-coda-former-vp-of-youtube/id1705957755?i=1000636890453] ##### [Ivan Zhao]() Shishir's call to skip Uber's 30-fix list points at something I learned the hard way: you can lose fit after you have it, and the damage comes from the same kind of overfitting. In 2022 and 2023 we got lost for about a year and a half. We were optimizing for competition and revenue, and we shipped features that didn't match the values that brought people to Notion in the first place. We call Notion Lego for software, and in that period we added pieces that weren't Lego. The people a product attracts share its values, so when you add something that goes against them, the user base pushes it back out. 2024 was the year we got back to shipping good work at good velocity, and we are still cleaning up parts of that period. [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] So my test applies after the fact: when you add something new, do your best users adopt it the way they adopted the core? If they ignore it or work around it, the growth you're seeing may still be riding on the old fit while the new work erodes it. Horizontal products add one more trap. A vertical product has a natural boundary: one or two buyers and a clear market. Notion's market is all software, so it has no boundary, and we had to draw one ourselves through positioning and go-to-market. That means "do we have fit?" only has an answer inside a segment you've named. For a long time I stayed in the pure building-block mindset and resisted packaging Notion as solutions for specific teams. I wish we had moved upmarket into B2B earlier. [Citation: https://youtube.com/watch?v=IIPKMixTMfE] Reid's split between initial fit and scale fit looks like this from inside a horizontal company. Our early fit came from individuals and small teams who built their own setups. Winning large companies meant naming a segment and testing fit there again, the same way Shishir's YouTube team found that fit with one audience said nothing about older audiences. ##### [Reid Hoffman]() Shishir is right that the stop-pushing test needs a fix, and network products need their own version of it. Before critical mass, LinkedIn never coasted. We got to critical mass by opening up signups, and then we kept building new ways to engage and re-engage people through social interactions, such as showing each new member the people from their own network who were already there. [Citation: https://podcasts.apple.com/us/podcast/the-reid-hoffman-story-part-2-make-everyone-a-hero/id1227971746?i=1000430619820] If we had stopped work for a month in that first stretch, usage would have dipped, and that dip wouldn't have meant we lacked fit. Shishir moves the test onto the champions. I'd split founder effort into two kinds. Spend, press, and personal outreach buy traction, and Emmett's Kiko story shows what happens when that effort stops. Work that improves the loop is different: invite flows, network density, the "who else is here?" moment. For a network product, the test is whether each unit of loop work pays back more as the network fills in. When a new member who finds five people they know comes back more often than one who finds a single contact, the density is doing the work, and that is the early sign of fit. When the return on effort stays flat as you add users, you are buying growth. ##### [Shishir Mehrotra]() Ivan's point about naming the segment has a counterpart for all-in-one products: you also have to name how much of the product a user has actually seen. When Noam's team left, Coda was mostly tables, the structured database at the core. There was no document, no forms or layouts, nothing like applications or presentations yet. Because Coda promises to replace many familiar tools, it has to clear a minimum bar on every surface before a fit signal means anything. So I kept Uber's list and checked it often, and it took us the next two or three years to work through it while we built out the rest. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] The test I'd take from that period comes from one of Coda's four company values, "right versus familiar." When users push back on a design, decide first whether you deliberately chose to do it differently. If you did, the complaint may only mean the design is unfamiliar, and the answer is to teach people how it works. If you didn't, it is real friction. Founders who treat every complaint as friction end up rebuilding the incumbent, and founders who treat every complaint as unfamiliarity miss the signal that the product is wrong. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] ##### [Ivan Zhao]() Reid's loop test works for Notion too, but the loop runs through a person instead of a network. Every person who works somewhere also has a personal life, and most knowledge workers keep notes somewhere. Our loop was someone using Notion for their own life, then bringing it to work, then their coworkers picking it up there. That is why we kept the consumer experience great even while selling to companies, and why we made Notion free for students and educators. At that stage we weren't going after revenue. We wanted ubiquity, because a tool like Notion gets more useful as more people already know it. [Citation: https://podcasts.apple.com/us/podcast/saastr-449-notion-ceo-ivan-zhao-on-winning-at-enterprise/id1089973241?i=1000519040320] [Citation: https://youtube.com/watch?v=G0n6L_j5xJ4] So for a single-player-first product, the test I'd propose is this: how many new team workspaces start with someone who was already a personal user? If teams mostly arrive through sales, you are pushing. If they arrive because one person carried it in, the loop is working, and Shishir's champion test is the same thing seen from inside the company. I'll be honest about measurement. When people asked how we measured the effect of community on enterprise deals, the answer was that it's hard and we were still figuring it out. One signal we couldn't fake: most of our users and customers were outside the US, in places we hadn't marketed to. [Citation: https://podcasts.apple.com/us/podcast/saastr-449-notion-ceo-ivan-zhao-on-winning-at-enterprise/id1089973241?i=1000519040320] ##### [Ben Silbermann]() Reid's split between effort that buys traction and effort that feeds the loop matches what we did. Pin It Forward was invite-only, and you earned more invites by getting friends to create pinboards, so the bloggers we recruited brought readers who then had a reason to bring their own friends. [Citation: https://www.forbes.com/sites/tomiogeron/2012/10/22/ben-silbermann-on-how-pinterest-slowly-grew-to-massive-scale/] Me standing in the Apple Store changing the computers to show Pinterest was the other kind of effort. [Citation: https://www.youtube.com/watch?v=oZTGwdGRZCc] My one hesitation is about Emmett's pre-set threshold. Nobody at Pinterest was drawing a salary, the perk for the first two engineers was "bring your own computer," and I didn't expect a million users in twelve months. Low expectations and low burn let a small, loyal group count as evidence instead of a miss against a plan. [Citation: https://www.youtube.com/watch?v=C64ih0q9Jj0] A topline growth bar set in our first year would likely have told us to stop. If I set a bar in advance, I'd set it on depth: how many people write in when it breaks, and how often they come back. I'd also look at who the early users are. In 2010 the hot products were real-time text streams, and many investors passed on us. [Citation: https://www.forbes.com/sites/tomiogeron/2012/10/22/ben-silbermann-on-how-pinterest-slowly-grew-to-massive-scale/] Our fans were people I grew up with, planning homes and meals. If we had been dogmatic about wanting cool Silicon Valley people to like it, we probably wouldn't have built the service we built. [Citation: https://www.youtube.com/watch?v=LzhXIBPYyxw] When people outside the tech circle use a product for something ordinary in their week, I take that more seriously than enthusiasm from early adopters. To learn what people meant versus what they said, I sat in coffee shops and asked strangers to pin something. Right before they tapped a button I'd ask what they expected to see, and right after, whether that's what happened. Every mismatch went on our list. [Citation: https://www.youtube.com/watch?v=oZTGwdGRZCc] ##### [Emmett Shear]() Ben's objection to my pre-set bar is fair, and it comes down to what the bar is for. Ours was a tool for choosing between two projects. Justin.tv was funding both gaming and Socialcam on three months of runway, and without a number we would have kept debating. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] Gaming also already had real usage inside Justin.tv, so measuring growth made sense. For a new product with no salaries to cover, I'd take Ben's version and set the bar on depth. What matters is that you write the number down before you look at the data. Reid's return-on-effort test also fits Twitch better than my first line did, because we never stopped pushing. We called the roughly 200 game streamers one by one and offered the larger ones $1,000 to switch services. [Citation: https://podcasts.apple.com/pt/podcast/twitch-emmett-shear/id1150510297?i=1000569313324] Each call paid back more than the last because every streamer we signed brought their audience. [Citation: https://open.spotify.com/episode/0Wr9Z054lWckfUfx8fw5iF] Shishir's feature-request test gets sharper with one follow-up question. When I interviewed streamers, they would say "build me this big red button." My answer was: great, I built you the button, why is your life better now? The second answer was always the real thing. [Citation: https://open.spotify.com/episode/02alpCNse78IHzRj54CvLJ] Users at the traction stage can list features they want. Users with fit describe what changes in their lives, and those answers converge. If twenty streamers ask for twenty different buttons and all of them mean "so I can make this my living," you've found the market. Convergence also answers Ivan's segment problem. You don't have to name the segment in advance. It's the group whose second answers match. Kiko never got that far. We had good instincts on how to design a calendar and no instincts about finding an actual customer, so we had nobody to ask the follow-up question. [Citation: https://open.spotify.com/episode/0W6i3PT0wY6o3YoObPmhKy] ##### [Shishir Mehrotra]() Ben's suggestion to set the advance bar on depth fits with Ivan's arrival test, and a disappointment score can measure both. Split survey respondents by how they arrived: people a champion brought in, and people who came through sales or a launch. Then ask each group how disappointed they would be if they lost the product. If only the champions say "devastated," you have one enthusiastic person per account and the rest is traction. If the people they brought in answer the same way, the fit has carried over to them. At Coda, the "devastated" answers came from customers who had already changed how their team works, and that is why our churn stayed very low even while NPS lagged. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] ##### [Reid Hoffman]() Shishir's "right versus familiar" split is the call we faced at LinkedIn, and I'd make it measurable. Early reactions ran from neutral to negative. Our bet was that we would build something valuable to the vast majority of professionals, and that people's views would convert once they saw it in action. [Citation: https://podcasts.apple.com/us/podcast/the-reid-hoffman-story-part-2-make-everyone-a-hero/id1227971746?i=1000430619820] That bet makes a prediction you can check. Track the skeptics as a cohort, meaning the people who signed up after a lukewarm invite or who told you they didn't see the point. If their usage rises after a few weeks of real exposure, the pushback was about unfamiliarity. If it stays flat while enthusiasts keep using the product, the enthusiasts are a niche, and you have traction inside that niche only. The bet also needs a stopping rule, because conviction without one is just stubbornness. Ben and Emmett have now agreed on a depth bar written down in advance. The skeptic cohort is a good place to set it: before you look at the data, decide how much of that cohort you expect to convert and by when. A topline bar would probably have told Pinterest and early Airbnb to quit. When I meet founders, I push on their vision to see whether they are learners as well as persistent. [Citation: https://open.spotify.com/episode/50cnOO5ujq2zX8eRRZGKZI] The ones who end up with real fit can tell you in advance which result would make them change their minds. ##### [Ivan Zhao]() I agree with Ben that low burn changes which signals you can afford to trust. Simon and I are both programmer-designers, and I could do marketing and brand, so we kept Notion very small and grew headcount slowly. Because we burned so little, we became profitable fast. Once you're profitable, you don't spend the next 18 to 24 months raising money, so you can focus on building. [Citation: https://youtube.com/watch?v=6soNEcjVSIc] [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] Low burn also keeps the read on fit clean. When we were a team of 25, about 99% of our growth came from word of mouth, and I said openly that we were at a baby stage on marketing and sales. [Citation: https://open.spotify.com/episode/15yZi5NMR5tYIHdAla2GpU] With almost no spend, nearly every new user came because someone told them about Notion, so we could read the growth as pull. A company that raises a big round early and buys distribution loses that clarity, because spend now produces some of the growth. So before running Emmett's threshold or Reid's stop-pushing test, check how much of your growth you paid for. If the answer is close to zero and people keep arriving, you are already running the stop-pushing test every day. #### [What to do when the search for product-market fit stalls]() **Question:** What should a founder do after a long search for product-market fit that hasn't worked? ##### [Emmett Shear]() By a failed search, I mean repeated launches that produce no sustained use, rather than a fixed number of months. At Kiko we launched a family social network that got 12 users, tried other ideas, and kept returning to the calendar for roughly eighteen months. A TechCrunch feature had produced a small bump and then nothing. We eventually sold Kiko after Google launched its calendar. More time spent on the same loop was not evidence that the loop worked. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] I would make the next decision a bounded experiment. Pick the strongest specific group of users you have found, set a usage target and a deadline that fit the cash you have, then commit to changing course if you miss it. At Justin.tv, we gave two competing directions small teams and three months; the gaming project aimed for roughly 20 to 25 percent monthly usage growth and achieved 30 percent a month over those three months. That result made the choice much easier. If there is no similarly promising group to test, or no runway to run a meaningful test, return remaining capital rather than fund another vague search. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] ##### [Ben Silbermann]() I think the first decision is whether a few people would genuinely miss the product. Pinterest had only about 3,000 registered accounts after its early launch, yet some users wrote when it went down because they depended on it to plan a wedding or redecorate a home. We spent time with those people and found others who had the same need. [Citation: https://podcasts.apple.com/ph/podcast/pinterest-ceo-ben-silbermann/id1205997729?i=1000422633011] [Citation: https://www.forbes.com/sites/tomiogeron/2012/10/22/ben-silbermann-on-how-pinterest-slowly-grew-to-massive-scale/] So I would speak to the people who return on their own, learn exactly what they use the product for, and try to reach more people with that need. If you find that group, concentrate the product and your outreach there. If you cannot find anyone who depends on it, change the product or the problem you are working on. Before Pinterest, our shopping app Tote was struggling; Pinterest gave us a product we used every day and could improve every day, so we shifted our attention to it. [Citation: https://podcasts.apple.com/ph/podcast/pinterest-ceo-ben-silbermann/id1205997729?i=1000422633011] ##### [Ivan Zhao]() I would separate the long-term goal from the product people have in front of them. We spent two or three years on an easier programming environment because we wanted more people to make their own tools. Most people we showed it to wanted to finish the work already on their desk. We kept the goal and changed the product to a productivity tool they could use immediately. [Citation: https://podcasts.apple.com/us/podcast/first-block-interview-with-ivan-zhao-and-simon-last/id1422389039?i=1000649918460] So I would ask three questions. Which people have a task they already do often? What have they tried in your product, and where do they stop? Can you make their next attempt useful before they have to adopt your broader vision? If the answers point to a different product, change the product and show it to those people. If repeated attempts still produce no use, decide how much time and money you can commit to one more specific approach. Our years of trying happened to lead somewhere; their length alone was never evidence that the next version would work. ##### [Emmett Shear]() There is a third option between finding rapid growth and closing the company: make the existing business pay for its own search. At Justin.tv, after raising equity and venture debt, we decided we could not count on another financing round. We shared burn and runway with the team, reviewed the P&L every month, and worked through ideas to earn revenue and cut costs until we reached profitability. The constraint made us measure whether each change actually brought in money. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] That option requires users who already get enough value to pay, or a clear route to earning revenue before the cash runs out. If the economics work, a founder can keep learning without asking investors to fund an indefinite search. ##### [Shishir Mehrotra]() For a team product, I would watch for use spreading beyond the first person who tried it. At Coda, we released an alpha to one six-person team about six months in. Their daily use fell from six to zero, and they gave us a list of roughly 30 changes. We took the feedback seriously, but built out the broader product rather than treating that list as the roadmap. We began to feel product-market fit three to three and a half years in, when use spread within teams and then into other teams. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] A useful test is to put the product in a team's real workflow and watch whether colleagues adopt it, and whether someone carries it into another team. If the first team quits, ask where the experience failed. Their requests may expose a gap, but following every request can leave the broader product unfinished. If repeated attempts yield no sustained use or spread, change the product or the customer you serve. ##### [Emmett Shear]() When cofounders disagree about the next direction, stop debating once each has made a clear case. At Justin.tv, Michael and I each preferred a different idea, and neither could convince the other. We gave each idea a small team and a three-month test with a target set in advance. The gaming project's measured usage growth gave us a reason to commit; the test replaced a debate neither of us could settle by argument. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] ##### [Reid Hoffman]() Financing can make a weak search look viable longer than it is. At SocialNet, we accepted a business model proposed by the investor willing to fund us, even though we had little sense of whether it would work. About two years in, I no longer believed in the model. I helped with the leadership transition and left. It was emotionally difficult, and the mistakes later shaped my thinking about LinkedIn. [Citation: https://open.spotify.com/episode/5SYTVI3GxYRBwVAl0SsrVq] Emmett's point about paying for the search sharpens the decision. Put the next plan in front of the team and board with its cost, the customer behavior that would justify continuing, and the date on which you will judge it. If current revenue can cover the search, you have more time to learn. If the plan depends on another raise, the customer evidence must justify spending other people's money on it. If you cannot make that case, arrange a responsible transition while there is still cash to protect employees and obligations. ##### [Ben Silbermann]() Emmett’s bounded test needs a product that gives people a fair chance to feel its value. Our earlier iPhone app had many features, yet none was special enough to make someone return. My advice from that experience is to ship when you have one thing you are proud of, something worth a person’s time. With Pinterest, we chose to make collections look good enough that people would want to share them. [Citation: https://www.startuparchive.org/p/ben-silbermann-explains-how-pinterest-found-product-market-fit] I would define that one experience before setting the test deadline, then watch what people actually do with it. A weak first experience can make a useful idea look like a failed one; a polished first experience followed by no repeat use gives you a clearer reason to change course. ##### [Ivan Zhao]() Emmett's point about paying for the search adds a useful constraint: a product can earn attention and still fail as a business. I would put the founder on support for a while. I handled Notion's support tickets in our early years, and each request told us both where someone got stuck and what they were trying to accomplish. We also used the product ourselves every day, which gave us another check on whether a change really helped. [Citation: https://podcasts.apple.com/ec/podcast/outliers-s05-e05-ivan-zhao-how-notion-works/id1190890158?i=1000479520195] If the same people keep returning with a real task, use their requests to narrow the next version and ask what they would pay for. If support only produces one-off requests across unrelated tasks, the founder has less reason to fund another broad build. The decision needs evidence of both continued use and a way to support the company. ##### [Shishir Mehrotra]() There is also a founder-level decision after the experiment: would you still choose this problem and this team if you were starting today? When people asked me about starting a company, I asked whether they had an idea they could not imagine leaving alone and a teammate they could not imagine working without. I started Coda because I could answer yes to both. [Citation: https://alejandrocremades.com/shishir-mehrota/] That conviction earns a founder the energy to run the next specific test. It does not replace evidence of use. If the product keeps failing and either answer has changed, I would take that seriously when choosing whether to pivot with the same team, join a different effort, or close. ##### [Emmett Shear]() I would keep investor skepticism separate from customer evidence. We talked to 40 VCs about Twitch, and they told us watching people play games was too small a niche. Its eventual size was genuinely uncertain. Meanwhile, people were using it, and the gaming project had grown 30 percent a month for three months during our test. A founder who has that kind of usage evidence should investigate whether the niche can expand; a founder with little repeat use needs to fix the experience or change the target user. [Citation: https://podcasts.apple.com/pt/podcast/twitch-emmett-shear/id1150510297?i=1000569313324] [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] ##### [Reid Hoffman]() Ben's point about a fair test matters especially for a network product. LinkedIn's launch was quiet, and a sparse network gave each new user limited value. We worked on People You May Know and address book uploads so people could find connections. [Citation: https://foundr.com/articles/building-a-business/reid-hoffman] Before treating a usage target as a verdict, ask whether the test group had enough relevant people or content to complete the task the product promises. If it did, low repeat use is strong evidence to change course. If it did not, run a bounded test in a group where the product can actually work, and count people who return after that initial experience. ##### [Shishir Mehrotra]() Emmett's point about a niche matters. At YouTube, we had strong use among younger audiences and still asked how to reach older viewers and new countries. Product-market fit changed as we considered each new audience. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] I would make the founder write down which decision the evidence supports. Repeat use in a small group supports serving that group well. Expansion into a larger market needs its own test with users in that market. An investor's guess about market size should prompt the second test, while the first group's actual use deserves its own weight. #### [Pressure-testing a startup idea before you commit]() **Question:** How should a founder pressure-test a startup idea before committing to it? ##### [Emmett Shear]() I would make the first test about whose behavior the product depends on. At Justin.tv, I initially reasoned from my experience as a viewer. But viewers followed streamers, so the streamer was the person I needed to understand. I called roughly 40 to 50 streamers and asked how they got started, why they kept streaming, and what they would change. Their answers surprised me: even a few dollars of income mattered to them. That led us to let streamers run an ad and share the revenue. Ask questions that can change your idea, rather than questions designed to confirm it. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] Then set a time limit and a behavior-based bar before the team gets attached. When we had three months to compare directions, we set a target of roughly 20% to 25% monthly usage growth for Justin.tv Gaming; it grew about 30% a month during the test. Later, gaming was growing 25% to 30% monthly for almost nine months before Twitch had its own domain. For a new idea, the exact threshold depends on the product, but I would specify in advance what repeat use or growth would justify committing more time and money. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] [Citation: https://podcasts.apple.com/tw/podcast/twitch-emmett-shear/id1150510297?i=1000569313324] ##### [Shishir Mehrotra]() I think a founder should run three tests before committing. First, name the specific change that makes the idea possible now. With Coda, my insight was that the line between documents and apps was artificial. I asked what had kept people from crossing it earlier, because a good idea needs an answer to that barrier. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] Second, test your own commitment and your ability to recruit. I ask aspiring founders whether they have an idea they cannot imagine leaving alone and a person they cannot imagine building it without. Alex and I made a list of people we wanted on the founding team and agreed to start only if all of them joined. They did. That gave us a stronger signal than encouragement from friends. [Citation: https://alejandrocremades.com/shishir-mehrota/] [Citation: https://podcasts.apple.com/us/podcast/episode-5-shishir-mehrotra-of-coda/id1539670094?i=1000502477962] Third, put an early version into someone else's working day and watch whether they keep using it. We moved our own work into Coda and brought in our first outside customer four months later. That team's eventual refusal to continue exposed specific product problems that our own use had missed. [Citation: https://podcasts.apple.com/us/podcast/episode-5-shishir-mehrotra-of-coda/id1539670094?i=1000502477962] ##### [Ben Silbermann]() I think the strongest early test is whether someone makes room for the product in a real task. With Pinterest, Evan and I wanted a place to collect things we found online, and we used the first version constantly. Early users later wrote to us when the site went down because they depended on it to plan a wedding or redecorate a home. Those messages told us more than a signup count could. [Citation: https://podcasts.apple.com/ph/podcast/pinterest-ceo-ben-silbermann/id1205997729?i=1000422633011] I’d pressure-test in three passes. Build one experience you’re proud to ask someone to spend time on. Put it in front of people with the underlying need, then watch whether they return and what they try to do with it. Finally, look for more people with that same need and see whether you can reach them. Pinterest had roughly 3,000 registered accounts after its first few months, yet a small group loved making collections; we spent time with those users and found more people like them. [Citation: https://www.startuparchive.org/p/ben-silbermann-explains-how-pinterest-found-product-market-fit] Set a time and savings limit for the test, since a meaningful signal can take longer than an initial launch. ##### [Emmett Shear]() Shishir's outside-customer test also needs enough time to tell you what failed. During Kiko, we'd spend about three weeks building another idea, launch it, and move on when it failed to take off immediately. We tried roughly six startups in eighteen months that way. The launches taught us how people reacted, but our assumption that a good product would grow instantly was wrong. [Citation: https://podcasts.apple.com/tw/podcast/twitch-emmett-shear/id1150510297?i=1000569313324] I would define the test as a sequence: get a specific user to try the product, learn why they return or leave, change the product, and see whether the behavior changes. One weak launch is evidence about that launch. Repeated failure to get the same users back after addressing their stated problems is much stronger evidence about the idea. ##### [Reid Hoffman]() Find the smartest critics you can and ask them to name the specific reasons the idea could fail. When I started LinkedIn, those conversations exposed the hard problems early. My reason for continuing was a set of beliefs I could defend: people would use real identities online, professional relationships deserved their own place, and invitations could help the network grow. Smart disagreement gave us a list of risks to work through; conviction required a reason to believe we could solve them. [Citation: https://open.spotify.com/episode/5SYTVI3GxYRBwVAl0SsrVq] A second question is whether the first use case can grow into the larger opportunity. For LinkedIn, job hunting and finding talent were the first applications of a broader professional network. I would ask who gets value on day one, how those people bring in the next group, and whether that activity makes the product more useful. If your plan depends on network effects, explain how the first group gets value before the network is large. [Citation: https://open.spotify.com/episode/5SYTVI3GxYRBwVAl0SsrVq] [Citation: https://podcasts.apple.com/us/podcast/product-led-ai-reid-hoffman-on-ai-powered-networks/id1089013200?i=1000654930736] ##### [Ivan Zhao]() People need to recognize the task your product helps them finish. My friends needed websites, and I kept building them by hand. That gave me a problem I could observe rather than a concept I had to explain. [Citation: https://open.spotify.com/episode/15yZi5NMR5tYIHdAla2GpU] We still chose the wrong first form for the product. We spent a couple of years trying to make it easy for everyone to create software, then showed it to friends and investors. The response was weak. Most people wanted to get their work done, so we built around the document and productivity tasks they already understood. [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] I would put a rough version in front of someone doing a real task and watch whether they come back to finish another one. Then I would check whether the product can reach those people and whether the value gives them a reason to pay. Those were separate problems we had to solve at Notion. [Citation: https://podcasts.apple.com/ec/podcast/outliers-s05-e05-ivan-zhao-how-notion-works/id1190890158?i=1000479520195] ##### [Shishir Mehrotra]() One caution on the early-customer test: a detailed complaint list can tell you where the first version hurts, while still giving you the wrong build order. Our first outside Coda team stopped using the product and handed us roughly 30 requested fixes. We took the feedback seriously, yet the team had only seen the table-based part of a product meant to combine documents and apps. We built the rest of that experience instead of treating their list as the roadmap. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] For a team product, I would set a second behavioral test after repeat use: does someone take it into another team without the founder doing the selling? We began to feel product-market fit when Coda spread that way, around three to three and a half years in. The time it took is specific to Coda; the useful test is whether the product moves through the organization on its own. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] ##### [Emmett Shear]() Reid's question about value before the network is large has a concrete test: recruit the first supply-side users one at a time and see whether they bring their audience. For Justin.tv Gaming, we improved the service around streamers' problems, then asked streamers to switch. Some larger streamers received a payment to do so. Each streamer told their followers where to watch, and those viewers discovered other channels. That cycle was already working before we launched Twitch as a separate site. [Citation: https://podcasts.apple.com/tw/podcast/twitch-emmett-shear/id1150510297?i=1000569313324] A founder with a two-sided idea should write down the first group's reason to join, the cost of recruiting them, and the next group's path in. Then test whether each new participant actually helps recruit or retain the next one. Otherwise the growth story is still an assumption. ##### [Ben Silbermann]() Emmett's point about a weak launch matters because reach is part of the test. When Pinterest had a small audience, we met with people who loved making collections and reached out to bloggers whose readers shared that interest. We also tried Pin It Forward, where people made a board and invited others to make one. That let us see whether the product had meaning beyond our own circle and whether its users could bring in more people. [Citation: https://www.forbes.com/sites/tomiogeron/2012/10/22/ben-silbermann-on-how-pinterest-slowly-grew-to-massive-scale/] [Citation: https://www.startuparchive.org/p/ben-silbermann-explains-how-pinterest-found-product-market-fit] I’d treat distribution as a separate question in the trial: can you find a group with the need, show them one useful thing, and see some of them introduce it to peers? If the first people care but you cannot find a repeatable way to reach others like them, you have learned where to focus the next test. ##### [Reid Hoffman]() A usage test leaves some of the biggest risks unanswered. When I evaluated Airbnb, the possibility of failure was obvious. I wanted a plan for each major risk, plus another route if the first plan failed; I also asked whether solving those risks would give the company time to build a marketplace before competitors followed. [Citation: https://open.spotify.com/episode/50cnOO5ujq2zX8eRRZGKZI] Before committing, write down the two or three ways the company could fail even if early users love the product. For each, specify what you can learn now, what resources the answer requires, and what you will do if the first approach fails. A difficult risk can make an idea more compelling when the team has a credible way through it. [Citation: https://open.spotify.com/episode/50cnOO5ujq2zX8eRRZGKZI] ##### [Ivan Zhao]() Shishir's point about the 30 fixes gets at a hard distinction: feedback should change your understanding of the person's task, while the founder still has to choose the product. I handled Notion's support tickets for the first couple of years and treated each request as feedback about the product and the business. We also used Notion for our own work every day, which let us check a request against how the product actually felt to use. [Citation: https://podcasts.apple.com/ec/podcast/outliers-s05-e05-ivan-zhao-how-notion-works/id1190890158?i=1000479520195] In a pressure test, I would sort the complaints by the work people were trying to finish, then change one part of the product and see whether the same people can finish that work. A feature request is a clue about a blocked task. The next version has to prove it solves the task. ##### [Ben Silbermann]() There’s also a limit to what a spare-time test can tell you about your own willingness to build the company. Before Pinterest, I kept starting side projects and finding reasons to stop. I eventually realized that my own commitment was the variable I could change, and leaving my job made me follow through. That was a personal lesson, rather than a rule that everyone should quit first. [Citation: https://www.forbes.com/sites/tomiogeron/2012/10/22/ben-silbermann-on-how-pinterest-slowly-grew-to-massive-scale/] I’d give the idea a bounded, serious period of work, with enough time to build something useful and meet the people using it. Then I’d ask whether I still want to work on the hard parts I’ve actually encountered, including finding users and responding when the product fails them. ##### [Shishir Mehrotra]() I would measure two different signals once people have tried the product: how much they would miss it, and whether they recommend it. In a conversation about product-market fit, I described Rahul Vohra's question, "How disappointed would you be if you could no longer use the product?" His suggested benchmark was 40% answering "very disappointed." I also compared it with net promoter score, which asks about recommending the product. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] Those questions can point in different directions. Early Coda users gave high disappointment scores and low recommendation scores; Superhuman showed the reverse pattern in our comparison. I would use the gap to choose the next investigation. When users depend on the product yet hesitate to recommend it, ask what makes sharing it difficult. When they recommend it yet could easily switch away, ask what job would make it part of their regular work. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] ##### [Ivan Zhao]() Reid's risk test should include the technical foundation when the idea depends on a new way of building software. At Notion, we bet on web components before the technology had settled. When the product broke, we struggled to tell whether the bug came from our code or the underlying libraries, and we had to rebuild on a different foundation. [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] Before committing to a product with that kind of dependency, I would build its hardest interaction first and run it through ordinary use with a small group. Record what breaks, how long it takes to fix, and whether the same experience works with a simpler foundation. Early demand tells you the work is worth doing; the technical test tells you whether this team can deliver it with the time and money available. ##### [Emmett Shear]() There is another test inside the streamer story: take an answer that sounds implausible to you and make it concrete. Streamers told me they wanted to earn money. I assumed a tiny amount would feel pointless; one streamer was excited by the prospect of making about $17 a month. We invited roughly 50 streamers into an early partner program where they could trigger an ad and receive half the revenue. They liked it even while earnings were small. [Citation: https://podcasts.apple.com/tw/podcast/twitch-emmett-shear/id1150510297?i=1000569313324] For a new idea, ask what people would actually do for the proposed value, then offer a small working version and watch them use it. The size of the action matters: a streamer choosing to run an ad tells you something a favorable interview answer cannot. ##### [Ben Silbermann]() The first people who need the product may surprise you. We expected interest from the tech crowd, yet Pinterest’s early users included people in the Midwest planning meals, homes, and events. Their everyday uses gave us a clearer picture of whom we were serving. [Citation: https://podcasts.apple.com/ph/podcast/pinterest-ceo-ben-silbermann/id1205997729?i=1000422633011] [Citation: https://www.youtube.com/watch?v=pIVVD6StbOs] I’d record who actually returns during a test, what they are trying to do, and whether that differs from the customer you imagined. That may change which group you test next. ##### [Reid Hoffman]() The test also has to produce evidence before the company runs out of time or capital. In PayPal's early plans, cash on mobile phones looked like a promising use for our encryption technology, but the team estimated it would take at least three years to build. We shifted to the Palm Pilot because we could get a version working in roughly six months and start learning. Later, eBay sellers adopted the email-payment feature in ways we had not planned, and we followed that demand. [Citation: https://youtube.com/watch?v=aFPngGEsB70] I would ask a founder to name the next proof point and its date: who must use the first version, what action would show value, and how much money is needed to see that action. If the first meaningful proof lies beyond the team's runway, change the first product or the financing plan before committing. [Citation: https://youtube.com/watch?v=aFPngGEsB70] ##### [Ivan Zhao]() I would also compare the trial product with the tool people already use for the same work. With Notion, the need for documents was established long before we started. Google Docs was free and good enough for many people, so the real question was whether our first version could do enough of that everyday work while offering a reason to switch. It took us years to reach a remotely comparable baseline. [Citation: https://open.spotify.com/episode/15yZi5NMR5tYIHdAla2GpU] For the pressure test, ask someone to complete a familiar task in both products. Watch where they switch back to the old one and where they choose yours again. That gives the team a more useful target than a general statement that the idea sounds interesting. ##### [Reid Hoffman]() A founder should also test whether the behavior that creates value can support a business. At PayPal, we first focused on getting people to use email payments, but we had promised the service would remain free and still needed a way to earn revenue. We eventually required sellers who accepted more than a limited amount of credit-card payments to switch to a merchant account. About 88% of payment volume converted within six weeks, without changing the growth curve. [Citation: https://youtube.com/watch?v=aFPngGEsB70] I would ask where a fee fits into the user's actual task, then run a small test that measures paid use and whether it slows adoption. A popular free product and a viable paid product are two separate findings. ##### [Ben Silbermann]() Reid’s date for the next proof point should include the time it takes to make a change after someone uses the product. Our shopping app had a slow cycle: building a change and getting it approved through the App Store could take weeks or months. With Pinterest, Evan and I could use it ourselves and improve it every day, which helped us learn while the audience was still small. [Citation: https://podcasts.apple.com/ph/podcast/pinterest-ceo-ben-silbermann/id1205997729?i=1000422633011] I’d choose a first version that lets the team respond quickly to observed use. If each change takes too long, the founder may spend the whole test period waiting to learn whether a fix helped. ##### [Shishir Mehrotra]() Ivan's technical-foundation test is a good example of choosing the risk that could invalidate the product promise. For Coda, our promise had two parts: a document had to feel familiar, and someone had to be able to build something as powerful as an app. I put product feasibility ahead of monetization because a pricing test would have told us little if we could not deliver both parts of that experience. [Citation: https://alejandrocremades.com/shishir-mehrota/] I would ask a founder to write one sentence describing what users must be able to do, then identify the assumption most likely to make that sentence false. Build the smallest real use case that tests that assumption. If it works, test the next risk, such as repeat use, how people find the product, or willingness to pay. The order depends on which unanswered question could stop the company first. ##### [Emmett Shear]() Ben's bounded period tests something the usage numbers cannot: whether you want the work that follows the first signal. I have said that founders can spend four or five years with early signs of success that fade, trying new things while the outcome stays unclear. Before committing, I would ask whether solving this particular user's problem still appeals to me under those conditions, including the long stretch of recruiting users, fixing failures, and trying again. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] ##### [Reid Hoffman]() The financing plan can change the idea you end up testing. At SocialNet, we needed venture funding to keep going. The only offer asked us to partner with newspapers, television stations, and magazines, so we accepted and changed the business model, even though we lacked a clear view of how that approach would work. I later stopped believing in the model we were pursuing. [Citation: https://open.spotify.com/episode/5SYTVI3GxYRBwVAl0SsrVq] Before taking funding tied to a new customer or sales path, I would test that path with the same care as the product. Talk to those customers, find out who makes the purchase, and work out whether the deal still lets the team pursue the reason it started the company. ##### [Ivan Zhao]() A useful distribution test for a flexible product is whether people create examples you never asked them to make. Notion's community grew in part because people could build their own setups, share templates, teach others, and even sell what they made. The people sharing had their own reasons to do it. [Citation: https://podcasts.apple.com/us/podcast/saastr-449-notion-ceo-ivan-zhao-on-winning-at-enterprise/id1089973241?i=1000519040320] For an early product that depends on a community, I would give a small group the basic tools and watch whether anyone makes something useful for another person, then whether that second person can adopt it. That test asks more than whether people like the founder's demo. It checks whether the product gives its first builders a reason to keep creating and sharing. #### [The warning signs that should end a startup idea]() **Question:** Which warning signs should make a founder drop an idea? ##### [Ben Silbermann]() I would look for a missing reason to come back. With our shopping app before Pinterest, we kept adding features and still couldn't name one thing the product did especially well. My test now is whether a founder can ship one experience worthy of someone's time, then find people who choose to use it again. If repeated conversations and improvements still leave you unable to find even a small group that cares about that experience, I would drop or change the idea. [Citation: https://www.startuparchive.org/p/ben-silbermann-explains-how-pinterest-found-product-market-fit] Pinterest gives an important check on that test. After roughly four months we had only about 3,000 accounts, yet a few people really loved making collections. We met those users and found more people like them. Weak signups alone would have sent us away from an idea with a real early audience. [Citation: https://www.forbes.com/sites/tomiogeron/2012/10/22/ben-silbermann-on-how-pinterest-slowly-grew-to-massive-scale/] ##### [Shishir Mehrotra]() I think a founder should reconsider an idea when its central assumption fails a hard test. Ask what everyone else assumes about the market, what you believe instead, and what has kept others from building it. If the difference amounts to a missing font or line-spacing option, there may be a feature to build, but little reason for a new company. For Coda, our bigger claim was that the boundary between documents and apps was artificial; we believed web-based tools had removed a file-format barrier that held earlier products back. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] Then watch behavior. Our first customer's daily users went from six to zero, and the team said they would quit if their CEO made them keep using Coda. That was a severe warning about the product. They also gave us a specific list of changes and still believed in the goal, so we kept working. Later, a customer asked to expand Coda to another team. For a team product, repeated abandonment after you address specific problems, with nobody asking to bring colleagues in, would make me question whether the idea solves a problem people care enough to change their work for. [Citation: https://podcasts.apple.com/us/podcast/shishir-mehrotra-ceo-of-coda-former-vp-of-youtube/id1705957755?i=1000636890453] ##### [Ben Silbermann]() Shishir's example makes the customer conversation more useful than a bare usage count. I would ask the people who stopped using the product what they hoped to do and where the experience broke, then ask the few who stayed what they actually value. At Pinterest, the small group who loved making collections helped us see that reaching and connecting with similar people was a problem we could work on. We held meetups and asked bloggers to introduce the service to their readers. [Citation: https://www.forbes.com/sites/tomiogeron/2012/10/22/ben-silbermann-on-how-pinterest-slowly-grew-to-massive-scale/] If the people who stay cannot describe a specific use worth sharing with someone else, another round of outreach has little to build on. ##### [Ivan Zhao]() Our first version of Notion exposed a warning sign: the people we hoped to reach had a different job to do. We spent more than two years making it easier for people to build their own software. Friends and investors tried it, and the idea failed to resonate. Most people had a report due or a task to finish; they wanted a tool they could use right away. We changed the product into familiar productivity software while keeping the underlying ability to make your own tools. [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] A second warning sign is a technical foundation that prevents you from fixing what people need. We had chosen web components, found the foundation too unstable to debug reliably, and rebuilt. I separate three decisions: stop selling the current product, replace the implementation, or give up on the underlying need. Our evidence called for the first two. [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] ##### [Emmett Shear]() Kiko taught me to watch for a team that responds to weak usage by changing products while staying vague about whom it serves. Kiko got a TechCrunch feature, grew a little, and then stopped. We launched a family network that got 12 users and quit that experiment. The deeper failure was that we described Kiko’s customer as “everyone who uses a calendar.” Each new project improved our execution, yet the work never accumulated around one customer. I’d stop that cycle and choose a specific customer before building the next product. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] [Citation: https://open.spotify.com/episode/0PeWI8MHSdfoLgfFxCQc5g] I’d treat investor rejection separately. Forty VCs turned down Twitch because gaming seemed too small; we had evidence of people actually using it, so their market-size judgment was a poor kill signal. [Citation: https://podcasts.apple.com/pt/podcast/twitch-emmett-shear/id1150510297?i=1000569313324] ##### [Shishir Mehrotra]() Ben's point about asking the people who stopped is right, and I would ask which part of the intended product they actually tried. Our first Coda customer gave us about 30 specific requests after the entire six-person team stopped using it. We took the feedback seriously, yet we did not build that list first. They had mainly seen our tables; the document, forms, and layouts that were central to the product were still missing. We kept building those pieces, and it took another two or three years to work through their list. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] A useful kill signal is repeated rejection by customers who have actually experienced the full claim behind the idea. Feedback on an incomplete slice can tell you what to fix, but it can give you the wrong answer about whether the idea itself works. ##### [Reid Hoffman]() Ben's small group of committed Pinterest users is the right guard against quitting on low signups alone. Track whether each serious iteration improves your odds of success. When the next version of the plan has worse odds than the first, given what you have learned about customers and the market, I would consider a major pivot. [Citation: https://open.spotify.com/episode/50cnOO5ujq2zX8eRRZGKZI] SocialNet gave me a different warning sign: financing can keep an idea alive after the founder has lost confidence in its business model. We accepted the only funding offer we had and changed our model to serve media partners, even though we had little sense of how that would work. Roughly two years in, I told the board I no longer believed in that model and helped with the transition. If a funding condition forces an untested business model that you cannot make credible through customer evidence, reconsider the idea before the new capital locks you into it. [Citation: https://open.spotify.com/episode/5SYTVI3GxYRBwVAl0SsrVq] ##### [Ivan Zhao]() Another kill test comes after people start using the product: can the value support a business? Early at Notion, I learned that a product people like still needs a way to reach people, a reason for them to pay, and unit economics that let the team keep building. I would ask a founder to name who pays and why, then check whether the cost of delivering the product leaves room to keep serving them. If repeated attempts to find a workable answer fail, the current business idea needs to change even if the product has devoted users. [Citation: https://podcasts.apple.com/ec/podcast/outliers-s05-e05-ivan-zhao-how-notion-works/id1190890158?i=1000479520195] ##### [Emmett Shear]() Shishir’s distinction about testing the full claim matters. When we had three months of runway to choose between gaming on Justin.tv and another project, we set a bar for gaming of roughly 20 to 25% monthly usage growth. Gaming grew 30% a month for all three months. A founder can set a time-bound usage bar before the test, then decide against the current direction if it misses. The number should fit the business; the useful part is making a decision rule while the team still has room to act. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] ##### [Shishir Mehrotra]() A survey can obscure the warning sign if you treat one score as a verdict. At Coda, we saw high scores when we asked how disappointed people would be if they lost the product, alongside low scores when we asked whether they would recommend it. In the same interview I described Superhuman seeing nearly the reverse pattern. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] I would ask both questions, then investigate the gap. People who would miss a product but hesitate to recommend it may be pointing to friction in using or sharing it. If the people doing the target job would barely miss it, that is stronger evidence against the underlying idea. ##### [Ben Silbermann]() Ivan's point about the intended user matches a mistake I nearly made with Pinterest. Friends in tech were polite but didn't use it. The people who kept making boards were using them for ordinary plans, like a home or a meal, so we went to the bloggers those people already followed and invited their readers in. [Citation: https://www.youtube.com/watch?v=LzhXIBPYyxw] Before treating silence as a kill signal, I would check whether the people who actually have the problem can find the product and use it. If they can, and the one thing you built still gives them no reason to return, I would change the idea. #### [Where founders find a problem worth a company]() **Question:** How do founders find a problem worth building a company around? ##### [Ben Silbermann]() I think you start with a problem you care enough about to work on every day, then look for people whose behavior tells you they care too. Evan Sharp and I wanted a place to collect things we found online, return to them, and see what other people collected. We built Pinterest around something we wanted to use ourselves. When early users wrote because the site was down and they depended on it to plan a wedding or redecorate a home, their use gave us a reason to commit to it. [Citation: https://podcasts.apple.com/ph/podcast/pinterest-ceo-ben-silbermann/id1205997729?i=1000422633011] The first audience can be small. After months of sharing Pinterest, we had around 3,000 registered accounts, yet a few people really loved it. We found more people like them and learned what they were trying to do. A founder can test an idea the same way: make one useful thing, watch who returns to it, and ask what would happen if it went away. [Citation: https://www.startuparchive.org/p/ben-silbermann-explains-how-pinterest-found-product-market-fit] ##### [Ivan Zhao]() I found the starting point by making three or four websites for artist friends who needed portfolios. They could describe what they wanted, but making it in software required someone who could code. That gap stayed with me: people should be able to shape the tools they use for their own work. [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] The harder test came after we started building. We spent roughly two years on an easy developer tool and showed it to friends and investors. It did not resonate. People had reports due and work to finish, so we moved toward productivity software they already used every day. For a founder, I think the problem has to hold your interest long enough to keep working on it, while the first product has to meet a task people already care about. Our original idea survived because we changed the way people encountered it. [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] ##### [Emmett Shear]() Find people doing something intensely, then ask what they need to keep doing it. At Justin.tv, I had been reasoning from the viewer's perspective. But viewers followed streamers, and I didn't understand streamers. So I called roughly 40 or 50 of them and asked how they got started, why they kept streaming, and what they would change if they ran Justin.tv. One answer surprised me: even earning $2 a month mattered to them. Earning *anything* from streaming made the activity feel real. That led us to build ways for streamers to make money. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] I used to take an idea to customers and ask questions that would validate it. That can tell you whether your existing idea survives contact with users; it gives you little chance to discover a better one. Ask about their behavior and motives first. In our case, the important problem was what streamers needed in order to keep creating, and those interviews changed what we built. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] ##### [Ben Silbermann]() Ivan's example points to a useful distinction between a lasting interest and the first product built around it. Our shopping app before Pinterest tried to do too many things, so people had little reason to choose it for any one task. With Pinterest, we focused on making a collection look good enough that someone would want to make one and show it to a friend. I would test a possible company problem with one experience that earns someone's time; feedback on a rushed, unfocused version can tell you very little about the underlying need. [Citation: https://www.startuparchive.org/p/ben-silbermann-explains-how-pinterest-found-product-market-fit] ##### [Reid Hoffman]() I look for a change I understand deeply enough to see a useful service before it seems obvious to everyone else. With LinkedIn, I believed people would use their real identities online and would need a professional identity distinct from their social one. Job hunting and recruiting gave us a specific first use case; the larger opportunity was helping people find professional relationships and information. I took the idea to smart critics, asked what would break, and used their objections to identify what we had to solve. [Citation: https://open.spotify.com/episode/5SYTVI3GxYRBwVAl0SsrVq] A founder should also ask why they can make progress where others see too much risk. Customer enthusiasm can show demand, but it also means competitors can see the same opening. A surprising idea needs a clear reason you know something others have missed, then an early use case that lets you learn whether you are right. [Citation: https://open.spotify.com/episode/50cnOO5ujq2zX8eRRZGKZI] ##### [Ben Silbermann]() An early sign that the problem reached beyond our own taste was who used Pinterest. Some of our first committed users were people from the Midwest and York, Pennsylvania, who were planning meals, homes, and weddings. They came back because it helped them with something in everyday life, and friends learned about it when they asked where an idea came from. When you see someone outside your own circle use a product for a task they already have, you learn more about its reach than you do from a room of founders saying the idea sounds promising. [Citation: https://podcasts.apple.com/ph/podcast/pinterest-ceo-ben-silbermann/id1205997729?i=1000422633011] ##### [Emmett Shear]() Ben's point about a small first audience is right, but the decision to commit still needs a bar. At Justin.tv we gave a small team three months to make the gaming category better and set a usage growth target of roughly 20% to 25% per month. That turned a disagreement about which direction felt promising into a test we could actually run. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] Gaming grew roughly 25% to 30% a month for almost nine months before we launched Twitch as its own site. We had repeated use and growth in a specific group before we committed to the separate brand. [Citation: https://podcasts.apple.com/tw/podcast/twitch-emmett-shear/id1150510297?i=1000569313324] ##### [Ivan Zhao]() Ben's point about a focused first experience fits a second test: can the product carry your own taste while earning repeat use and revenue? In our early years, we leaned so hard toward the tool we wanted to see in the world that friends and investors could not see why they would use it. I think founders have to watch both sides. Too much of your own vision can become an art project; following demand alone can leave you building a commodity. [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] I would keep asking what part of the idea you would still care about after the first version fails, then use people's actual work to decide what to build next. For us, the aim of helping people make their own tools persisted through several versions; productivity work gave us a concrete place to begin. ##### [Ben Silbermann]() Emmett's growth target gave his team a clear decision point. With Pinterest, I also wanted a way to hear the reason behind the numbers, so I put my phone number in outbound emails and early users called when something failed. A founder deciding whether to keep building should stay close enough to users to hear the exact task they were trying to finish, then choose a measure that captures whether the product helps them finish it. [Citation: https://podcasts.apple.com/ph/podcast/pinterest-ceo-ben-silbermann/id1205997729?i=1000422633011] ##### [Emmett Shear]() Reid's question about why *you* can make progress mattered in gaming. Live video usually creates hard rights problems: content owners have reasons to restrict distribution. With game streams, I saw a different incentive. Watching skilled players could lead viewers to buy a game or play it more, so game publishers had reason to welcome exposure. That made gaming a much better category for us to build around than a generic live-video service. [Citation: https://podcasts.apple.com/tw/podcast/twitch-emmett-shear/id1150510297?i=1000569313324] ##### [Shishir Mehrotra]() At YouTube, I became interested in how teams built their own ways of working with documents and spreadsheets, even when those tools made the process clumsy. I kept asking whether a simple, ambitious idea could solve that recurring problem. That observation led Alex and me to Coda's thesis: anyone should be able to make a doc as powerful as an app. We had to build the product to show what that meant, because the pitch alone left many people wondering what was wrong with the tools they already used. [Citation: https://podcasts.apple.com/us/podcast/why-your-company-needs-new-rituals-w-shishir-mehrotra/id1227971746?i=1000488470233] [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] For the decision to start a company, I use two questions: Can you imagine working on this idea for years, and is there a teammate you deeply want to build it with? I had both with Coda. If you can articulate the problem but cannot convince even one person to work on it with you, keep testing the idea before making the company commitment. [Citation: https://alejandrocremades.com/shishir-mehrota/] ##### [Reid Hoffman]() Ben's point about committed users outside your circle also raises a distribution question: how will the first users make the product useful for the next ones? LinkedIn began with a professional profile that others could search and use to reach you. A member could add value to the service simply by creating a profile, even before they became a frequent visitor. That helped us address the early network problem while we built toward stronger network effects. [Citation: https://open.spotify.com/episode/50cnOO5ujq2zX8eRRZGKZI] For a founder choosing a problem that depends on a network, I would ask who gets useful results with the first hundred people and what brings the next hundred in. That answer changes which product you build first. #### [The first moves for a first-time founder]() **Question:** What are the first steps for a first-time founder? ##### [Shishir Mehrotra]() Before you write any code, treat starting a company as an investment decision. It's an extreme one, because an investor spreads bets across many companies and you can only invest in one. When I look at a company, I ask three questions, and I'd ask the same three about your own idea [Citation: https://youtube.com/watch?v=WLvuvLEkjiw]: 1️⃣ Is this a team that doesn't know how not to win? Whatever it runs into, it can't picture losing. 2️⃣ Is it working in a space that matters? 3️⃣ Does it have a secret that's at least good enough to get started? You don't need the whole answer on day one, only enough of an insight to earn the next step. My second concrete step is a document I ask every would-be founder to write, titled "Things I'd like to see different in the world." Once you have it, go talk to the people already working in your space, competitors included. First-time founders often tell me they can't do that because they'd be a competitor. I think that's backwards. If you can't talk to the people who know the space best, your odds in it are poor. Those conversations might talk you into joining them. They might convince you the space isn't worth it, or show you they've got it backwards and you should compete [Citation: https://youtube.com/watch?v=KOjsOdYHuFk]. I put this much weight on diligence because starting a company is about the most extreme one-way door decision there is. Joining a company is closer to a two-way door, since you can leave. Coda began with my co-founder Alex DeNeui asking me to help him brainstorm a pivot for his startup. One of us wrote "what if you could make docs as powerful as apps" on the wall, and I couldn't stop thinking about it. That obsession, plus 20 years of working with Alex, is what took me through the door [Citation: https://youtube.com/watch?v=iA9YT0g9Dg8]. ##### [Ben Silbermann]() The first step is to start. I spent years at Google talking about wanting to do a startup, and my girlfriend (now my wife) finally told me to either do it or stop talking about it. That push mattered more than any plan I had. [Citation: https://www.youtube.com/watch?v=-mM62APV7R8] So once you've started, here is what I'd do, in order: 1. Build something you personally want. Pinterest came from an idea Evan and I shared: a place to collect the cool things we found online and browse other people's collections. I collected insects as a kid, so it felt natural to me. [Citation: https://podcasts.apple.com/ph/podcast/pinterest-ceo-ben-silbermann/id1205997729?i=1000422633011] 2. Ship when you have one thing you're proud of. Our first product was an iPhone shopping app called Tote. We crammed in too many features, it was hard to ship, and nothing about it was special. My rule since then: ship when one thing is worth a stranger's time, because if it isn't, you get no useful feedback. [Citation: https://www.youtube.com/watch?v=68GqJRLbWHo] [Citation: https://www.forbes.com/sites/tomiogeron/2012/10/22/ben-silbermann-on-how-pinterest-slowly-grew-to-massive-scale/] 3. Serve the small group who love it. Three months after launch, Pinterest had about 3,000 registered accounts. We built around those few fanatical users, and they described what Pinterest was better than we could. [Citation: https://www.forbes.com/sites/tomiogeron/2012/10/22/ben-silbermann-on-how-pinterest-slowly-grew-to-massive-scale/] 4. Weigh advice, then decide for yourself. Many VCs passed on us because we focused on images and had two non-technical cofounders. Their concerns deserved a look, but the future is unwritten, and every investor has a list of companies they passed on that went on to succeed. [Citation: https://www.youtube.com/watch?v=vjDBcvzux2c] My goal at the start was simple: I wanted to see a stranger using something I made and finding it useful. That is a good first milestone for any founder. ##### [Reid Hoffman]() Start by writing down your theory of the game, and then go find smart people who think you're wrong. Being contrarian means having a theory of the market that differs from the skilled competitors around you. Customer research doesn't count, because other founders can find the same research. So step one is a short written thesis. Step two is pressure-testing it with smart skeptics who will tell you exactly why it fails. Contrarian and right is the target. Interest from investors doesn't confirm your conviction. Next, pick your cofounder with the seriousness you'd bring to choosing a life partner, because you'll go through the valley of the shadow together. That person doesn't have to be someone you've known since high school. I go to my network and ask, "Who are the best people I can meet?" Then I meet great people without a fixed job description in mind. [Citation: https://podcasts.apple.com/us/podcast/reid-hoffman-kickstarters-everette-taylor-take-your/id1227971746?i=1000723855783] Third, treat your plan as an experiment. A common mistake is building an elaborate plan for a product without testing one assumption with one customer. Get something imperfect in front of users early. Mark Zuckerberg's test is useful here: "Is this going to destroy the company? If not, let them test it." [Citation: https://medium.com/p/d6ead87d99a9] Fourth, go where the networks for your kind of company already exist. In my Stanford lecture I used the example of a fashion company. It might be a great idea, but moving to Silicon Valley to build it makes little sense, because the supporting networks for fashion are somewhere else. [Citation: https://youtube.com/watch?v=dQ7ZvO5DpIw] Finally, accept the trade-off. Starting a company is jumping off a cliff and assembling the plane on the way down. Founders don't get balance in the early years, so make the jump knowing the cost. ##### Question > Shishir's first step is diligence before writing any code. He has the founder write a document called "Things I'd like to see different in the world" and then talk to people already working in the space, competitors included. From your public work at YC and on Justin.tv, would you tell a first-time founder to do that first, or to build and launch something rough first? What specific step comes first in your order, and what public source backs it? ##### [Emmett Shear]() Build and put something rough in front of users first, and treat those users as your diligence. When Justin, Kyle and I started, the only thing we knew starting a company meant was "make a product and put it on the Internet." We applied to YC with Kiko, the calendar we were already building, and the early YC interview was heavy on "show us a demo." [Citation: https://podcasts.apple.com/tw/podcast/twitch-emmett-shear/id1150510297?i=1000569313324] What YC was really testing was whether you'd thought about how you'd get customers and make money, and how you responded when challenged, because everyone knows the answer is going to change. [Citation: https://podcasts.apple.com/tw/podcast/twitch-emmett-shear/id1150510297?i=1000569313324] The reason I put building first: the hard part of a startup is figuring out *which* problem to solve. Solving it is the easy part. I expected the opposite coming out of a CS degree. [Citation: https://open.spotify.com/episode/6GN1lnptRlE3tisi2lY9Hg] A document of what you'd like to see changed in the world is a fine starting point, but real users answer the "which problem" question better than competitors do. I'd also be careful with the strong thesis Reid describes as a first step. On Twitch I held off on forming hypotheses, because once you have one you hunt for evidence that confirms or disconfirms it and miss the bigger picture. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] So my order is: 1- Find a cofounder. It's one of the few pieces of startup advice I think holds broadly. [Citation: https://open.spotify.com/episode/6GN1lnptRlE3tisi2lY9Hg] 2- Ship something rough. 3- Talk to users to *learn*, and let them change the idea. Founders who only validate end up with worse ideas. [Citation: https://open.spotify.com/episode/6GN1lnptRlE3tisi2lY9Hg] A gamer told us he'd love to make $17 a month streaming, so we built a button that ran a mid-roll ad and split the money 50/50. We invited about 50 streamers, and they loved it. [Citation: https://podcasts.apple.com/tw/podcast/twitch-emmett-shear/id1150510297?i=1000569313324] 4- Before committing, ask yourself honestly whether four to five years of mild success that often feels like failure sounds tolerable. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] ##### Question > Reid, you put a short written thesis first, then pressure-testing it with smart skeptics. Emmett says that on Twitch he deliberately held off on forming hypotheses, because once you have one you hunt for confirming evidence and miss the bigger picture. For a first-time founder with no idea yet, which should come first: writing the thesis or open-ended customer conversations? How do you keep a written thesis from turning those conversations into validation exercises? Please cite a public source. ##### [Reid Hoffman]() If you have no idea yet, start with the open-ended conversations. Write down a purpose before you write a thesis. Sara Blakely is my favorite example. While she was selling fax machines door to door, she wrote in her journal, "I want to invent a product that I can sell to millions of people that will make them feel good." That sentence had no market theory in it. It gave her a question to keep asking, "Is this my big idea?", until cutting the feet off a pair of pantyhose answered it. Big ideas rarely arrive as a lightning strike. Founders who find them were already on the hunt. [Citation: https://medium.com/p/9bb2b3b7b491] The purpose statement is close to Shishir's "things I'd like to see different" document. It points your attention somewhere without deciding the answer for you. The thesis comes once an idea shows up, and that's where Emmett's warning applies. Three habits keep it from turning user conversations into validation: 1. Write it as a falsifiable hypothesis, in the lean startup way, so you know in advance what evidence would kill it. [Citation: https://podcasts.apple.com/us/podcast/reid-hoffman-challenges-your-thinking-on-ai-governance/id1744818044?i=1000665461820] 2. Spend real time with smart people who are convinced you're wrong. Agreement is cheap. Even investor interest doesn't confirm your conviction. 3. Test with customers in the same weeks you write. One of the most common mistakes is building an elaborate plan without testing one assumption with one customer. So my order for a first-time founder is purpose first, then conversations and rough experiments, then a written thesis that you keep trying to break. Emmett's $17-a-month streamer is the kind of signal a loosely held, written thesis helps you recognize, because you notice when a user contradicts what you wrote down. ##### Question > Ivan, the group splits on how finished a first product should be. Emmett says ship something rough and let users change the idea. Ben says ship only when one thing is worth a stranger's time, because his feature-crammed first app, Tote, got no useful feedback. Early Notion was rebuilt from scratch before it found its footing. What would you tell a first-time founder to do first, and how do they know when their product is ready to show strangers? Please cite public interviews or writing. ##### [Ivan Zhao]() I'd side with Ben on the bar and with Emmett on the speed. The first Notion was too rough in the wrong place. We spent two to three years on what was roughly "Webflow meets Figma," an easier programming environment, and the craft was fine. What failed was the premise: most people don't want to program their own tools. They want something off the shelf that solves their problem quickly. No amount of polish would have fixed that, and shipping it sooner would have told us sooner. [Citation: https://podcasts.apple.com/us/podcast/first-block-interview-with-ivan-zhao-and-simon-last/id1422389039?i=1000649918460] So the first step is to check whether strangers have the problem you're solving, before you put serious craft into the solution. Notion itself started because my artist friends needed websites and I got tired of building them by hand. That was a real problem. My wrong assumption was that they wanted to become toolmakers like me. [Citation: https://open.spotify.com/episode/15yZi5NMR5tYIHdAla2GpU] How do you know it's ready? On a founding team you need someone to argue that question. I'm the obsessive one. I spent literally months on our marketing site, and two days on a single tweet storm about the history of relational databases. Simon was the one saying "it's good enough, let's get it out." Those arguments were healthy. If you are a solo founder with my temperament, find someone who will play Simon for you. [Citation: https://podcasts.apple.com/us/podcast/first-block-interview-with-ivan-zhao-and-simon-last/id1422389039?i=1000649918460] After you ship, expect the signal to come slowly. Product-market fit never arrived for us as one moment. People started caring, then paying, then investors started reaching out (one mailed dog treats to an office address we had never published). Until then, the useful measure is whether the product is closer to what's in your head than it was last week. [Citation: https://youtube.com/watch?v=IIPKMixTMfE] Keep the team small while you search. A low headcount kept Notion profitable, so we didn't have to raise every 18 to 24 months and could keep building. [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] #### [Sizing a market you haven't entered yet]() **Question:** How do you research and size a market before you build? ##### [Ivan Zhao]() So, I never built a bottoms-up TAM spreadsheet for Notion. I sized it with two ratios and one historical question. The first ratio came from our 2021 funding post. The world runs on software used by 1B+ knowledge workers, but only about 25M developers can modify it. Our whole product mission is closing that gap, so the gap told us how big the market could be. The second number was the entry use case. Notes and docs are something 1B+ people do every day. We picked that wedge because it gave us the biggest possible top of funnel, and our internal strategy, "B2C2B," depended on it. Individuals use Notion for simple note taking, find the relational database underneath, and then bring it to work. Half our B2B customers came in that way. [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] The historical question was whether the market is bundling or unbundling right now. Microsoft bundled software in the 90s, and SaaS unbundled it until the average company was running close to 100 tools. Reading where you are in that cycle tells you whether to build vertical or horizontal, and those two choices produce very different businesses. [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] Now the honest trade-off. A vertical company gets a market boundary for free: you sell to one or two personas, and it's clear who buys. A horizontal company has no boundary, because Notion goes after the entire software market, so you have to draw that boundary yourself through positioning and go-to-market. I stayed in the "Lego brick" mindset for too long. I wish we had started selling the "solution Lego box" to businesses and moved upmarket earlier. [Citation: https://youtube.com/watch?v=IIPKMixTMfE] I also wouldn't recommend the horizontal path to most founders. The problem space is too large for a playbook to exist, and horizontal products usually take years to find fit. Ours did too. ##### [Ben Silbermann]() Ivan had a number to start from, 1B+ people taking notes. Pinterest didn't have a clean one, and a market-sizing exercise in 2009 would probably have talked us out of it. Pinterest went against what was popular in Silicon Valley at the time. Facebook, Twitter and FriendFeed were real-time text streams, and we were images that weren't real-time. Many VCs passed on us for that reason. [Citation: https://www.forbes.com/sites/tomiogeron/2012/10/22/ben-silbermann-on-how-pinterest-slowly-grew-to-massive-scale/] So I'd research a human behavior before I'd research a market. I've been a collector since I was a kid: stamps, baseball cards, and insects I caught growing up in Iowa. I think most people collect something in some part of their life, and that habit was the real size of the opportunity. [Citation: https://m.youtube.com/watch?v=xiPOS4_4jV4] The research that mattered came after we shipped. When we launched, I emailed all my friends and nobody really got it. A small group did love it, and they weren't typical early adopters. They were people I grew up with, using it to plan their homes and pick recipes. So we asked where those people gathered. The answer was a blogger conference, and we had each blogger introduce Pinterest to their own readers. [Citation: https://www.youtube.com/watch?v=oZTGwdGRZCc] Three months after launch we had about 3,000 accounts, and we built around that small group of fanatical users. [Citation: https://www.forbes.com/sites/tomiogeron/2012/10/22/ben-silbermann-on-how-pinterest-slowly-grew-to-massive-scale/] One more step comes before any of this. Ship once you have one thing you're proud of. Our first attempt, an iPhone shopping app called Tote, crammed in too many features and wasn't great at any of them. If a product isn't worth someone's time, the feedback you get only tells you what you already knew. [Citation: https://www.startuparchive.org/p/ben-silbermann-explains-how-pinterest-found-product-market-fit] The limit of my approach is that it was slow. It worked for a consumer product with almost no capital needs. If your build costs a lot up front, you'll need a harder number earlier. ##### [Shishir Mehrotra]() Ivan's ratios tell you how big the prize could be. Before I trusted a number like that, I wanted to know why nobody had won it yet. So I start by writing down one simple insight: the assumption everyone else in the market has made that we see differently. Then I put that insight through two tests. First, is it actually true? Second, what has kept anyone from acting on it already? [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] At Coda, the insight was that companies draw an artificial line between docs and apps. The sizing came from that line. The average company buys something like 265 different software applications, and a large share of them exist because a doc couldn't do the job. [Citation: https://open.spotify.com/episode/2wt9r0JskUo7ngjUIMBKzS] The second test, about the barrier, is where I spent most of my research time. I went back through the history of documents and found that Apple was arguably the best innovator of the late 1990s and early 2000s. Numbers, Pages and Keynote were all different from Word, Excel and PowerPoint, and none of them took off. File formats were the reason. If you sent someone a Keynote file, they needed a Mac to open it, so format compatibility became real lock-in. Even Google Docs had to import and export Office files with full fidelity. Web-based tools removed that barrier, and that change is what made the timing work for us. The feature-or-product question filters out a lot of ideas. When I described Coda, people kept suggesting things like "Office with different line spacing." If Microsoft could ship an idea in a release, it's probably only a matter of time before they do, and you can't build a company on it. Once something is live, I watch two signals. I ask users what Coda replaced, and the answers varied a lot, which told us the market was a bundle of smaller ones. That fits Ivan's bundling cycle from the customer's side. [Citation: https://open.spotify.com/episode/7zj5hXzYs6rDAzeARFW2J4] Early on I also trust Rahul Vohra's disappointment score more than NPS, because a differentiated product in a young market shows up there first. ##### [Emmett Shear]() Ivan's ratios work for setting a ceiling on a horizontal product. They don't tell you whether anyone wants your version yet. By the time a TAM figure feels convincing, someone else has usually seen it too, so what I measure is whether a real slice of the market is already pulling on you, and then I study that slice's customers until I understand what they actually want. Twitch is the clearest example. Justin.tv gaming had been growing 25 to 30% a month for almost nine months before we launched Twitch as its own site, so it was fully validated before we bought the new domain [Citation: https://podcasts.apple.com/pt/podcast/twitch-emmett-shear/id1150510297?i=1000569313324]. We set that bet up deliberately. Instead of arguing about which pivot was bigger, we funded gaming and mobile as small side projects with growth goals, and agreed to pivot to whichever one hit them [Citation: https://podcasts.apple.com/pl/podcast/ep-69-emmett-shear-co-founder-twitch-behind-the/id1606770839?i=1000617294292]. In that setup, observed growth stands in for market size. The research itself was interviews with streamers. I asked what tools they used, what they liked, and what they wished someone would build for them. Their feature ideas didn't matter much. The follow-up did: "I built you the big red button. Why is your life better now?" That question gets you to the real need [Citation: https://open.spotify.com/episode/02alpCNse78IHzRj54CvLJ]. The first round boiled down to three things streamers wanted: money, fame, and love. Once we met those needs and had a way to acquire streamers, growth took off almost immediately [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H]. I also held off on forming hypotheses during those interviews. Once you have a theory, you start hunting for evidence that confirms or disconfirms it, and you miss the bigger picture [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H]. The failure case is Kiko. We built a calendar app even though we didn't really use calendars beyond the one in Gmail. So when Google launched its own calendar, we had no idea what to do next [Citation: https://podcasts.apple.com/us/podcast/emmett-shear-co-founder-of-twitch/id1677066062?i=1000635708653]. If you aren't the customer, you have to go talk to the customer before you build. ##### [Ben Silbermann]() Shishir's "what did it replace" question has a cousin I'd add to the list: what are people using it for that you didn't design for? On early Pinterest, a lot of people made boards of things that made them happy or relaxed them, mostly inspirational quotes. We didn't know what to make of it and assumed it was a quirk of a small early user base. It held up all the way to a quarter billion users, and it's still one of the most common ways people use Pinterest. [Citation: https://www.youtube.com/watch?v=2LT_sDkHYRc] That pattern told me something no market report would have. People want tools that help them feel good about themselves, and they'll put in the work to build those spaces. Technology companies rarely talk about emotions, so no category existed to size it against. So the evidence was the behavior itself, repeated at scale by people we'd never met. So my sizing test is about persistence. When an unexpected use shows up in your first few thousand users, watch whether it survives as you grow into people who aren't like your early adopters. If it does, you've found a need that's larger than the product you thought you were building. ##### [Ivan Zhao]() Emmett is right that my ratios say nothing about whether anyone wants your version, and Notion paid for learning that. I agree with Ben that the behavior is the real research. Notion's first version took our mission literally: everybody should be able to create software, so we built a developer tool simple enough for anyone. We spent a couple of years on it, gave it to friends and investors, and it didn't resonate. Most people wake up with a report due. They want to get the job done, and they don't care about building software to optimize it. [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] So we hid the vision inside a form people already used every day, which was productivity software. That changed how I think about market research. When people asked how we knew Notion was wanted, my answer was that its previous versions had existed for decades. Word and Office. Lotus 1-2-3 in the 80s. Before that, typewriters and file cabinets. [Citation: https://open.spotify.com/episode/15yZi5NMR5tYIHdAla2GpU] If people have done something for 50 years on paper and then on three generations of software, the demand is already proven. Your research question becomes narrower: what does the current version get wrong? Ben's point about fit holds for this approach too. Finding the right market didn't save us on its own. We had the right shape, productivity software with the building blocks underneath, but we were on the wrong technical foundation and running out of money. We laid off all five people, leaving Simon and me, and rebuilt it in Kyoto. [Citation: https://youtube.com/watch?v=IIPKMixTMfE] My concrete method is to find a behavior with a long history, check that people already pay for an older tool that serves it, and then put your more ambitious idea inside that familiar form. ##### [Reid Hoffman]() Emmett's warning about forming hypotheses too early is right for customer interviews, but I'd still write the market thesis down before building. When a market doesn't exist yet, it usually ends up either huge or non-existent. A thesis written in advance is how you tell which one you're facing before you've spent two years finding out [Citation: https://youtube.com/watch?v=pkAum45ubWc]. The thesis has two halves. Shishir covered the first, the thing you believe that others don't. For LinkedIn, mine was that recruiting through classified ads belonged to the newspaper era. In a networked age, you recruit people directly through their professional networks. That one was relatively easy to validate [Citation: https://youtube.com/watch?v=pkAum45ubWc]. The second half gets skipped more often. Write down what would lead you to conclude there's no market. Then put the thesis up against the classic entrepreneurial misfire, the assumption that once the product exists, people will realize they love it and line up in droves. A few founders, Steve Jobs among them, could make that happen. Most of the time it doesn't work, because people adopt slowly when they don't know they want the category. So your thesis has to explain how you'll get fast adoption anyway [Citation: https://youtube.com/watch?v=pkAum45ubWc]. Ben's story with the bloggers and Emmett's 25 to 30% monthly growth on Justin.tv gaming are both examples of that adoption path, observed in practice. Each one gives you a concrete signal you can name in advance. You keep the thesis and the listening separate: you commit up front to what would prove you wrong, and you go into the interviews open. How much you validate before building depends on the domain. In software, speed of learning matters most, so you ship early and let usage do the research [Citation: https://open.spotify.com/episode/0YOnTcWDsIEwLjpZd4AUbO]. In hardware, shipping the wrong thing can kill you, so the thesis needs much harder evidence before you commit capital [Citation: https://youtube.com/watch?v=pkAum45ubWc]. ##### [Shishir Mehrotra]() I partly disagree with Emmett on studying the slice that's already pulling on you. That works when the slice is using the whole product. At Coda, our first real users were a single team led by Noam. One day they stopped using it, all of them, which was 100% of our user base. They handed us a list of about 30 things to fix and said they'd come back when we were ready. Every idea on the list was good, and it felt like someone had given us a roadmap. We chose not to build it. Coda was mostly tables then, with no document, forms or layouts yet, so Noam's team only saw a small part of what we were building. If we had overfit to that list, we would have missed what the product was meant to become. It took us two to three years to get through their list, and we did it while building everything else. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] So for an all-in-one product, the early customer signal tells you about the piece they can see. It won't tell you how big the whole market is. That's why we defined product-market fit as spread. For a team product, fit means someone on one team carries Coda into a different team. We didn't see that until our first beta, about three and a half years in. Disappointment score and NPS can also disagree, which says something about the market itself. Coda had high disappointment scores and low NPS early on, and Superhuman had the reverse. Superhuman users loved it and recommended it, but Gmail was right there if it went away. Coda users had no substitute, though they weren't yet confident recommending it. When disappointment is high and NPS is low, you've found a gap nobody else fills. High NPS with low disappointment means you're competing in a market that already has alternatives. Emmett's Kiko story passes the same test I described earlier. A calendar was something Google could ship as a feature, and it did. ##### [Emmett Shear]() Ben's unexpected-use test is how Justin.tv found its real market. We started out producing a live reality show about Justin's life, and we were bad at it. That turned out not to matter. As a customer, we weren't that different from everyone else who wanted to broadcast live, and lots of people wanted to stream live on the internet [Citation: https://open.spotify.com/episode/0PeWI8MHSdfoLgfFxCQc5g]. The show was the product we designed. The thing people actually used was the streaming tool underneath it. The second lesson took longer. In a two-sided market, you have to work out which side the market really is. When I started watching gaming streams, I saw that my loyalty was to the streamer and not to Justin.tv. If my favorite streamer moved to YouTube, I would have followed them. I also didn't want to be a streamer, so my own intuition as a user was pointing at the wrong customer [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H]. That changes how you count. Viewers look like the big number, but streamers decide where viewers go. Every streamer we recruited brought their audience with them by telling followers to come watch on Twitch. We paid some of the larger streamers $1,000 to switch, and we used their complaints as our feature request list. Shishir, this is where I'd answer your Noam story. We often didn't build the thing a streamer suggested. We went and solved the problem they were pointing at [Citation: https://podcasts.apple.com/pt/podcast/twitch-emmett-shear/id1150510297?i=1000569313324]. Studying the slice that's pulling on you means learning what they need, and their roadmap is only evidence of it. So the market worth researching was a few thousand gaming streamers and what they needed, and the viewer numbers followed from that. If you're sizing a marketplace or a creator platform, start by finding the side that the other side is loyal to. Research that side first. Reid, I agree with keeping the written thesis separate from the listening. Our side-project setup was exactly that: we wrote the growth goals for gaming and mobile down in advance, and missing them would have killed the project [Citation: https://podcasts.apple.com/pl/podcast/ep-69-emmett-shear-co-founder-twitch-behind-the/id1606770839?i=1000617294292]. The goals were the kill criterion. I kept theories out of the streamer interviews themselves, because those were where the money, fame, and love answer came from. ##### [Ben Silbermann]() Ivan's test of checking that people already pay for an older tool works well for productivity software. It breaks down for some of the needs I've cared most about, because people meet them without paying anyone. Collecting is a good example. For most of history it lived in shoeboxes, binders, and cases of beetles on a bedroom shelf, and you couldn't put a revenue line on any of them. [Citation: https://m.youtube.com/watch?v=xiPOS4_4jV4] So when there's no paid predecessor, I look for experts who have studied the behavior for years and build with them. How We Feel started in 2020 as a nonprofit app I built with scientists at MIT and the Harvard School of Public Health. You entered your ZIP code and how you were feeling, and the researchers used that data. [Citation: https://open.spotify.com/episode/2mDoEoUeLtctdf2xx5onGS] We went to the scientists first because they already knew which questions mattered and what was still unknown. In that same conversation, I agreed with Jeff Lawson's view that you work backwards from what people are experiencing and the problems they need to solve, and everything else follows. [Citation: https://open.spotify.com/episode/2mDoEoUeLtctdf2xx5onGS] For a free or nonprofit product, the "market" you're sizing is a count of people with a real need. You measure it in how often they come back, because they'll never show up as dollars. ##### [Ivan Zhao]() Ben, fair correction. The part of my test that carries weight is the long history of the behavior. A paid predecessor makes the signal stronger for a business product, and How We Feel shows you can do without one. Reid's question of how adoption actually happens was half of our mission statement from early on. In our 2021 funding post I described Notion's mission as two parts: software toolmaking and ubiquity. Missing either half meant missing the whole. A tool for building software only becomes useful once lots of people around you already use it, so we planned for adoption as deliberately as for the product. Concretely, we gave $1,000 Notion credits to startup incubators to get the startup community using it. Later we made Notion completely free for students and educators. Companies are the ones who pay, so charging students who can barely cover tuition made no sense. Our goal was for Notion to show up everywhere, the way everyone uses Microsoft Office for something. [Citation: https://youtube.com/watch?v=G0n6L_j5xJ4] So my answer to Reid's test looks like this. Write down where the paying customer is (businesses). Then write down who reaches that customer first for free (individuals, students, early-stage startups). Then check whether the free use case is something people do every day, like taking notes. If it is, individuals will bring the tool to work themselves, which is the B2C2B path I mentioned earlier. B2C2B is Shishir's spread test one level down: one person carries the tool into their team. If the free use case is occasional, you won't get the fast adoption Reid is asking about, and you'll have to sell top down from day one. #### [Standing out when every product looks the same]() **Question:** How does a startup win in a market full of similar products? ##### [Ben Silbermann]() I think you win by choosing one experience people care enough about to return to and show someone else. Our earlier shopping app had lots of features, but no single thing we were proud to put in front of people. With Pinterest, Evan and I cared about making a collection look good enough that someone would want to share it. [Citation: https://www.startuparchive.org/p/ben-silbermann-explains-how-pinterest-found-product-market-fit] The first response was small: about 3,000 registered accounts after three months. We spent time with the people who loved it, held meetups, and reached out to bloggers who could use it for their own collections. [Citation: https://www.forbes.com/sites/tomiogeron/2012/10/22/ben-silbermann-on-how-pinterest-slowly-grew-to-massive-scale/] My test for a crowded market would be specific: can you name the thing your first users come back to do, and can they show you where it helped in their lives? Early Pinterest users told us they depended on it to plan a wedding or redecorate a home. [Citation: https://podcasts.apple.com/ph/podcast/pinterest-ceo-ben-silbermann/id1205997729?i=1000422633011] ##### [Emmett Shear]() Start by finding whose choice determines where everyone else goes. At Justin.tv Gaming, I realized viewers followed streamers. If a streamer moved to YouTube, I would follow them there. So our customer work had to start with streamers, whose needs I did not understand well enough. I called roughly 40 or 50 of them and asked how they got into streaming, what they liked about it, and what they would change. Their answer surprised me: even a tiny amount of income from streaming mattered to them. We built a button that let streamers run an ad and split the revenue with us 50/50. Viewers sometimes reminded streamers to use it because they wanted them to earn money. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] My framework is to identify the participant who brings the others, talk to enough of those people to learn something you could not have guessed, then build around that need. Give the idea a growth test. When we gave Justin.tv Gaming three months and a target of roughly 20% to 25% monthly usage growth, it grew about 30% a month. That gave us a concrete reason to keep investing in it. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] ##### [Reid Hoffman]() When products look similar, I ask where value will compound as people join and use yours. LinkedIn entered a market with other professional-network efforts already underway. The hard question was how to make the network useful early enough for members to stay; I estimated it would take roughly a million registered members before the first member had a fairly functional value proposition. [Citation: https://podcasts.apple.com/us/podcast/product-led-ai-reid-hoffman-on-ai-powered-networks/id1089013200?i=1000654930736] My framework is three questions. Which specific group gets value first? What gets better for that group as each new customer joins or participates? And how will you reach enough of them before early users leave? In a marketplace, sellers leave when buyers are absent, and buyers leave when sellers are absent. A founder needs a concrete plan for building density on both sides. The growth method also has to fit the current market: the email invitations we used to grow LinkedIn would be a poor default for starting that network today. [Citation: https://podcasts.apple.com/us/podcast/product-led-ai-reid-hoffman-on-ai-powered-networks/id1089013200?i=1000654930736] ##### [Ben Silbermann]() Emmett’s question about whose choice brings others applies to Pinterest in a slightly different way. People often found us after a friend cooked a meal, threw a party, or redecorated a room and someone asked where the idea came from. The person sharing that result mattered more than a generic invite prompt. [Citation: https://podcasts.apple.com/ph/podcast/pinterest-ceo-ben-silbermann/id1205997729?i=1000422633011] So for a product with lots of lookalikes, I’d watch what happens outside the app. Which user gets a question from a friend because the product helped them make something worth showing? That tells you whom to serve first and which part of the experience has to work beautifully. ##### [Ivan Zhao]() So I would start with the job a person already needs to finish, then build a distinct way for them to finish it. We spent our early years making a tool for people to create their own software. People wanted to finish their reports and organize their work. We found a way in through notes and documents, then let people use the same flexible parts for tasks, databases, and team work. [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] For a startup in a crowded market, my framework is: choose a frequent job, make the first useful result easy to reach, and give people a reason to stay as their needs grow. The first job must work on its own. For business customers, show the finished workflow they can use immediately and the specific cost or risk it addresses. We learned that selling flexible parts alone asked customers to do too much setup. [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] ##### [Emmett Shear]() Reid's density question has a very practical version for a creator platform: why would the first creator move before the audience is there? We recruited game streamers one by one, sometimes offering larger streamers $1,000 to switch. When a streamer said the service lacked a feature, we used the request to find the underlying problem and solve it, even if the eventual fix differed from their suggestion. That is how you make the first side of a network willing to show up while the network is still small. [Citation: https://podcasts.apple.com/pt/podcast/twitch-emmett-shear/id1150510297?i=1000569313324] ##### [Reid Hoffman]() Ben's example points to a useful test: identify the behavior competitors have misread. When I considered investing in Airbnb, a highly experienced consumer investor told me it could be the deal I failed on. That increased my conviction because, if our view was right, other capable investors and competitors might dismiss the same opportunity while the team built it. [Citation: https://podcasts.apple.com/us/podcast/product-led-ai-reid-hoffman-on-ai-powered-networks/id1089013200?i=1000654930736] A founder should be able to state the disputed belief plainly and show evidence from early users. For Pinterest, the friend asking where a finished project came from is evidence of a behavior with room to grow. The next test is whether serving those users creates repeat use and brings in more people through their work. ##### [Shishir Mehrotra]() At Coda, the first use was deliberately ordinary: write a doc or run a meeting. The advantage emerged as a team shaped that doc around its own work. Tables and buttons let people turn familiar documents into shared processes. At Pinterest, Ben's team began with meeting notes. Later, someone built their goal-tracking system in Coda, so a product manager could pull team goals into a project doc and update the shared system there. [Citation: https://p3adaptive.com/rawdatapodcast/episode/shishir-mehrotra/] For a crowded market, I would track the second use after the first successful task. If a team uses the product for meeting notes, do they choose to build their next recurring process there? That choice tells you whether the product can grow with the customer's work. ##### [Ben Silbermann]() Ivan’s point about a finished job matters for positioning, too. When Pinterest started, real-time text feeds were popular, while we built around visual collections people could return to as they planned a wedding or a room. [Citation: https://www.forbes.com/sites/tomiogeron/2012/10/22/ben-silbermann-on-how-pinterest-slowly-grew-to-massive-scale/] [Citation: https://podcasts.apple.com/ph/podcast/pinterest-ceo-ben-silbermann/id1205997729?i=1000422633011] A founder can ask early users what they were trying to finish, then make that purpose obvious in the first session. A feature list makes similar products sound even more similar; a collection that helps someone finish a real plan gives them a reason to return. #### [What an AI startup can build that a model company won't easily copy]() **Question:** Where can a new AI startup build something the big model companies are unlikely to copy? ##### [Ben Silbermann]() I think a promising place is a product where people keep contributing their own taste and intent. On Pinterest, people saved ideas for a home, a meal, or a wedding, and those choices made future recommendations more useful. The work was understanding each person's interests and helping them act on them, including finding products that fit their taste. [Citation: https://www.youtube.com/watch?v=Cc5ZFWrOYQA] A small AI company could start with one group, say people planning a garden, and help them save ideas, compare what fits their space, and record what they actually planted. Each season gives the product more useful feedback about that person's preferences and local conditions. A model company can build a garden chat feature; earning a group of people who return to plan and share their results takes sustained product work. Pinterest had just 3,000 registered accounts three months after launch, and we focused on a small group of deeply engaged users. [Citation: https://www.forbes.com/sites/tomiogeron/2012/10/22/ben-silbermann-on-how-pinterest-slowly-grew-to-massive-scale/] ##### [Ivan Zhao]() I would look for a narrow team workflow where the startup can own the setup and the daily work: for example, an agency's client intake, approvals, and handoffs. Build the records, permissions, and review steps around how that team actually works, then use models to find the right context and draft the next action. A model company can ship a similar assistant, but matching each team's process and earning its place in that process is a different kind of work. We learned at Notion that giving people flexible building blocks alone was too much setup. People wanted to finish a task, so we packaged the blocks into ready-to-use templates and use cases. [Citation: https://open.spotify.com/episode/2iQ9h7QtscRLqlLX1xJ9VM] For an AI startup, I would start with one complete workflow that a team can use on day one, then let the team change it over time. The setup and ongoing use are where the product learns what the team needs. ##### [Emmett Shear]() I would build around a group whose work you understand better than the model companies do, then make the product better as that group uses it. Kiko was our calendar for Gmail. Google shipped essentially the same calendar attached to Gmail, and our opening disappeared. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] Twitch developed differently. We talked to game streamers, solved the problems behind their feature requests, and recruited them one by one. Each streamer brought an audience; viewers discovered other streamers; that made recruiting the next streamer easier. [Citation: https://podcasts.apple.com/pt/podcast/twitch-emmett-shear/id1150510297?i=1000569313324] For an AI startup, I'd look for a similarly specific group: say, a tool that helps a particular kind of creator produce work *and* helps that creator reach and retain an audience. A model company can ship the generation feature. The repeated work with creators and the audience relationships take a different kind of effort. ##### [Reid Hoffman]() I would build around a transaction that requires trust between people, with AI making the transaction easier. Take a marketplace for specialized medical referrals: the model can help match a patient to a clinician, but the startup would still need clinician participation, verified availability, patient consent, and a way to learn whether the referral worked. Each completed referral can make the service more useful to the next patient and clinician. That is a harder product to copy than the matching prompt alone. I would test the claimed advantage carefully. In a public conversation about AI networks, I argued that a hospital data deal alone is a weak moat because other hospital systems exist and synthetic data may narrow the gap. I also distinguished strong marketplace network effects, where participants have a real reason to gather in one place, from communications networks where people can easily use several services. [Citation: https://open.spotify.com/episode/3OOLPpLmPaTPonDfBFoB2x] For the referral startup, the hard question is whether clinicians actually keep their availability current and patients get a better referral as participation grows. If that behavior never appears, the model company can copy much of the feature. ##### [Ivan Zhao]() Emmett's Kiko example is a good warning for the agency workflow I mentioned. A ready-made intake assistant is easy to copy. I would build a way for the people who set up that workflow to keep improving it across clients: reusable intake forms, approval rules, and review practices that they can adapt for each team. At Notion, consultants began selling templates and helping companies set up workspaces; some made a living doing that work. We also worked on connecting people inside larger companies who were introducing Notion and needed to share best practices. [Citation: https://podcasts.apple.com/us/podcast/saastr-449-notion-ceo-ivan-zhao-on-winning-at-enterprise/id1089973241?i=1000519040320] A startup serving agencies could give operations consultants a product they can install and maintain for clients. That creates a practical route into teams and a reason the workflow improves after the initial AI feature ships. ##### [Ben Silbermann]() Ivan's agency example raises another useful test: who takes responsibility when the answer affects someone's life? With How We Feel, the team includes scientists, designers, engineers, and therapists, and the product helps people understand and manage their emotions. [Citation: https://www.linkedin.com/feed/update/urn:li:activity:7009184042298335232/] We describe its guidance as science-backed strategies for hard times. [Citation: https://www.linkedin.com/feed/update/urn:li:activity:7421296016752066561/] An AI startup could pick one well-defined emotional wellbeing task, such as helping someone name an emotion and choose an appropriate exercise. The product would need expert review of the exercises, careful boundaries for sensitive situations, and a way to learn whether people found the guidance useful. At Pinterest, we held back health content until we could draw on credible institutions because the information people see can affect their lives. [Citation: https://open.spotify.com/episode/6iIrIn893TJMnlGuudYWiE] That level of care is ongoing work a general assistant may have little reason to do for a narrow group. ##### [Shishir Mehrotra]() Imagine a sales rep writing a customer memo while an assistant pulls in the account update, product roadmap, and last support issue. I would focus on workflows like this, where the customer’s context lives across several applications. That requires trusted connections to the systems the team already uses, permission-aware context, and a place in the rep’s daily workflow. I described this as the opportunity behind combining Coda’s integrations with Grammarly’s presence across applications. [Citation: https://sloanreview.mit.edu/article/building-an-ai-platform-for-the-future/] [Citation: https://www.grammarly.com/blog/company/meet-shishir/] For a startup, customer renewal briefs are one concrete wedge. Connect the account record, support history, and roadmap, then put the brief where the account team already prepares for a renewal. For the renewal brief, the test is whether the startup can keep trusted access across account, support, and planning systems and earn regular use from the account team. ##### [Emmett Shear]() The creator example needs a stricter test than whether people like the AI output. In my first streamer interviews, I heard three needs: money, fame, and love. We gave streamers an ad button that paid them, discovery by game and transcoding so more people could find and watch them, and chat moderation that made the audience relationship workable. We also recruited streamers directly. Growth took off when those pieces worked together. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] For an AI product for creators, I would measure whether a creator comes back because the product helps them earn, get discovered, and have useful exchanges with their audience. Better generation alone leaves the startup exposed. If the audience follows each creator into the product and starts finding other creators there, you have a reason to believe the startup is building something a model company would need to operate, rather than simply add as a feature. #### [Putting an early startup idea in front of real customers]() **Question:** How do you validate a startup idea with real customers before building much? ##### [Emmett Shear]() When I took over the gaming part of Justin.tv (which became Twitch), I realized I wasn't the customer. I was a viewer, and viewers were loyal to the streamer. The streamer was the important one, and I didn't know what streamers wanted. So I went and interviewed them. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] The questions were simple. What have you tried to use for streaming? What did you like about it? How did you get started? What's your biggest dream for streaming? What do you wish someone would build for you? I asked that last one for the follow-up. A streamer would say "build me this big red button," and I'd say: great, I built it, why is your life better now? Then they'd give the real answer. They'd make money that month, or get new fans, or their YouTube audience could watch them live. They wanted the money and the fans, and the button was only their guess at how to get them. [Citation: https://open.spotify.com/episode/02alpCNse78IHzRj54CvLJ] The common mistake is treating customers as your product designers. Customers are bad at product design, and if they were good at it they'd be starting the company. They *are* the experts on their own problems and the realities of their business. You need to understand those problems deeply enough that your intuition can design the product. [Citation: https://podcasts.apple.com/pl/podcast/ep-69-emmett-shear-co-founder-twitch-behind-the/id1606770839?i=1000617294292] So I'd push back a little on the word "validate." Founders who go out to confirm an idea mostly get confirmation. The ones who go out to learn come back with an idea that's better than the one they started with. Ask yourself whether the conversations have actually changed what you plan to build. [Citation: https://open.spotify.com/episode/6GN1lnptRlE3tisi2lY9Hg] ##### [Ben Silbermann]() I think the honest test is putting something small but real in front of people and watching who comes back on their own. You learn far less from asking people whether they like the idea. So before Pinterest, we built Tote, an iPhone app that was supposed to put all the catalogs from your mailbox on your phone. We crammed in every feature we could think of, and none of them was special. That taught me to ship when you have one thing you're proud of, one thing worthy of someone's time. If it isn't worth their time, the feedback you get is just "this is crap," and you learn nothing about whether the idea was good. [Citation: https://www.youtube.com/watch?v=68GqJRLbWHo] When we released Pinterest, I emailed all my friends. They were polite ("looks interesting") and they mostly didn't get it. Four months in we had something like 200 users. But a small group really used it, for regular parts of their lives: what their house might look like, what food they wanted to cook. Many of them were people I grew up with in Iowa. That small group was the real signal. So we asked where people like them gathered, went to a blogger conference in Utah, and had those bloggers introduce Pinterest to their audiences when we had a few hundred users. [Citation: https://www.youtube.com/watch?v=oZTGwdGRZCc] [Citation: https://www.youtube.com/watch?v=-mM62APV7R8] Two things helped us read those early users well. I put my cell phone number on the customer support emails and took the calls myself, so I heard directly when something broke or delighted someone. [Citation: https://www.youtube.com/watch?v=LzhXIBPYyxw] And years later I still visited Pinners in their homes, because sitting with someone shows you what's going on in their life that never comes through in metrics or interviews. [Citation: https://open.spotify.com/episode/2mDoEoUeLtctdf2xx5onGS] Expect your early adopters to look different from the tech crowd you'd predict. Ours were mostly people outside Silicon Valley. ##### [Ivan Zhao]() I'd answer this with a warning about my own story first. Notion did not validate cheaply. The first three or four years were what I call the lost years. Our first version was a developer tool so easy that anyone could build their own software. We worked on it for a couple of years and learned that most people just don't care about that. [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] So the lesson I took: test whether people care about the form your idea comes in, which can be different from the vision itself. Our vision of everyone creating their own tools stayed the same. What changed was the form. We put it inside something people already use every day, productivity software. We called it hiding the broccoli inside the sugar. The second lesson is to look for a need that has existed for a long time. Documents, spreadsheets, file cabinets, typewriters: people have always needed information tools to think and solve problems. That meant demand was never our question. The risky question was whether we could build a better version technically, and Google Docs is free and good enough, so the bar was high. [Citation: https://open.spotify.com/episode/15yZi5NMR5tYIHdAla2GpU] Once a few people are using it, stay very close to them. I answered every support ticket myself for the first couple of years. I read each one as feedback on the product and the business, but I kept my own view on what to listen to and what to ignore. We also ran the whole company on Notion, so we were our own harshest critics. [Citation: https://podcasts.apple.com/ec/podcast/outliers-s05-e05-ivan-zhao-how-notion-works/id1190890158?i=1000479520195] ##### [Reid Hoffman]() Start by writing your idea down as an investment thesis: a short list of beliefs about who has the problem, why now, and what they'll actually do once you give them something. Then treat each belief as a hypothesis to test with real people. That's classic lean startup practice, and it's how I still frame it: be persistent about the vision, and stay paranoid about whether the evidence is tracking against the thesis. [Citation: https://youtube.com/watch?v=dQ7ZvO5DpIw] Next, put something real in front of customers early. My line is that if you're not embarrassed by the first version of your product, you launched too late. [Citation: https://open.spotify.com/episode/0YOnTcWDsIEwLjpZd4AUbO] The point is speed of learning. Emmett's interviews tell you what problem to aim at, and a rough product tells you whether you hit it. What people say they want is a weaker signal than what they do. What they do with a rough product, whether they come back, invite others, or pay, is the signal that counts. Watch behavior rather than stated interest. At Socialnet, we built one of the best matching engines of its time and got good reviews, but we focused too much on profiles and too little on whether users actually engaged with each other. Admiration from users isn't validation. Repeated use is. When you're early, stay close enough to talk to almost every user. When I was looking at Airbnb for its Series A, its volume was so small that the founders could have phoned everyone who booked that week. [Citation: https://youtube.com/watch?v=eTkFItOG3Kk] At that scale, every conversation with a customer is a data point you can act on the next day. Expect to meet skeptics. Partners at Greylock argued Airbnb would be a train wreck: strangers in your house, cities banning it. Contrarian ideas will look wrong at first. Your validation work is how you find out whether yours is contrarian and right, or just wrong. ##### [Emmett Shear]() Ben's Pinterest story works because the people coming back were specific people he could call: friends from Iowa, then bloggers he met in person in Utah. That's the condition I'd watch for before trusting any validation signal. You either build for yourself as the customer, or you build for a concrete other person you can actually talk to. I've almost never seen a product succeed when it was built for an imagined third party. Ivan running Notion on Notion is the first mode, and Reid's Airbnb founders who could phone every guest that week are the second. [Citation: https://open.spotify.com/episode/0PeWI8MHSdfoLgfFxCQc5g] Justin.tv is my example of losing that condition. For the first six months we ran a live show, so we *were* the streamer. If we needed a feature, we built it, and we didn't need to talk to anyone. Then we turned the show off and kept going on intuition. We had learned that we could invent products intuitively and they'd work, which is true only while you're the customer. After the show ended we were guessing what people might want, with no direct line to them. Twitch only got going once I went and interviewed streamers. [Citation: https://open.spotify.com/episode/0PeWI8MHSdfoLgfFxCQc5g] So, a cheap check before building much: name the five people this is for, and either be one of them or have their phone numbers. The big exception is a product every human uses, like search or online shopping, because there you're always the customer by definition. Most startups build something much narrower than that. ##### Question > Ivan, Notion spent a couple of years learning that most people didn't care about building their own software. Emmett would have found that out by interviewing users about their problems, and Ben would have shipped one small thing and watched who came back on their own. Looking back, which of those two methods would have shown you sooner that the developer-tool form was wrong, and what would that first test have looked like? ##### [Ivan Zhao]() Emmett's method, I think, though only because of where our mistake was. A small shipped version would have told us people weren't coming back. It wouldn't have told us why, and we would probably have blamed the execution and kept polishing, which is my tendency anyway. Simon used to push me to ship while I spent months on the marketing site. [Citation: https://podcasts.apple.com/us/podcast/first-block-interview-with-ivan-zhao-and-simon-last/id1422389039?i=1000649918460] Our mistake was asking people to want our vision directly. The vision, that everyone should be able to make their own tools, came from building websites for my artist friends after college. They were talented, and they didn't have tools that let them create with software. [Citation: https://open.spotify.com/episode/15yZi5NMR5tYIHdAla2GpU] My friends did have that problem, but "make your own software" was my answer to it. They never asked for it. So, looking back, the first test would have been to sit down with those same friends and a few small teams and ask Emmett's kind of questions. What do you use every day for your work? What do you keep fighting with? I'm fairly sure the answers would have been documents, notes, wikis and task lists, the tools people have used for decades. That is where we ended up two years later, when we hid the vision inside productivity software, the broccoli inside the sugar. [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] Ben's method becomes the right one after that point. Once you know the form, you need a version good enough on one thing to be compared with Google Docs, and then you watch who comes back. ##### [Ben Silbermann]() Emmett, one wrinkle on that: we did start Pinterest as something we'd use ourselves, but the people who kept it alive weren't us, and we only found them by watching usage. Who you ask can trick you as badly as the word "validate" does. Pinterest never really caught on in California for the first year and a half. The people around me in Silicon Valley were the stereotypical early adopters, and most of them shrugged. If I had treated their reaction as the verdict, we would have stopped. [Citation: https://www.youtube.com/watch?v=-mM62APV7R8] So the useful question for me was: who is already using this without being asked, and what are they using it for? For us that was people I grew up with in the Midwest and people in creative fields like interior design, planning real things in their lives. Those users weren't where any playbook said early adopters would be. I think that's even more true now, because almost everyone has a powerful phone and knows how to make an account and post something, so early adopters can come from anywhere. [Citation: https://www.youtube.com/watch?v=1JLc2PYyCa0] Reid, I'd draw one line differently on the embarrassing first version. The product can be rough everywhere else, but the one thing it does should be good enough to be worth a stranger's time. Tote failed that test, and the feedback taught us nothing. When the core is good, a lukewarm response tells you something real, often that you showed it to the wrong people. [Citation: https://www.youtube.com/watch?v=68GqJRLbWHo] ##### [Shishir Mehrotra]() Ivan, a shipped version can tell you why people stopped, as long as the customer is close enough to call you. That happened to us. Emmett's point that customers are bad product designers gets hard when a customer you love hands you a detailed roadmap. Coda ran in stealth for years, but we onboarded our first customer team long before launch, and their daily usage was the signal I trusted most. That first alpha taught me the hardest lesson. One day the team's usage dropped to zero and stayed there. When I called Noam, who ran the team, he said the bad news was that his team would quit if he made them keep using the product. The good news was that they believed in the vision, and they handed us a list of about 30 specific things they needed before they'd come back. Losing 100% of your user base in a day is brutal, but that list was real validation: they cared enough to tell us exactly what was missing. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] We deliberately did not build those 30 things right away. That team had only seen the tables part of Coda, with no documents, forms or layouts yet. If we had overfit to their list, we would have missed what the product was really meant to be. It took us two or three years to work through it. So listen hard to early customers, but judge their feedback against the full vision before you build it. On measurement, I like Rahul Vohra's disappointment score: ask users "How disappointed would you be if you could no longer use this?" and aim for 40% saying "very disappointed." Coda had high disappointment scores from the start but low NPS early on, because switching required real change management. [Citation: https://open.spotify.com/episode/2wt9r0JskUo7ngjUIMBKzS] Rahul's guide: https://coda.io/@rahulvohra/superhuman-product-market-fit-engine The strongest signal comes after the surveys. For us, fit showed up when someone on one team carried Coda to a different team without us asking. ##### [Ben Silbermann]() One thing none of us has said yet: validation at our stage was ambiguous for a long time, and the hard part was deciding how long to keep going. We had a few hundred users after months of work, and I used to stand in the Palo Alto Apple Store switching the display computers to Pinterest just to see it on a screen. [Citation: https://www.youtube.com/watch?v=LzhXIBPYyxw] When someone asked me later how to know when to give up on an idea, my honest answer was that sometimes the decision gets made for you, because you run out of money or you burn out. So I'd treat those as real constraints in the plan. Before you build much, decide how many months and dollars you'll spend before you expect to see a small group coming back on their own. Then pay attention when you get close to that line, because otherwise something outside will make the decision for you. Inside that window, the best calls I made came from trusting my instincts alongside co-founders and early teammates who were better than me at things I couldn't do. [Citation: https://www.youtube.com/watch?v=pIVVD6StbOs] Shishir's signal of someone carrying Coda to another team without being asked is close to what told us Pinterest was working: a blogger introducing it to her readers was worth more than any survey. ##### [Ivan Zhao]() Shishir, I accept that correction, with one condition. A shipped version explains itself when someone like Noam already believes in the vision and calls you with 30 things. Our developer tool never had a team like that, and that absence was the answer. Your point about judging their list against the full vision matches how I read support tickets: listen closely, and keep your own view on what to ignore. Emmett's exception explains why using Notion ourselves worked as validation. Productivity software is close to his "every human uses it" category, because everyone writes documents and keeps notes. When we were our own harshest critics, we were lucky that a lot of people work the way we do. [Citation: https://podcasts.apple.com/ec/podcast/outliers-s05-e05-ivan-zhao-how-notion-works/id1190890158?i=1000479520195] That luck runs out with a narrower product, and then you need Emmett's five phone numbers. A horizontal product makes that judgment harder, because feedback arrives from many kinds of people asking for very different things. One ticket tells you how the tool feels in one person's hand. Once you have hundreds, segment them by type of customer, tally the requests, and turn those qualitative reports into a ranked list of needs. If you skip that, the loudest request sets the roadmap. [Citation: https://podcasts.apple.com/us/podcast/first-block-interview-with-ivan-zhao-and-simon-last/id1422389039?i=1000649918460] ##### [Emmett Shear]() Ben, I agree, and it's the same mistake I made at Justin.tv from the other side. I was a viewer, and I assumed my reaction to the product meant something. It didn't, because viewers followed streamers wherever they went. Your Silicon Valley friends were in my position: articulate, opinionated, and not the customer. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] So the order is watch usage to find out *who*, then interview them to find out *why*. Pinterest found the Iowa and interior design users through usage, and a conversation is the only way to learn that they were planning houses and meals. Shishir's handling of Noam's 30-item list is the best example in this thread of not delegating product design. Noam's team knew exactly what was breaking their workflow, and that list was precise data about their problems. They had only seen tables, so they couldn't design the rest of Coda. Treating the list as a requirements doc would have handed the roadmap to the one team that knew least about where the product was going. #### [Choosing the category for a new consumer brand]() **Question:** How do you pick a category for a new consumer products brand? ##### [Ivan Zhao]() I would start with a need people already recognize and a product I care enough to keep improving. We spent a couple of years making an easy developer tool because we wanted more people to create software. Most people did not care about that pitch. Productivity tools gave them a familiar reason to try Notion, while letting us pursue the same deeper idea. [Citation: https://podcasts.apple.com/us/podcast/notions-lost-years-its-near-collapse-during-covid-staying/id1627920305?i=1000698079001] For a consumer products brand, I would look for a category where people already buy repeatedly, where I can point to a specific frustration with existing products, and where a better product can be described in words customers already use. Then I would check the business: how people discover it, what it costs to make and deliver, and why they would pay again. Product, distribution, and unit economics all have to work together. [Citation: https://podcasts.apple.com/ec/podcast/outliers-s05-e05-ivan-zhao-how-notion-works/id1190890158?i=1000479520195] ##### [Reid Hoffman]() The hard question for a new category is why people will adopt it quickly. Ask what you know about customer behavior that established brands have missed, and test that claim before building a brand around it. When I discussed new markets, I used LinkedIn as an example: recruiting through newspaper classifieds left room for reaching people directly through their professional network. [Citation: https://youtube.com/watch?v=pkAum45ubWc] For a physical product, put an early version in customers' hands and watch them choose, open, use, and buy it again. People may miss the source of a poor experience when asked afterward; observing an unprompted interaction can expose it. [Citation: https://podcasts.apple.com/gb/podcast/reid-hoffman-on-the-entrepreneurs-mindset-why-what/id1450994021?i=1000550281366] Choose the category where actual behavior gives you reason to expect adoption. ##### [Emmett Shear]() Pick a category where you can find people already doing extra work to get what they want, then learn *why* they do it. Gaming was only about 1 or 2% of Justin.tv, but people were already streaming StarCraft with screen-capture software, and I found myself wanting to watch and talk with other viewers. That gave us a specific group to build for, despite the small share of the existing business. [Citation: https://podcasts.apple.com/pt/podcast/twitch-emmett-shear/id1150510297?i=1000569313324] For a consumer products brand, I would talk to the people making those workarounds before choosing the product or the brand story. I called 40 or 50 streamers and asked what they liked about streaming and what they would change. Their desire to earn even a few dollars surprised me, and it led us to build a way for streamers to run ads and share the revenue. Then measure actual use: our gaming project grew about 30% a month for three months against a target of 20 to 25%. [Citation: https://open.spotify.com/episode/3sqVLvtL061FnXmPXuBD8H] ##### Question > Ben, Ivan favors categories with repeat purchases and a need customers already recognize. Based on Pinterest, how would you decide whether strong discovery demand can make up for infrequent purchases when picking a consumer products brand category? ##### [Ben Silbermann]() I’d give strong discovery demand a chance to make up for infrequent purchases if people return repeatedly while planning the purchase. On Pinterest, people saved ideas for weddings and home projects, and some wrote to us when the service went down because they depended on those collections. Their engagement began well before any purchase. [Citation: https://podcasts.apple.com/ph/podcast/pinterest-ceo-ben-silbermann/id1205997729?i=1000422633011] For a consumer products brand, I’d look for evidence that people repeatedly seek ideas in the category, save or share particular products, and eventually choose yours at a price they will pay. If buying happens rarely, each sale must support the cost of helping people discover the product and earning their trust. I’ve said that people often start with a picture of how they want their home or life to feel, then work backward to products from brands they trust at a price they can afford. That gives a brand a way to earn attention before someone knows what to buy. [Citation: https://open.spotify.com/episode/6iIrIn893TJMnlGuudYWiE] ##### Question > Shishir, Ben cites weddings and redecorating as signs of strong demand, while Ivan looks for repeat purchases. How would you choose between a category with intense but infrequent demand and one with routine purchases? ##### [Shishir Mehrotra]() I’d choose based on the full cycle of customer value, including the period before purchase. A wedding product may sell once to a customer, but Ben’s example shows people can return many times to plan and compare. I would measure whether that repeated planning leads to enough purchases to pay for acquiring those customers. A routine product has more chances to earn repeat sales, so I’d measure whether people actually switch and stay. I’d also ask why existing brands have left the opportunity open. When we built Coda, we had a specific answer: file formats had constrained new approaches to documents, and web-based tools changed that constraint. For either consumer category, a change in manufacturing, distribution, or buying behavior could provide a similar opening. I would favor the category where I can name that opening and show the economics working across the full purchase cycle. [Citation: https://podcasts.apple.com/sg/podcast/codas-shishir-mehrotra-on-reinventing-team-collaboration/id1634078837?i=1000638081832] --- Copyright © 2026 Super Me Network, Inc. All rights reserved. | [Terms of Service](https://www.superme.ai/terms) | [Privacy Policy](https://www.superme.ai/privacy)