An early-stage north star: what to count, and when to drop it

Define the north star from how often the problem occurs, the action that shows it was solved and the user it describes, treat it as a retention scoreboard with activation underneath, and drop a candidate the team can raise without users getting more value.

• Start from the value and its natural frequency. Hila Qu names the action that proves a user got value, like Airbnb's nights booked, and would skip revenue because a SaaS customer can keep paying after they stop using the product. Brian Balfour's Use Case Map adds how often the problem occurs, the core action and who is measured. Teams want to be higher-frequency than the problem allows, he says, and a daily metric on a monthly problem makes a healthy product look broken.
• Treat it as a scoreboard with inputs underneath. Brian says you can't run experiments against a scoreboard, so he would break the north star into inputs the team can move and add one or two counter-metrics. Pinterest's WARC combined repins and clicks, so an experiment could trade one for the other and still look like a win. Casey Winters adds that Grubhub's revenue metric pushed it to care which restaurants it grew, where Seamless's GMV rewarded orders and order size.
• Retention on top, activation as its leading input. Lauryn Motamedi would draw the funnel first and says the early metric is almost always some form of activation that roughly 20% of new users should be able to reach. Brian argues activation has to be derived from retention, and reads Airtable's week-4 multi-user active as a retention definition. Lauryn takes that framing but would still require one high-value action in the activation definition, so people poking around don't count.
• In a marketplace, count what needs both sides. Gustaf Alströmer recounts a panel story of Uber picking weekly trips over monthly active riders, because a trip needs a rider and a driver. Casey would put a supply retention measure under that count: at Grubhub, the team tracked how many orders a week a restaurant needed so it wouldn't leave the platform.
• Validate the definition with small numbers. With 50 to 200 users, Brian would skip the statistical test: recruit a deliberate pool, plot retention at the chosen frequency, check whether retained users did the core action, and name a cause for each churned user. Hila would change the product so more users reach the action and compare the next cohorts with earlier ones, then interview 10 to 20 retained users. Lauryn would check whether the metric would have rewarded recent decisions that clearly helped customers.
• Drop it when it moves without value. Brian would drop a candidate the team can raise without users getting more value, one that retained users routinely skip, or one whose curve only flattens at a lower frequency than assumed. Hila adds a fourth case: an intervention raises the share doing the action and retention holds flat across cohorts. Gustaf warns that a north star that drops once you stop pushing is counting activity you bought.