Check If Your 70% Bets Hit 70%: Decision Journal for Founders

A decision journal, in the sense that matters for founders and product leaders, is a shared system for recording a bet’s hypothesis, probability, and success metrics before the outcome is known, then reviewing what actually happened. The core payoff is calibration: teams stop confusing luck with skill and start trusting their own judgment because they can check it against a written record. There are purpose-built tools for running this practice at a team level.

TL;DR:

• Lock original probabilities and details at each stage to prevent revision bias and ensure accurate calibration over time.
• Review calibration through probability bins, aiming for actual success rates to match predicted probabilities within acceptable margins.
• Use simple, standardized fields in the journal, including hypothesis, timeframe, objective metric, probability, and disconfirming evidence for consistency.
• Conduct weekly and monthly reviews to update bets and analyze overall calibration, with a quarterly report to track long-term accuracy.
• Enforce strict governance by timestamping changes and deterring edits, and incorporate premortems to surface hidden risks before committing to decisions.

Why teams need a decision journal to fight hindsight bias

Once you know how something turned out, your brain rewrites what you “always knew.” Research in BMJ Quality & Safety found that outcome knowledge biases later judgments and inflates how likely people believe an event was, and the judges rarely notice it happening. That makes honest learning nearly impossible without something written down before the fact.

The problem is not limited to individuals. A National Bureau of Economic Research working paper tracking a five-year, ten-wave panel of over 6,000 firms found large, persistent forecasting errors and over-precision, and simple training did little to fix it. Meanwhile, aggregated human judgment can work well: a study of roughly one million resolved forecasts found that events assigned a given probability occurred at close to that rate across domains and platforms.

• Hindsight bias erases the lesson a bad call was supposed to teach.
• Firms misforecast persistently, and training alone rarely closes the gap.
• Aggregated, written-down probabilities tend to be well calibrated, even at scale.

Calibration research shows aggregated forecasts can track real-world frequency closely, according to the one-million-forecast study, which is the case for building the habit at the team level rather than trusting memory alone.

The minimal fields every team decision journal needs

You do not need a complicated system to start. You need a short, consistent set of fields that force the reasoning into the open before anyone knows how the bet turns out.

• Title: a short name for the bet, written like a headline.
• Hypothesis: what you believe will happen and why.
• Timeframe: the date you will check the result.
• Objective metric: the number that will decide win or lose, agreed in advance.
• Numeric probability: your confidence as a percentage, not a phrase like “pretty sure.”
• Trade-offs accepted: what you are giving up or risking to make this bet.
• Disconfirming evidence: what result would prove the hypothesis wrong.
• Owner: the person accountable for logging updates.
• Status checkpoints: dates for Running, Reviewing, and Decided.

The bins matter more than the precision because you will later sort many bets into them and check how often each bin actually won.

Take a pricing test as an example: hypothesis is “raising the starter plan by 20% will not increase churn beyond 2%,” objective metric is 30-day churn rate, probability is 70%, trade-off is slower signup growth, and disconfirming evidence is churn above 3% in the first two weeks.

Pro Tip: Write the probability before you look at any dashboard that day. Once you have seen a number, your estimate is contaminated.

From idea to post-mortem: running the workflow

A journal entry is only useful if it moves through clear stages with a fixed record at each one.

1. Idea: the bet is proposed with a rough hypothesis and no commitment yet.
2. Prioritized: the team agrees it is worth running and assigns an owner.
3. Running: the hypothesis, metric, probability, and disconfirming evidence are locked and timestamped.
4. Reviewing: the timeframe has passed and results are being pulled together.
5. Decided: the bet is closed as Won, Killed, or Inconclusive, with a post-mortem attached.

What you log changes at each stage:

• At the start, lock the original probability so nobody can quietly revise it later.
• During the run, log any material changes to scope or metric, timestamped separately from the original entry.
• At review, pull the actual result against the objective metric agreed at the start.

The post-mortem checklist closes the loop: record the outcome, compare the predicted probability to what happened, separate whether the result came from a sound decision or from luck, and write down one learning the team cannot rewrite later. A premortem study using controlled experiments found that framing a review around “assume this failed, why?” surfaces more usable risk information than a standard pros-and-cons list, which is worth borrowing for the post-mortem itself, not just the planning stage.

Turning entries into calibration you can measure

A pile of journal entries is not calibration.

• Group closed bets into probability bins (50%, 60%, 70%, 80%, 90%) and calculate the win rate inside each bin.
• Plot predicted probability against actual hit rate to see a calibration curve; a straight diagonal line is the goal.
• When combining multiple people’s forecasts on the same bet, weight toward forecasters with a track record instead of averaging everyone equally, and trim outlier guesses before combining.

28010, which shows how far overconfidence can drift without a feedback loop. Forecast-combination research also suggests that informed pooling, weighting toward better forecasters rather than simple averaging, can sharpen group estimates, as documented in research on calibrated combining.

A workable cadence: a short weekly triage to update Running bets, a monthly portfolio review across open and closed bets, and a quarterly calibration report showing hit rate by probability bin.

Keeping the journal honest: governance and common mistakes

A journal only works if entries cannot be edited after the fact. Lock the original hypothesis, metric, and probability the moment a bet enters Running, and timestamp every later change separately. Some teams anonymize calibration dashboards at first so individual track records do not turn into blame sessions.

• Vague metrics that let anyone declare victory after the fact.
• Rewriting the original rationale once the outcome is known.
• Premortems that turn into a polite pros-and-cons list instead of genuine prospective hindsight, a distinction Brookings researchers tie directly to whether the exercise surfaces real risk.
• No enforcement, so the habit quietly dies after a few weeks.

Pro Tip: Before any review meeting, require every attendee to write down what they predicted, before anyone reads the actual result aloud.

How founders and product leads actually use this day to day

A pricing test that missed its target is a common first entry. The learning got written down before anyone could blame the marketing calendar.

Success shows up as better alignment in planning meetings, a rising hit rate inside each probability bin over a few quarters, and fewer conversations that turn into blaming a person instead of examining a decision.

The record is worth more than the memory. Once a probability is written down before the outcome, nobody gets to quietly rewrite what they “always knew.”

— Cesar

Betlog: how it maps to the guide and where to get started

Running this system on a whiteboard works for a while, then entries go stale, probabilities get quietly revised, and nobody runs the quarterly calibration report because nobody owns it. Some digital tools have been built around the template in this guide, helping reduce upkeep.

• Staged workflow matching Idea, Prioritized, Running, Reviewing, and Decided, with timestamped, locked entries.
• Numeric probability fields and post-mortem reports that separate luck from skill automatically.
• A permanent record you can filter by bin, owner, or status for the quarterly calibration review.

Betlog’s One plan runs $390 per year or $39 per month. See how the workflow fits your team on the Betlog product page or go straight to pricing to start.

Sources

• The knowability of the future: Evidence from one million collective forecasts
• Hindsight ≠ foresight: the effect of outcome knowledge on judgment under uncertainty
• Rationalizing Firm Forecasts (NBER Working Paper No. 33384)
• Premortem: Evaluating two structured analytic techniques for group brainstorming

FAQ

What is a decision journal for a team?

A team decision journal is a shared record where a group logs the hypothesis, numeric probability, and success metric for a bet before the outcome happens, then reviews the result afterward. The goal is separating decision quality from luck, which is much harder to do from memory alone.

How is a decision journal different from meeting notes?

Meeting notes capture discussion; a decision journal captures a locked, timestamped commitment, specifically the probability and metric agreed before the result is known. Once a bet moves to Running, the original entry cannot be quietly edited, which is what protects the record from hindsight bias.

How often should a team review its decision journal?

A short weekly check on open bets, a monthly review of the full portfolio, and a quarterly report on calibration by probability bin cover most teams’ needs. The quarterly report is where you actually see whether your 70% bets are winning about 70% of the time, as calibration research suggests is achievable at scale.

What is a premortem and why does it matter for journaling?

A premortem asks the team to assume a bet already failed and explain why, which surfaces hidden risks better than a standard pros-and-cons list, according to controlled experiments on the technique. Running one before locking a bet’s probability gives the post-mortem something concrete to check against later.

Can Betlog replace a spreadsheet decision journal?

Betlog is built specifically for the staged, probability-based workflow described in this guide, including locked entries and post-mortem reports, which a spreadsheet has to recreate manually. Pricing and setup details are on the Betlog pricing page.

Recommended

• Betlog: Place better bets. Keep the receipts.