Six Rules for Effective Forecasting | Paul Saffo, 2007

Pauil Saffo; Six Rules for Effective Forecasting; In Harvard Business Review (HBR); 2007-07/2007-08.
Paul Saffo (paul@saffo.com) is a forecaster based in Silicon Valley, in California.

Listicle
  1. Define a Cone of Uncertainty
  2. Look for the S Curve
  3. Embrace the Things That Don’t Fit
  4. Hold Strong Opinions Weakly
  5. Look Back Twice as Far as You Look Forward
  6. Know When Not to Make a Forecast

Concept

Mentions

  • Verbs
    • forecast
    • predict
    • identify
  • Adjectives
    • Preordained
    • Predestined
    • Uncertainty
  • Nouns
    • Signals
    • Possibilities
  • Work Products
    • Map of uncertainty
    • S-Curve of Adoption
  • <quote><snip/>, the forecaster’s task is to map uncertainty, for in a world where our actions in the present influence the future, uncertainty is opportunity.</quote>
  • Prediction is concerned with future certainty
  • Forecasting is concerned with the identification of (all) possibilities, not a limited set of illusory certainties.
  • A prediction does not have to have an internal logic,
    A forecast must have a logic to it.
  • The consumer of a forecast is not a trusting bystander
    The consumer of a forecast is a participant and a critic of the work product.
  • Forecasting identifies an S-curve pattern as it begins to emerge, well ahead of the inflection point.
  • Personas
    • forecasters
    • seers
    • prophets
  • Work Product
    • good forecasts
    • bad forecasts
  • Forecasts are meant to be scribbled on, disagreed with, and tossed out—and replaced with new, better ones.

Argot

  • VUCA → Volatility, Uncertainty, Complexity, Ambiguity
    per Jimi Wales’ Wiki.
  • Decision space
  • Intuition … the “spidey sense,” “the $gender’s intuition”
  • Cone of uncertainty
  • Frequentist (belief systems)

Who

  • Roy Amara, futurist.
  • William Gibson, bookist, fiction.
  • Marshall McLuhan, theorist, prophet.
  • Erich Honecker, prime minister (what did they call him?), East German, 1989-01.

Quoted

  • <paraphrase> there is a tendency to overestimate the short term and underestimate the long term. </paraphrase> Roy Amara to Paul Saffo circa 1977.
  • <quote>The future’s already arrived. It’s just not evenly distributed yet<quote>, attributed to William Gibson.
  • <quote>we live in a world where the sole remaining superpower is too powerful to ignore but too weak to make a difference.</quote>, on deep background.

Aphorisms

  • <paraphrase>Son, never mistake a clear view for a short distance</paraphrase>, attributed on deep background to “a rancher.”
  • <paraphrase>history doesn’t repeat itself, but sometimes it rhymes</paraphrase>

The Rules

Rule 1: Define a Cone of Uncertainty

Method
  • Define the cone of uncertainty to support strategic judgment.
  • The geometric analogy
    • The closer to the center of the cone’s main axis they are, the more likely these events are to transpire.
    • A dotted line across the middle of the cone, the “expected normal” case
    • The edges are wild speculations
Other
  • factors—relationships among elements
  • distinctions in degree vs distinctions in kind;
    c.f. utility usage contra entertinment usage, e.g. of robots.
  • outliers, “wild cards”
    • trends or events that have low probability but high impact
      probabilities of occurrence under 10% or unquantifiable.
    • e.g. finding radio evidence of intelligent life somewhere else in the universe.
  • acknowledge sufficiently outlandish possibilities without losing your audience.
Humor

Given aliens show up…

  • One-third of the world’s population would probably worship the remote intelligences,
  • One-third would want to conquer them,
  • One third (the readers of this magazine article) would want to do some extraterrestrial market research and sell them something.
And…
  • human nature
    • is hardwired to abhor uncertainty.
    • is fascinated by change
  • Uncertainty, e.g. Y2K
  • Claim: <quote>The result of the Y2K nonevent was that many people subsequently rejected the possibility of other wild cards ever coming to pass. As a result, 9/11 was a much bigger surprise than it should have been.</quote>
  • consider the whole cone

Rule 2: Look for the S Curve

  • e.g. Moore’s Law
  • Viewpoint: Very large, broadly defined curves are composed of small, precisely defined and linked S curves.
  • Therefore, look for
    • larger S-curves containing
    • smaller S-curves subsumed
  • Claim: curve of Moore’s Law is still unfolding—it is still a “J”—with the top of the “S” nowhere in sight.
    • This is now known in 2017 to be false.
    • Moore’s Law and deep submicron design has hit scaling limits.
    • We have more cores but not faster cores.
  • Generalized Moore’s Law
    • There is an effect on density regardless of the material
    • Claim: Generalized Moore’s Law is still in force
  • Forecasting identifies an S-curve pattern as it begins to emerge, well ahead of the inflection point.
  • Forecasters can do worse than ordinary observers when it comes to anticipating inflection points (the question of timing)
    <ahem>as stated, they can also do better</ahem>
    <quote>Ordinary folks are simply surprised when an inflection point arrives seemingly out of nowhere, but innovators and would-be forecasters who glimpse the flat-line beginnings of the S curve often miscalculate the speed at which the inflection point will arrive.</quote>
  • Diffusion of innovation
    requires: (roughly) “a generation”
  • Example
    • Television → 20 years + WWII.
    • Silicon Valley → 20 years
    • Internet → 20 years (since invention)
  • the left-hand part of the S curve is much longer (slower) than most people imagine.
  • events will unfold slowly; no shift is in the wind.
  • the opportunities will be very different from those the muggle predictions
  • e.g. Personal Computer (PC) about entertainment, not work, not book-copied media, encyclopedias for education.

Rule 3: Embrace the Things That Don’t Fit

  • Become attuned to “things that don’t fit.” Intuition, spidey-sense. Needs systematization.
  • <quote>But by definition anything that is truly new won’t fit into a category that already exists.</quote>
  • Examine the failures.
  • Indicators come in clusters.
Exemplars
  • Online multi-player games with sales of virtual goods.
  • DARPA Grand Challenge circa 2004.
  • Roomba by iRobot.
Online Multiplayer Role-Playing (Shooter) Games

Sales of characters and in-game objects for EverQuest on eBay in the late 1990s.
Claim: such is now $1B/year run-rate business.

  • EverQuest
  • Habitat, an online environment developed by Lucasfilm Games in 1985.
  • Multiple User DimensionsDungeons (MUDs)
  • Second Life, by Linden Lab; twenty years after Habitat
  • Ultima
DARPA Grand Challenges, circa 2004

100-mile-plus race across the Mojave Desert.
for $1 million prize

  1. 2004-03 → none finished.
  2. 2005-10? → five finished
Roomba, iRobot
  • 2007 → seemed like an indicator.
  • 2017 → no longer “a thing” has come and gone.

Rule 4: Hold Strong Opinions Weakly

  • DO NOT: (over-)rely on one piece of seemingly strong information because it happens to reinforce the conclusion he or she has already reached.
  • <quote>In forecasting, as in navigation, lots of interlocking weak information is vastly more trustworthy than a point or two of strong information. </quote>
  • Paradigm shifts
  • Thomas Kuhn, The Structure of Scientific Revolutions, WHEN?
  • <quote>Good forecasting is the reverse: It is a process of strong opinions, weakly held.</quote>
  • <quote>Having strong opinions gives you the capacity to reach conclusions quickly, but holding them weakly allows you to discard them the moment you encounter conflicting evidence.</quote>
Parable

<vignette>This lesson was tragically underscored when nine U.S. destroyers ran aground on the shores of central California on the fog-shrouded evening of September 8, 1923.
The lost ships were part of DesRon 11, a 14-ship squadron steaming from San Francisco to San Diego. Misled largely by overreliance on the commander’s dead-reckoning navigation, the squadron undershot the turn into the Santa Barbara Channel and instead ended up on the rocks at Point Pedernales, several miles to the northwest. The squadron had navigated by dead reckoning for most of the trip, but as the ships approached the channel, the squadron’s commander obtained bearings from a radio direction station at Point Arguello. The bearing placed his ship, the Delphy, north of its dead reckoning position. Convinced that his dead reckoning was accurate, the commander reinterpreted the bearing data in a way that confirmed his erroneous position and ordered a sharp course change towards the rapidly approaching coast. Nine ships followed the disastrous course. Meanwhile, the deck officers on the Kennedy, the 11th boat in the formation, had concluded from their dead reckoning that they in fact were farther north and closer to shore than the position given by the Delphy. The skipper was skeptical, but the doubt the deck officers raised was sufficient for him to hedge his bets; an hour before the fateful turn he ordered a course change that placed his ship several hundred yards to the west of the ships in front of them, allowing the Kennedy and the three trailing destroyers to avert disaster. The essential difference between the two skippers’ responses was that the Delphy’s skipper ignored evidence that invalidated his dead-reckoning information and narrowed his cone of uncertainty at the very moment when the data was screaming out to broaden it. In contrast, the Kennedy’s skipper listened to the multiple sources of conflicting weak information and concluded that his ship’s position was much less certain than assumed. He hedged their bets and, therefore, saved the ship. </quote>

Rule 5: Look Back Twice as Far as You Look Forward

  • Marshall McLuhan, is quoted.
Parable

Something about “The New Economy”

  • Google
  • Yahoo
  • Google
  • Bubble I
  • Dow Jones Industrial Average
Exemplars
  • From Mergers&Acquisitions (M&A)
    • Jerry Levin, for instance, sold Time Warner to AOL
  • From warfare
    • Vietnam
    • Iraq I
    • Iraq II

Rule 6: Know When Not to Make a Forecast

  • There are vastly more elements that do not change than new things that emerge.
  • e.g. unchanging laws of economics.
  • Be skeptical.
  • <quote>At the end of the day, forecasting is nothing more (nor less) than the systematic and disciplined application of common sense.</quote>
  • <quote>The best way to make sense of what lies ahead is to forecast for yourself.</quote>

Previously filled.

Comments are closed.