Perspectives on Problems
What the pioneers saw about problems, and what still holds in 2026.
- The question
- What is a problem, and when is prediction enough?
- You will learn
- How pioneers classified problems: ill structured, wicked, messes; predict, explain, or intervene
- How to test a classic claim against current evidence, and give a verdict with its limits
- How to tell problems where prediction is enough from problems that need causal reasoning
- Recurring failures of deployed systems: proxy labels, selective labels, feedback loops, leakage, underspecification, hidden dependencies
- Fourteen questions to examine a new problem before you build
- Slides
- Slides (PDF)
For seventy years, researchers and practitioners have argued about what a problem is and how data and machines should help solve it. This session takes their strongest claims, tests each one against 2026 evidence and deployed systems, and keeps what still works as rules for practice.
What it highlights
- The question comes before the method.
- The label you choose is the policy you set.
- Prediction is not intervention.
- Most of a deployed system is not the model.
- Purpose and accountability stay with people.
The question chain
Each question comes from perspectives in this session. Together they form a sequence you can walk on any new problem.
- What decision, by whom, and what does an error cost?The problem · Tukey, Obermeyer, Passi and Barocas
- Does our action change the outcome?Causal or not · Kleinberg 2015, Pearl, Hernán
- Which representation makes the pattern learnable?Deep learning · Sutton, Halevy, Geirhos
- Can we reuse what is already learned?Foundation models · Vapnik (revised), Silver and Sutton
- How uncertain is the answer?Probabilistic · Box, D'Amour, Salganik
- What should we do, given all of that?Decision · Simon, Meehl, Wiener
- Can it run at our scale, latency, and cost?Engineering · continues in course 05
- How do we know it works, and keeps working?Evidence · continues in course 02
The loop: actions change the world, and the world changes the data. Return to the first question.
Try this
Before you open the slides, two minutes, on paper: write one belief about solving problems with data and AI that you expect to still hold in 2036.
Then, after the slides, choose one perspective and one case from this session, or a system you know well, and produce a one-page analysis:
- State the claim in one sentence, with its source.
- Summarize what the case shows, with numbers and the type of each source.
- Give a verdict: holds, holds with revision, under pressure, or contested. Say which evidence would change it.
- Write the "keep it today as" rule for practice.
- Name who the system serves, who it leaves out, and what it rewards.
Read before Session 2
- Kleinberg et al., Prediction Policy ProblemsAmerican Economic Review, 2015
- Passi and Barocas, Problem Formulation and FairnessFAT*, 2019
- Bernardi et al., 150 Successful Machine Learning ModelsKDD, 2019
- Raji et al., The Fallacy of AI FunctionalityFAccT, 2022
- Hardt and Recht, Patterns, Predictions, and Actions, introductionmlstory.org
Settled · 11 Oct 2026
Questions about this session, or a case that argues with a verdict? Write to mepc3197@columbia.edu · the title goes in the subject line
Related
- Preferred Problem Spaces — Building My Career Around What Keeps Me Curiousessayphilosophy
- The Next Starting Lineessayphilosophy
- Complex Systems, Not Integrations: What Forward-Deployed/AI Engineering Is Missingessayscience · engineering