Thoughts on learning to "fail fast" in research. When you have a new idea, and the project is worthwhile only if P, then you need to check P first, not last. Examples: 1. You can only establish causality with a certain kind of data.

2. Your theory is only interesting if a certain conjecture holds.
3.Your experiment is only convincing if you can rule out a certain confound.

When the relevant condition can be cheaply checked, this has to be the first thing you do, not the last.

Why do people make this error? Many grad students have the mindset that ideas are scarce and time is plentiful. "Having an active project" is reassuring and gives structure. Discarding a project feels like a setback, so it's tempting to put off checking necessary conditions.

By contrast, experienced researchers have the mindset that ideas are plentiful and time is scarce. Most projects fail, and discovering this quickly lets you put your time to better use.

Hence, one should prioritize tasks that can quickly reveal that a project is not worthwhile, even if those tasks are not directly productive to completing the project!

Examples of such tasks:
1. Running online pilot experiments to see if the data is too noisy to detect reasonable effect sizes.
2. Writing code that tries to generate counterexamples to key conjectures.
3. Searching for confounds that, if plausible, would torpedo the project.

In early-stage discussions, it's tempting to focus on the upsides of the project. (it's your idea, you're excited about it!) but your time is valuable, most projects fail, and you should check the necessary conditions first.

Further conjecture: this is partly why students can feel like faculty at brown bag lunches are "too critical", while faculty feel like they're "being helpful". They have different understandings of success rates and order of operations.