What Does a p-value Really Mean?

Agronomy
Statistics
Decision Making
Author

Dr. Adrian Correndo

Published

January 21, 2026

🌾 Why talk about p-values on the farm?

Modern agriculture is full of experiments:

  • 🚜 Does this new fungicide really pay?
  • 🌱 Should I push nitrogen from 150 to 200 kg N/ha?
  • 🧪 Is this biological product doing anything, or is it just marketing?

When agronomists and researchers answer these questions, they often report p-values:
“Yield increase was significant at p < 0.05.”

But what does that actually mean, probabilistically?
And more importantly: does it answer the questions farmers care about?

This short article unpacks the meaning of a p-value and contrasts it with a more intuitive, Bayesian way to talk about probability for farm decisions.

🎲 What is a p-value, probabilistically?

Suppose we are testing:

  • H₀ (null hypothesis): there is no true difference between two treatments
  • H₁ (alternative): there is a true difference

We run an experiment, calculate a test statistic (like a t-statistic), and obtain a p-value.

Probabilistic definition:

The p-value is the probability of observing data at least as extreme as what we observed, if the null hypothesis were true.

In mathematical notation:

[ p = P( H_0 ) ]

So if p = 0.03, it means:

If there were truly no effect, then in repeated experiments you would see a result this extreme (or more extreme) only 3% of the time, just by random chance.

It does not mean:

  • ❌ “There is a 3% chance the null is true.”
  • ❌ “There is a 97% chance the treatment works.”
  • ❌ “There is a 3% chance this result is due to chance.”

The key is the direction of the conditioning:

  • p-value: probability of the data, given H₀
  • What farmers want: probability of H₀ or H₁, given the data

These are not the same thing.

🌱 An agronomy example: nitrogen rate comparison

Imagine a trial comparing 150 vs 200 kg N/ha in corn across several fields.

You find:

  • 🌽 Mean yield difference: +0.6 t/ha (200 > 150)
  • 📉 p-value: 0.04

The strict interpretation is:

If there were really no yield difference between 150 and 200 kg N/ha, you would see a difference of 0.6 t/ha or larger (in either direction) only about 4 times in 100 experiments like this.

Statistically, we might say this result is “significant at the 5% level.”

But the farmer’s questions are different:

  • 💭 “What is the chance that 200 kg N/ha will actually give me more yield this year?”
  • 💰 “What is the chance it will give me more profit, considering N price and grain price?”
  • ⚖️ “How big is the upside, and how big is the downside risk?”

The p-value does not directly answer any of those.

⚠️ Limitations of p-values for farm decisions

P-values are useful for researchers, but they are awkward for decision-makers. Some key limitations:

1️⃣ They answer a different question

  • Statistics question:
    > If the treatment does nothing, how surprising is my data?
  • Farmer question:
    > Given my data (plus what we knew before), how likely is it that the treatment helps?

These are related but not the same.

2️⃣ The 0.05 cutoff is arbitrary

A result with p = 0.049 is “significant”; p = 0.051 is “non-significant.”
In practice, these two situations often provide very similar evidence, yet the label changes.

For communication and decision-making, this can be misleading.

3️⃣ Magnitude and economics can be ignored

It’s possible to have:

  • 📌 A tiny yield gain with very small p-value (statistically significant but not profitable), or
  • 📌 A large yield gain with p = 0.10 (not “significant” at 0.05), which might still be attractive if the downside risk is small.

Farmers (and advisors) care about effect size and profit, not just whether p < 0.05.

4️⃣ Intuition is backwards

The p-value starts from assumming “no-effect” and asks about the probability of the data.
But most people naturally think in the Bayesian direction:

Given what I’ve seen, how likely is it that there’s a real effect?

🎯 A Bayesian way to think: probability of benefit

Bayesian thinking flips the conditioning:

Start with what we already know (prior), then update with new data to get a posterior: a direct probability statement about the effect.

Instead of talking about p-values, we talk about distributions and probabilities of benefit.

Imagine we combine:

  • 📚 Prior knowledge (multi-location trials, previous years, expert opinion)
  • 🧪 New data from this year’s on-farm trial

We might end up with statements like:

  • “Given everything we know, there is a 75% chance that 200 kg N/ha yields at least 0.4 t/ha more than 150 kg N/ha.”
  • “There is a 60% chance that 200 kg N/ha increases profit at current prices, and a 20% chance that it reduces profit by more than $20/ha.”

This maps much more directly onto real decisions:

  • If you are risk-neutral, you might choose the treatment with higher expected profit.
  • If you are risk-averse, you might prioritize avoiding the downside and accept a lower expected profit.

The key point: Bayesian outputs describe how likely different outcomes are, which is exactly what farmers care about.

👂 What farmers actually hear (and what they need)

In extension work and on-farm research networks, results are often summarized as:

  • “The fungicide treatment was not statistically significant.”
  • “The new practice was significantly better than the control at p < 0.01.”

From a farmer’s perspective, more helpful summaries would be:

  • “Across 10 trials, the fungicide increased yield in 7 out of 10 site-years.”
  • “Given this data and previous research, there is about a 70% chance the fungicide increases yield, and about a 50% chance it increases profit at current prices.”
  • “The risk of losing more than $20/ha with this fungicide appears low.”

You can deliver this kind of language using fully Bayesian models, or by combining frequentist estimates with simple probability approximations. The important shift is conceptual:

Move from “Is it significant?” to “What is the probability and size of benefit, and what is the risk?”

🧪 Practical implications for agronomy and on-farm research

For researchers and students:

  • 🧠 Understanding p-values is still essential because they dominate the scientific literature.
  • Always pair p-values with:
    • 📏 Estimated effect size (e.g., t/ha difference)
    • 📊 Confidence (or credibility) intervals
    • 💸 Economic interpretation

For farmers and advisors:

  • Emphasize:
    • 📈 Probability the practice increases yield
    • 💰 Probability it increases profit
    • ⚖️ Distribution of possible outcomes (risk vs reward)

This mindset naturally leads toward a Bayesian way of thinking, even if you don’t explicitly present the statistics as “Bayesian.”

✅ Take-home messages

  • Probabilistic meaning of a p-value:
    [ p = P( H_0 ) ]
    It is a probability about the data, given the assumption of no true effect.

  • Farmer relevance:
    P-values help researchers avoid being fooled by noise, but they do not directly answer
    > “What are my chances that this will help me?”
    or
    > “Will it pay?”

  • Bayesian perspective:
    Focuses on (P( )): probabilities of benefit and profit, which align much better with real decision-making on the farm.

In other words:

🧰 Keep p-values in the statistical toolbox, but speak to farmers in terms of probabilities of benefit, risk, and profit—which is, in spirit, a Bayesian way of thinking.