Skip to main content
added 87 characters in body; edited tags
Source Link
amoeba
  • 109.1k
  • 37
  • 325
  • 350

Merkle & Steyvers (2013) write:

To formally define a proper scoring rule, let f$f$ be a probabilistic forecast of a Bernoulli trial d$d$ with true success probability p$p$. Proper scoring rules are metrics whose expected values are minimized if f = p$f = p$.

I get that this is good because we want to encourage forecasters to generating forecasts that honestly reflect their true beliefs, and don't want to give them perverse incentives to do otherwise.

Are there any real-world examples in which it's appropriate to use an improper scoring rule?

Merkle, E. C., & Steyvers, M. (2013). Choosing a strictly proper scoring rule. Decision Analysis, 10(4), 292-304.Reference
Merkle, E. C., & Steyvers, M. (2013). Choosing a strictly proper scoring rule. Decision Analysis, 10(4), 292-304

Merkle & Steyvers (2013) write:

To formally define a proper scoring rule, let f be a probabilistic forecast of a Bernoulli trial d with true success probability p. Proper scoring rules are metrics whose expected values are minimized if f = p.

I get that this is good because we want to encourage forecasters to generating forecasts that honestly reflect their true beliefs, and don't want to give them perverse incentives to do otherwise.

Are there any real-world examples in which it's appropriate to use an improper scoring rule?

Merkle, E. C., & Steyvers, M. (2013). Choosing a strictly proper scoring rule. Decision Analysis, 10(4), 292-304.

Merkle & Steyvers (2013) write:

To formally define a proper scoring rule, let $f$ be a probabilistic forecast of a Bernoulli trial $d$ with true success probability $p$. Proper scoring rules are metrics whose expected values are minimized if $f = p$.

I get that this is good because we want to encourage forecasters to generating forecasts that honestly reflect their true beliefs, and don't want to give them perverse incentives to do otherwise.

Are there any real-world examples in which it's appropriate to use an improper scoring rule?

Reference
Merkle, E. C., & Steyvers, M. (2013). Choosing a strictly proper scoring rule. Decision Analysis, 10(4), 292-304

added 38 characters in body
Source Link

Merkle & Steyvers (2013) write:

To formally define a proper scoring rule, let f be a probabilistic forecast of a Bernoulli trial d with true success probability p. Proper scoring rules are metrics whose expected values are minimized if f = p.

I get that this is good because we want to encourage forecasters to estimate thegenerating forecasts that honestly reflect their true probabilitybeliefs, and don't want to give them perverse incentives to othersdo otherwise.

Are there any real-world examples in which it's appropriate to use an improper scoring rule?

Merkle, E. C., & Steyvers, M. (2013). Choosing a strictly proper scoring rule. Decision Analysis, 10(4), 292-304.

Merkle & Steyvers (2013) write:

To formally define a proper scoring rule, let f be a probabilistic forecast of a Bernoulli trial d with true success probability p. Proper scoring rules are metrics whose expected values are minimized if f = p.

I get that this is good because we want to encourage forecasters to estimate the true probability, and don't want to give them perverse incentives to others.

Are there any real-world examples in which it's appropriate to use an improper scoring rule?

Merkle, E. C., & Steyvers, M. (2013). Choosing a strictly proper scoring rule. Decision Analysis, 10(4), 292-304.

Merkle & Steyvers (2013) write:

To formally define a proper scoring rule, let f be a probabilistic forecast of a Bernoulli trial d with true success probability p. Proper scoring rules are metrics whose expected values are minimized if f = p.

I get that this is good because we want to encourage forecasters to generating forecasts that honestly reflect their true beliefs, and don't want to give them perverse incentives to do otherwise.

Are there any real-world examples in which it's appropriate to use an improper scoring rule?

Merkle, E. C., & Steyvers, M. (2013). Choosing a strictly proper scoring rule. Decision Analysis, 10(4), 292-304.

When is it appropriate to use an improper scoring rule?

Merkle & Steyvers (2013) write:

To formally define a proper scoring rule, let f be a probabilistic forecast of a Bernoulli trial d with true success probability p. Proper scoring rules are metrics whose expected values are minimized if f = p.

I get that this is good because we want to encourage forecasters to estimate the true probability, and don't want to give them perverse incentives to others.

Are there any real-world examples in which it's appropriate to use an improper scoring rule?

Merkle, E. C., & Steyvers, M. (2013). Choosing a strictly proper scoring rule. Decision Analysis, 10(4), 292-304.