2
$\begingroup$

When doing Difference in Differences, we basically pretend to know the average treated outcome $\frac{\sum_{i=1}^n Y_i(1)}{n}$ and the average no-treatment outcome $\frac{\sum_{i=1}^n Y_i(0)}{n}$ of an initial group of units (by assuming a parallel counterfactual trend with a secondary group of units). Therefore, we can directly use the fact that the sample ATE is an unbiased estimator of the true ATE to estimate the true ATE by $\frac{\sum_{i=1}^n Y_i(1)-Y_i(0)}{n} = \frac{\sum_{i=1}^n Y_i(1)}{n} - \frac{\sum_{i=1}^n Y_i(0)}{n}$.

This is how I rationalize the Difference in Differences result. The parallel trend assumption spares us the effort of dividing the units into treatment and control groups and it means that there is no selection bias to worry about (which Wikipedia confirms)

LE: To clarify my reasoning, I painted this graph.

enter image description here

By making the Parallel Trends assumption and observing that the second group reaches point a, we automatically know that the first group would have reached point c had it not been for the treatment. We thus have everything we need to know about the first group:

  • the average (observed) treatment outcome = $\frac{\sum_{i=1}^n Y_i(1)}{n}$ = d
  • the average (assumed) no-treatment outcome = $\frac{\sum_{i=1}^n Y_i(0)}{n}$ = c

Therefore, the sample ATE is d-c, which is an unbiased estimate of the true ATE. There are no treatment or control groups, because we don't make any assignment. We literally know or assume both average potential outcomes of the first group.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ How do you know what $c$ is? It's unobserved. The whole point is that the parallel trends assumption implies that $a - b = c - e$; $a$, $b$, and $e$ are observed. So, to get $d - c$, which is the ATE, you need to compute $d - (a - b + e)$. Without a control group, you would not have $a$ or $b$, so you couldn't compute this quantity. $\endgroup$ Commented Sep 3, 2020 at 18:55
  • $\begingroup$ In my interpretation, there is no control group because I only consider the members of the first group to be units in the experiment. And I completely agree with you that we only know c thanks to the Parallel Trends assumption. $\endgroup$ Commented Sep 3, 2020 at 20:41

2 Answers 2

Reset to default
3
$\begingroup$

It seems like your question is asking why we use two-group, two-time point designs rather than one-group, two-time point designs. Why would we need a control group when we have the outcomes both under no treatment and under treatment for the group that receives treatment?

The answer is that the outcomes in the pre-period are not good estimates of the potential outcomes of the post-period had the units not been treated. If there is a natural trend for the outcomes in the absence of treatment (e.g. a maturation effect), only having one group would cause one to mistake the natural trend for a treatment effect. Including a control group that follows that natural trend allows one to subtract off the natural trend observed in the control group from the trend observed in the treated group and purify the portion of the trend due only to treatment. This relies on the assumption that the treated units would have followed the same trajectory as the control units had they not been treated (this is the parallel trends assumption). Without the control group, you have no way to defend your estimate from the argument that the treated units would have followed the observed trajectory even in the absence of treatment.

For example, let's say I'm testing the efficacy of a new drug against the common cold. I take a bunch of people with the cold, measures their symptoms on some continuous scale, give them my drug, and measure their symptoms a week later. I find that all my patients recovered; their cold symptoms are now close to zero. Was my drug effective? We don't know what would have happened had they not received the drug. Colds tend to dissipate in about a week anyway, so how do we know the drug caused the change? Only including a control group and measuring their symptoms along with the treated units would allow me to answer that question. Had I included a control group and found that their outcomes followed an identical trajectory to those of the treated group, I would not be able to claim my treatment was effective; the observed change in the treated group from pre to post was due simply to maturation.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I agree with your reasoning. However, I didn't claim that the pre-period outcomes of the first group are good estimates of the post-period, no-treatment outcomes of the first group. I meant that we can just use the post-period outcomes of the second group to derive the post-period, no-treatment outcomes of the first group (thanks to the Parallel Trends assumption). This is how we get the average alternative outcome of the first group. $\endgroup$ Commented Sep 3, 2020 at 12:10
  • $\begingroup$ I updated my question. Thank you for taking the time. $\endgroup$ Commented Sep 3, 2020 at 13:02
  • 1
    $\begingroup$ We do not actually “know” the first group would reach point ‘c’ by the second time period, which happens to be the first and only post-treatment period. And, applied researchers typically do not “pretend” the group trends would move in tandem in the absence of treatment, they demonstrate it—visually. This will require serial observations of your outcome pretreatment. Is your question based upon the basic two-group/two-period case? $\endgroup$ Commented Sep 3, 2020 at 13:44
  • $\begingroup$ @ThomasBilach yes, it is. Doesn't the Parallel Trends assumption assume that the two trends will move in tandem in the absence of treatment? (Whether we are willing to make that assumption or not, is another problem.) $\endgroup$ Commented Sep 3, 2020 at 14:03
0
$\begingroup$

I figured out the reason of my confusion. We can interpret the Diff-in-Diff technique in two ways:

  1. Only consider the members of the first group to be units in the experiment. This was my initial interpretation. In this case, we don't split the units in control and treatment sub-groups because we don't need to. We have both average possible outcomes under the Parallel Trends assumption, so we can directly estimate the ATE through the sample ATE (which is unbiased), as I described in the question. Note that here we didn't do any selection.
  2. Consider the members of the first group and the second group to be units in the experiment. In this case, the first group is the treatment group and the second group is the control group. And we know that the difference between their observed average outcomes is also an unbiased estimator of the ATE, but only if the selection bias is 0. Which it is. (The selection bias is defined as the difference in the expected outcome without treatment of the treatment group and the control group. The Parallel Trends assumption essentially tells us that this difference is 0.) Note that here we did a selection, but its bias is 0.

Both interpretations reach the same ATE estimate, but for different reasons.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Your first interpretation still makes absolutely no sense to me. How can you think you have both average potential outcomes? $c$ is not observed and the parallel trends assumption doesn't tell you what it is without a control group. $\endgroup$ Commented Sep 4, 2020 at 3:15
  • $\begingroup$ In both interpretations, we have the observational data from both groups at our disposal. The distinction between the two interpretations is how much of this observational data do we include in our simulated 'experiment' design? In the first interpretation, we only consider members of group 1 as being units in the experiment (but only after we observed group 2 and made the PT assumption, thus deriving c). In the second interpretation, we consider members of both groups as being units in the experiment. $\endgroup$ Commented Sep 4, 2020 at 12:32
  • $\begingroup$ This is why in the first interpretation we can't properly call the second group 'the control group'. Because it's not even part of the experiment. It's purely observational data. (I realize that people usually only think about Diff-in-Diff using the second interpretation and this is why it has become so customary to call the second group 'the control group'. However, the first interpretation is equally valid and, to me, it was the more obvious one.) $\endgroup$ Commented Sep 4, 2020 at 12:40

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.