Propensity Scores

The discussion surrounding propensity scores will use this working paper from the CALDER Center. This is the suggested citation for this article:

Benjamin Backes, James Cowan, Dan Goldhaber (2021). What Makes for a “Gifted” Education? Exploring How Participation in Gifted Programs Affects Students' Learning Environments. CALDER Working Paper No. 256-0821.

Why Propensity Scores

Data are from 300 school districts in Washington State.

We study the variety of gifted programs in Washington using three main sources of data. These data include longitudinal student records maintained by the Office of the Superintendent of Public Instruction (OSPI), an annual district report to OSPI on the content of their gifted programs, and an original survey of districts’ gifted program coordinators conducted in early 2019.

This sentence highlights why they decided to use propensity scores:

Gifted programs in Washington generally serve the highest scoring percentiles of the academic achievement distribution.

What could this imply more generally, about participation in gifted populations?

Propensity Score Details

Propensity scores use logistic regression where the outcome is binary.

• 1 = Participates in a gifted program
• 0 = Does not participate in a gifted program.

The goal is to predict the likelihood a student, given the other student attributes in the data would participate in the gifted program. This results for each student, a probability score that represents the likelihood of them participating in a gifted program.

For example, imagine the two students predicted probability of being in a gifted program were:

1. 0.05
2. 0.65

The first means the data characteristics suggest they are only 5% likely to be a in a gifted program, whereas the second student would be 65% likely.

After fitting the model, this is what the paper mentions the did to help limit the sample:

We then trim the sample using the rule-of-thumb method described in Crump et al. (2009), which retains observations with estimated propensity scores between 0.10 and 0.90. The resulting sample includes 18% of all students and 75% of all gifted students in our sample.

Essentially, this limits the sample to remove those with model predicted probabilities to have a very low likelihood of being in a gifted program and those that have a very high probability of being in the gifted program. There are pros/cons to this approach and there are other approaches to create the comparison group.