Difference-Differences Analysis of the Association Between State Same Sex Marriage Policies and Adolescent Suicide Attempts
Julia Raifman, ScD1; Ellen Moscoe et at: JAMA paediatrics, April 2017
WHAT DID THE AUTHORS STUDY?
This study looked at whether or not the implementation of same sex marriage legislation at the state level reduced ADOLESCENT suicide attempts. The authors analysed data from 762,628 students collected from Jan 1,1999 to Dec 31, 2015, during which time 32 states had implemented same-sex marriage laws. They controlled for state, age, race, and year of implementation and measured whether or not respondents reported 1 or more suicide attempts in last 12 months.
WHAT THE STUDY DID SHOW?
According to their analysis, same sex marriage laws reduce suicide attempts by from 8.6% of all participants to 8% ( 7% relative reduction). For students that identified as ‘sexual minority’ the reduction was from 28.5% to 24.5% (14% overall reduction).
WHAT THE STUDY DID NOT SHOW?
This seems convincing evidence that same sex marriage laws reduce suicide rates but a closer examination of the issue reveals a number of problems with drawing such a conclusion.). They can be placed in 3 broad categories:
A. Consistency with other SSM research
The results are contrary to a recent study from Sweden, which has had same sex marriage LONGER than any of the US states and still shows that suicide rates among same sex marriages are 2.8 times as high as heterosexual marriages.
B. Correlation DOES NOT MEAN causation.
Just because YOUTH suicide rates dropped at the same time as SSM laws were introduced does not mean that SSM laws were the cause. The study did not show any CAUSAL RELATIONSHIP between SSM laws and youth suicides. The authors admit this stating that ‘our analysis did not allow us to understand the mechanisms through which implementation of same sex marriage policies reduced adolescent suicide attempts. It did not show any CONVINCING EVIDENCE suggesting that SSM could reduce suicide rates. The authors again state that ‘the analyses on the association between implementation of SSM policies and adolescent suicide attempts among those identifying as sexual minorities should be interpreted with caution given the limited data availability on sexual orientation and the potential for SSM to affect sexual minority identity.
C.Sample and analysis problems.
There are many issues with the analysis used and they are summarised as follow:
- Sample size. It is well appreciated that studies using a small sample size tend to find ‘no differences’ when there REALLY ARE differences. The reverse is true for large samples- you tend to find differences when there REALLY ARE NO differences. The authors used a massive sample size and this means that the risk that ‘false differences’ would be found increases substantially
- The authors report 0.6% drop for ALL TEENS and 4% drop for sexual minority teens. What about the ‘straight’ teens? What happened to their suicide attempts to turn the 4 percentage points for minorities into 0.6 percentage points overall? Did they stay the same? Did they in fact increase? We don’t know because the authors do not tell us.
- The error bars for the 0.6 point drop were 0.1 to 1.2 and those for the sexual minorities, and for the 4 point drop were 1.2 to 6.9. This means that the real decrease could have been either negligible on the one hand and more extreme on the other.
- They did not account for socioeconomic status or other differences between states that may or may not have changed over time and that could have accounted for the differences. Having 50 different states to analyse only compounds this problem even more.
- They lumped ALL the sexual minorities together ie gay/lesbian (2% of population) with ‘not sure’ (3% of population with bisexuals (6% of population). Given that bisexuals outnumber homosexuals 3 to 1, it is reasonable to assume that the bigger reduction in suicide among sexual minorities could have been due to less bisexual suicide attempts. But, why should same sex marriage laws make ANY difference to bisexuals?
- The authors themselves recognise the difficulty in attributing changes in sexual minority suicide attempts to SSM. They state that ‘the analyses on the association between implementation of SSM policies and adolescent suicide attempts among those identifying as sexual minorities should be interpreted with caution given the limited data availability on sexual orientation and the potential for SSM to affect sexual minority identity.
- The study measured suicide attempts in ADOLESCENTS. Same sex marriage legislation is for ADULTS. By what logical mechanism could such laws directly impact adolescents so much that less attempted suicide?
- The reported decreases were the results of a statistical modelling process called ‘linear regression analysis’ but were not observed reductions from the raw data. Basically they come from the formula used to combine all states into one figure and represent ‘weighted averages’ rather than any overall observed change.
What should we take away?
- Assuming the study is accurate, there may be an association between SSM and reduction in youth suicide IN THE UNITED STATES. Association and causation are two different things however and the study provides no evidence of causation.
- For many same sex couples, SSM laws in the US DID MAKE A DIFFERENCE to rights, especially access to quality health care but this is not relevant to Australian situation where EVERYONE has access to health care via Medicare. So, given that the health care systems in the US and Australia are RADICALLY DIFFERENT, we cannot extrapolate these results to the Australian situation. The study is not consistent with Swedish research that shows that men in SSM are 2.8x more likely to suicide than men in heterosexual marriage. Sweden has a nationalised health care system similar to Australia and has had SSM or civil unions since 1995.
- So while the study may be a good introduction to SSM in the US, further research into whether or not there is an association and what the mechanisms could be is warranted. It also has very limited applicability to the Australian debate on SSM.
- Mark Regnerus and William Briggs have written critiques of this study . Regnerus sums up the issues very well as follows:‘SSM signifies many things to its fans and its foes..But is it a robust, long-term panacea for the emotional struggles of teenagers? My concerns about the fragility of this new study, together with the evidence from tolerant Sweden, suggest the answer is ‘unlikely’.
CLICK HERE to view full article