The Importance of Being Inquisitive

The Importance of Being Inquisitive

This post is related to (but not a part of) the Youth Suicide Rise project.

JAMA Research Letter on Pediatric SA/SI ED Encounters

A year ago, April 2019,  JAMA-P published a Research Letter titled Suicidal Attempts and Ideation Among Children and Adolescents in US Emergency Departments, 2007-2015 by Burstein et al. Its authors estimated suicide-related visits to U.S. emergency departments by children between 2007 and 2015. The paper included the following assertion: 

"Notably, 43.1% of SA/SI visits were for children aged 5 to younger than 11 years".

Note: The authors actually meant to write "younger than 12 years" -- and indeed the Table included with the paper had the age category defined as 5 to < 12. This minor error is not the subject of this post. The SA/SI in the paper stands for Suicide Attempts and Suicidal Ideation, respectively.

The 43% assertion was widely and prominently repeated in the media at the time by news outlets including CNN and USA Today.

HuffPost quoted the lead author as follows:

“These numbers are very alarming,” study author Dr. Brett Burstein, an emergency department physician at the Montreal Children’s Hospital of the McGill University Health Centre in Montreal, told HuffPost. “Not only was there doubling over the study period; we also found in this broad, nationally representative sample that there is a high proportion — more than had been previously identified — that are presenting at a very young age group.”

Besides media exposure, the Burstein et al. paper accumulated 28 scientific citations in less than a year.

More recently, an article titled What Happened to American Childhood? in The Atlantic also mentioned the 43% result: "Children’s emergency-room visits for suicide attempts or suicidal ideation rose from 580,000 in 2007 to 1.1 million in 2015; 43 percent of those visits were by children younger than 11."

Doubts

Sonia Livingstone, Professor of Social Psychology at the London School of Economics, read the "43 percent of those visits were by children younger than 11" assertion in The Atlantic and expressed doubts about it on an email list.

David Finkelhor, Director of the Crimes against Children Research Center, seconded Sonia's disbelief.

I had to agree with Sonia and David, since the 43% assertion seems implausible in view of statistics related to suicide.

Children under 12 constitute only 3% of child suicides, according to CDC Fatal Injury data, and  suicides by children under the age of 9 are cumulatively in single digits annually.  The suicide rates for the 5-11 age cohorts range from 1000 to 100 times lower than teen child rates.

Even if we assume that the SA/SI ED visits by young children concerned mainly suicidal ideation, the problem remains since per Burstein et al. the SI without SA visits constituted only one eight of all SA/SI visits.  Furthermore, middle-school YRBS (Youth Risk Behavior Surveillance) results indicate that suicidal ideation is less common among preteens than it is among teens.

Simply put, the numbers do not add up -- there is no credible manner in which the 43% result could be compatible with other statistics related to suicidal conduct. 

The Error

Burstein et al. used National Hospital Ambulatory Medical Care Survey (NHAMCS) samples data for their analysis. I downloaded the relevant data files for 2007-2015 and, contrary to the authors, found nothing indicating so high a share of pediatric SA/SI visits to ED by kids so young.

The difference between my calculations and those of the JAMA authors was severe: I found that children aged 5-11 constituted 7% of child SA/SI ED encounters in the NHAMCS sample -- 3% of SA and 11% of SI visits. The 7% share is far below the 43% share reported by the authors.

The lead author, Dr. Burstein, is currently preoccupied with fighting the pandemic. Despite being extremely busy, he kindly sent me some STATA code fragments.

The STATA code indicates that the authors mistakenly presumed an implied E prefix in the values of the DIAG fields (diagnoses), and consequently counted ICD-9 codes 950-959 (Injury other and unspecified) as if it was ICD-9 codes E950-E959 (Suicide and Intentional Self-injury).

In the end, the Burstein et al. results were more about pediatric head-and-face injuries, which dominate the 959 category, than about suicide.

Sanity Checks and Suicidal Babies

It is not troubling that a researcher makes a mistake parsing a data file.

It is a bit troubling, however, that a mistaken result gets through the peer review process despite having to be at least somewhat surprising to anyone remotely familiar with the topic.

Unexpected results, even if not viewed as outright implausible, should be subject to heightened scrutiny.

In the JAMA case, reviewers could have asked for more details on the ages 5-11 group.

For example, the paper gives separate SA and SI counts for pediatrics patients, but not for the 5-11 subgroup. Had reviewers asked for this, they would likely have received the strange result that the share of this young group is much higher for SA than for SI, since based on the code fragments sent to me, the 95/E95 mistake inflated only the SA share of this young group.

The reviewers could have also asked how frequent are SA/SI ED visits for 5-8 year-old patients, to see if there is any decline with declining age.  This should have led to the next question: does the methodology produce any SA/SI ED visits for 0-4 year-old patients?  

Again, based on the code fragments, the erroneous method would have identified many children aged 0-4 as being suicidal -- indeed many less than a year old (since head-and-face injuries are being mistaken for suicidal attempts).

The existence of such 'suicidal babies' in the results would have revealed beyond doubt a serious problem with the Burstein et al. data analysis.

In general, while looking a step deeper or wider may not catch mistakes where the effects are subtle or limited, it does tend to catch mistakes where the effects are absurd or massive.  Such 'sanity checks' are necessary, especially when an unexpected result is being reported.

The Importance of Being Inquisitive

Only Sonia Livingstone noticed an implausibility that so many others have missed for a year.

The 43% assertion coverage in news media must have been seen by thousands of professionals familiar with child suicide issues, hundreds of child psychologists, and dozens of experts on suicide.

The 28 current citations of the Burstein et al. letter are in articles written by a cumulative total of over 100 authors.

And yet no one before Sonia noticed the dubious nature of the 43% assertion -- or if they did, then failed to take steps toward its correction.

The reality is that overly dubious or outright implausible results may survive indefinitely as established facts despite wide media and academia exposure. That is the most troubling aspect of this story.

Until the scientific process is improved to better minimize such occurrences, we will have to depend on the acuity of people like Sonia.

Notes:

It seems the authors also over-counted suicidal ideation, since their STATA code indicates checking only the first 3 characters instead of all 5 in the V6284 code for SI.

One consequence of the errors is that the SA/SI visits estimate for the ages 5-11 group was incorrect by a factor of 30: their estimate was 3.16 millions instead of 100 thousand per my calculation.  Estimates for older children were also inflated, though to a lesser degree.

Furthermore, the authors reported more SA/SI visits by boys than by girls, which was surprising, given that girls are about twice as likely to report suicide attempts as boys.  In my results, girls are clear majorities: roughly 65% for SA and 60% for SI.

I did try to reproduce the erroneous analysis indicated by the STATA code fragments, and the results were nearly identical to those reported by the authors -- thus confirming the STATA coding was indeed the problem.

Warning: my own calculations were not checked by anyone else -- and certainly not peer reviewed -- so please consider them tentative.

See here for the output of the script used to analyse the data (script itself also included).

I will update this post with info regarding correction or withdrawal of the JAMA-P Research Letter once the authors have the time to address the matter (likely after the pandemic crisis ends).

(The reason I mention possible withdrawal is that lower visit counts for SA/SI children translate into high margins of error, especially for time trend analysis, since these calculations are made from small samples -- it is therefore unclear if the authors will be able to salvage enough statistically significant results.)

Suicide and Male Violence: Plausibility Model

Suicide and Male Violence: Plausibility Model

Note: this is part of the Youth Suicide Rise project.

Note: the context is presumed to be ages 15-19 from 1981 to 2018 -- so below 'male suicide' means male youth suicide 1981-2018, unless stated otherwise. This post is a follow-up to the the Suicide and Homicide Rates and Homicide and M/F Suicide Ratio posts.


Factor V

Let us postulate a 'violence' factor V that is responsible for male homicide rates.

This imaginary factor V is really an encapsulation of a family of causes behind homicide, which may include innate aggressiveness, social dysfunction, poverty, parental abuse, lead poisoning, access to guns, and so on.


Factor V and Causation

We measure factor V by one of its results, homicide rates, but we do not identify V with homicides.

The reason we differentiate between factor V and homicides is to keep in mind that a relationship between suicide and homicide could be due to underlying causes (factor V), rather than due to the influence of one on the other.

Consider these two possibilities:

A) When homicide increases, so does suicide because boys become desensitized to death and because of the trauma of witnessing homicides, or losing a relative or friend to homicide, or being victimized by violence.

B) When homicide increases, it is due to causes (factor V) that also increase suicide; for example, innate aggressiveness, social dysfunction, poverty, parental abuse, lead poisoning, access to guns, and so on.

In symbolic form, these two possibilities are A) V --> Hom --> Sui and B) V --> Sui + Hom.

If type A causation was the only possibility, we could identify V with Homicide, but in reality the type B causation mechanism is likely to be true as well.


The Goal


We will attempt to show that even when measurement of the underlying causes is simplified down to a single yearly value (homicide rates), it can help predict how male suicide rates differ from female suicide rates.

The influence of factor V on male suicide should be strong enough to explain much of the male-to-female suicide ratio fluctuations in the past (1981-2018).

At the same time, this influence on male suicide would have to be weak enough to be over-shadowed by other forces at times.

For example, between 2007 and 2014, male youth homicide rates decreased greatly (-35%)  but male suicide rates increased considerably (+21%).

Is existence of such a factor V compatible with our suicide data?


Minimal Plausibility Model

Let us try to create a model of suicide rates that is as simple as possible while fulfilling the above requirements.

So let us postulate that there is only one other factor besides V, and that this factor Z affects female and male suicide rates proportionally the same.

Therefore when factor V is stable, male and female suicide rates should move in tandem, doubling or halving according to factor Z only.  On the other hand, fluctuations in the male-to-female suicide ratio should be entirely due to fluctuations in factor V.

Let us measure V by male homicide rates (MH) and Z by female suicide rates (FS), and require that male suicide (MS) is predicted to be zero if both V and Z are zero:

Predicted_MS = k * FS + l * MH

This linear regression is optimized when k = 2.6 and l = 0.29.

The question now is this: how well can this linear model improve predictions of male suicide and the male-to-female suicide ratio over using a linear model based ONLY on female suicide rate?


Female Suicide Rates Model of Suicide

Female suicide rates alone do not suffice for very good modelling of male suicide rates: the linear correlation has r = 0.56, with an average prediction error of 1.6 points (11% off the mean rate).

Furthermore, such a linear model has male suicide only halved if female suicide is null.  Forcing the intercept to be zero -- that is requiring a more realistic model if the female rates are to truly explain the male rates -- leads to a much greater prediction error of 2.1 points (14%).


Male Violence plus Female Suicide Rates Model of Suicide


Now let us return to the Predicted_MS = 2.6 * FS + 0.29 * MH model.

The predicted rates versus the actual rates:

We see that this is a considerable improvement:  the average error is 1 point (instead of 2.1 points).

We also have a decent prediction of male suicide, since the mean error is only 7% off the average suicide rate(14.4).


Similarly with the male-to-female suicide ratio:

Here the mean error is 0.3 and -- quite good given that the M/F suicide ratio is a bit volatile.


Significance

We have previously noticed that the female share of child and youth suicides has been increasing during much of The Rise (especially 2010-2015), but it also turns out that this trend has been present long before The Rise.

We have now also shown that homicide rates can help predict much of the year-to-year fluctuations in the M/F suicide ratio since at least 1981.

This adds yet another reason not to jump to the conclusion that the causes primarily responsible for The Rise must be affecting girls much more than boys.


Violence and Suicide and Gender

As to the underlying relationship between violence and suicide and gender, we can only speculate.

It may be that high levels of violent crime desensitize youth to death; it may also be that inclination to violence is equally a risk factor in suicide as it is in homicide.  Such inclination may be due to not only social developments, but also due to biological factors such as lead poisoning.

Finally, since severe violence --  especially homicide -- is behavior an order of magnitude more common among males, it would be no surprise that fluctuations in levels of violence would not be linked to female suicide anywhere close to as much as to male suicide.


Discussion

We must keep in mind that this was a plausibility demonstration, not a probability analysis.  We did not estimate how likely it is that homicide is linked to suicide, nor did we calculate some related measure, such as how unlikely it would be to have our data if no link was present.

Notice also that we demonstrated plausibility despite simplifications, not due to simplifications.   Our modelling criterion has been actually made harder to achieve by decisions such as limiting the factors to two, stipulating that factor V affects suicide only among males, or forcing the intercepts to be zero (e.g. requiring MS to be 0 if FS and MH are both 0). What we did not simplify was the test of the model: how well we can approximate fluctuations of male suicide and the male-to-female suicide ratio by using a male violence indicator on top of knowledge of female suicide rates.


Notes:

Our model used only concurrent homicide rates, so trauma several years old was not included, but this too could be additional factor.  Note, however, that trauma due to homicide is likely to be psychologically different from trauma due to 'abandonment' deaths like parental suicide or drugs overdose.


Technical notes:

It is important to examine the models themselves, not just look at some number like r or R squared, because even models with a high r can predict nonsense.  For example, an eagles population growth data can have a linear model with a high r that predicts negative population in the recent past -- see this lecture.

Homicide and M/F Suicide Ratio

Homicide and M/F Suicide Ratio

Note: this is part of the Youth Suicide Rise project.

We previously noticed that youth male homicide fatality rates are moderately correlated to youth suicide fatality rates, but that this relationship is not robust and does not hold for females.

Let us now look at the relationship of youth male homicide fatality rates with youth male to female suicide ratios:

We see that the linear correlation is moderate, but might be fairly robust:

The Crack Epidemic

It is important to keep in mind that the crack epidemic of the late 1980's led to a massive rise in youth homicides in the early 1990's as much of the drug trade was delegated to youth gangs.

Such a huge anomaly might disrupt a linear relationship, and in the latter period of 1999-2018 the linear correlation is indeed considerably higher (R squared close to 0.5).  A further examination of the 1981-1998 period might also reveal stronger relationships once race and urbanity are taken into account.

1999-2018

Let us now look closer at the 1999-2018 period unaffected by the crack epidemic:

We have moderate correlation that does not seem to depend on any short period of time.

Let us now also look at the correlation with recent rates:

We see that the correlation increased considerably, suggesting that past trauma associated with male homicide tends to increase somewhat the M/F suicide ratio several years into the future.

Discussion

The simplest explanation would be that periods of rising male youth homicide increase male youth suicide -- but not female youth suicide --  and therefore increase the M/F youth suicide ratio.

It turns out, however, that there is a small negative correlation between male youth homicide and suicide 1999-2018:

We will examine this seeming paradox next.

Suicide and Homicide Rates

Suicide and Homicide Rates

Note: this is part of the Youth Suicide Rise project.

Is there a link between violence and suicide?

Let us first ask a more precise question: is there a correlation between suicide and homicide among youth?

Male Youth

Let us first look at males (age 15-19):

While there might be a correlation, it is not a very robust one, as there are long periods -- such as 2007 to 2014 -- when homicide and suicide trends were opposite.

If we turn to linear correlation, we do find some evidence of a link:

It is clear from the graphs, however, that this moderate correlation would largely disappear if we remove the 7 or so years in the 1990's period of high homicide and suicide (1992-1996).

Female Youth

The correlation between homicide and suicide among female youth is minuscule and negative:

Homicide and Gender

It is important to keep in mind here that females get killed primarily by males (typically in a domestic setting), so rising violence against women need not imply similarly rising violent conduct among these women.

Among males the situation is different: the vast majority of males are killed by other males, often outside their home; furthermore youth males tend to be killed by peers.  Therefore homicide rates among male youth are presumably a very good indicator of violent conduct among male youth.

Female teens are also much less likely to be victims of a homicide than male teens: their chances of being killed are nearly one third those of male teens.

Discussion

None of the above results exclude the possibility of a significant link between male homicide and suicide; it could simply be that its moderate strength gets overpowered by stronger trends.  For example, decreasing homicide rates might slower a rising suicide trend.

The fact that there is no correlation among females but there is a moderate correlation among males suggests that there is indeed some link, perhaps due to underlying factors (e.g. aggressiveness and impulsiveness).

We will next look at the relationship between homicide rates and the male/female suicide ratio.

Note: I do not go into statistical details such as p-values because we are at this point merely investigating plausibility; once I transfer data into the statistical language R environment, I will revisit topics like these in more statistical depth.

The Rise and Child Suicide Stigma

The Rise and Child Suicide Stigma

Note: this is part of the Youth Suicide Rise project.

Decreasing stigma of suicide could lead to increasing likelihood of fatal injuries being classified as suicides, especially when the deceased are children (due to the greater role of families in providing information and influencing coroners).

CDC Fatal Injury data classify Intent as Homicide/Suicide/Unintentional/Undetermined.  It is not specified which injuries classified as Unintentional/Undetermined were self-inflicted.

We will presume that nearly all child suffocation deaths -- unlike firearm deaths -- are self-inflicted. This presumption is supported by data: the ratio of suicide to homicide is more than 10:1 for child suffocation (but less than 1:1 for child firearm deaths).

Let us examine suffocation deaths of age groups 10-14, 15-19, and 30-59; we will sum Unintentional and Undetermined Intent suffocation as Accidents and sum Suicides and Accidents as Self-Inflicted deaths.

Share of Suffocation Deaths

Let us now look at the share of Accidents versus Suicides in all Self-Inflicted suffocation deaths:

As we can see, the share of accidents has decreased from 38% in 2007 to 13% in 2015 for tween kids (10-14), a very rapid decline (the 3-year average decreased from 35% in 2005-07 to 15% in 2016-18).

Age group 15-19 had a decrease from 9% to 4%, with the 3-year average decrease from 10% to 5%.

Middle-aged adults had a decrease from 20% to 13%, with the 3-year average decrease from 21% to 14%.

Question: why is the accidents share of non-homicide suffocation so much smaller for youth (age 15-19) than it is for middle-aged adults (age 30-59)?


Potential Impact

Let us for a moment presume that these declines are mainly caused by a decreasing stigma attached to suicide, and that they apply similarly to other suicide methods.

Let us also presume that declines in stigma lead to more accurate suicide classifications; from this we would conclude that in 2005-07 there were actually 31% more suicides in age group 10-14 and only 6% more actual suicides in age group 15-19.

Without adjusting for stigma, suicide increased from 2005-07 to 2016-18 by 136% and 56% for age groups 10-14 and 15-19, respectively.

With adjustment for stigma, suicide increased from 2005-07 to 2016-18 by 80% and 47% for age groups 10-14 and 15-19, respectively.

Despite stigma having a large potential impact, the rise of suicide within the younger (tween) age group is still much greater than in the older teen group: instead of being three times as fast, it is twice as fast.


Youths and Adults

The potential impact of declining stigma, at least within this data analysis, would actually increase the difference between suicide rise among older teens (15-19) and middle-aged adults (30-59), since the adult counts in 2005-07 would have to be increase by a third (7%/21%).

Limitations

It is absolutely crucial to keep in mind that we did not link the declining share of accidents in self-inflicted suffocation to decrease in suicide stigma -- that is so far merely plausible speculation.

An alternative explanation could be that there has been a real decline in actual accidents, perhaps because parents are getting more protective regarding their children (compare how kids could roam outside alone in the past without supervision -- I suspect anyone over 30 has noticed huge changes in such areas of parenting).

Differentiating between these two plausible explanations is complicated and likely impossible with the data available in the CDC WISQAR tool.

Furthermore, the stigma explanation would impact middle-aged adults more than older teens, at least until more detailed data is available.


Conclusion:

If the decreasing share of 'accidental' suicides is caused by decreasing stigma of suicide, then 'true' suicide increases among younger teens would be less extreme but still considerably higher than those of older teens; furthermore, youths would still have far higher proportional increases in suicide than adults.

Thus decreasing suicide stigma is unlikely to explain why youth have a far higher suicide rise than adults or why younger teens have higher increases than older teens, but it remains a plausible explanation (subject to further analysis) for significantly reducing the extreme rises in younger child suicide rates.


Note:

Unintentional suffocation deaths are not decreasing for the age group 0-9 (e.g. 1150 in 2007, 1252 in 2017); however, it turns out these counts are dominated by infant deaths. By age 9 such deaths are so rare (in single digits per year) that any trend is dubious.

A wider analysis, such as looking at firearms data or overall accidents data, may illuminate the issue a bit more, but I am skeptical it can go far without more detailed data and without the insight of practitioners.

Technical note:

The 31% calculation: 20/65 since we presume the 65% share should have been 20 percentage points (not percents) higher.  Similarly for age 15-19 (5/90) and for adults (7/21).