Post published on Twitter/X on 24 Apr 2012 22:25

Abstract

Recent community-based studies of ASD suggest a prevalence rate of almost 1% (Fombonne, 2009) or perhaps even greater (Kim et al.,2011), a rate which is not reflected in clinical administrative data. This would suggest that there are a very large number of children presenting with ASD behaviours who do not seek or receive a diagnosis, a finding highlighted in the recent paper by Kim et al (2011). The outcome of those “undiagnosed ASD” children is the subject of this intriguing article by Russell et al. (2012). There has been a great deal of emphasis in recent years in reducing the age at which children receive a diagnosis of ASD. Parents of children who later on become diagnosed with ASD usually become concerned with the development of their child around 12–18 months of age but the average age of diagnosis in most developed countries is often much later. This suggests an important lost opportunity in terms of providing those children with essential services to improve long-term outcomes. But that is only true if those community services make a difference. If long term outcome is not modified, then the emphasis on early diagnosis and intervention may be misplaced (an argument recently made by Al-Qabandi et al., 2011). The widely held view that earlier intervention leads to better outcome is based on a small number of tightly controlled randomized control trials. These are so-called “efficacy” trials in which inclusion criteria are strict and targeted to those children most likely to respond, the treatment is delivered in a rigorous fashion, the fidelity of the treatment is monitored closely and outcomes are measured with great precision. A recent systematic review of interventions for children with ASD does provide empirical support for such interventions (Warren et al., 2011). The demonstration that such interventions work “in the real world” though has not really been demonstrated with sufficient rigor. Treatment studies delivered in real world settings are known as “effectiveness trials” and they are few in number in the area of developmental psychopathology in general, let alone in autism. It is in real world settings that it is important to demonstrate that early diagnosis serving as a gateway to intensive early intervention actually makes a difference. The article by Russell and colleagues provides a partial answer but perhaps one that many in the field will find disquieting or will be reluctant to accept. Using secondary data from the ALSPAC trial, Russell et al.compared children who had received a diagnosis of ASD through a multidisciplinary medical assessment to those in the general population who had similar levels of symptoms and behaviors but who had never received a diagnosis. An optimistic hypothesis would be that those who were diagnosed early and presumably entered the intervention system would have a better outcome as adolescents compared to those with similar levels of psychopathology but with no diagnosis. In contrast, Russell et al. demonstrate that those in the diagnosed group had worse outcomes compared to those in the undiagnosed comparison group. Virtually all those who received the diagnosis were receiving some kind of special educational provision, and had much slower change in their developmental trajectory on their Strength and Difficulties Questionnaire than the undiagnosed comparison group. There were few differences with respect to mood and anxiety diagnoses and symptoms. Clearly these findings may have reflected factors at baseline that cause one group of children to seek referral. Indeed, it was true that the diagnosed group had poorer cognitive skills and more behavioral problems at baseline but did not differ in their core difficulties of ASD. But even adjusting for these potential confounding variables did not lead to a reversal of the conclusion. Based on these data, it is very hard to argue that children with ASD receiving a diagnosis as preschoolers have a better outcome as adolescents compared to those not receiving a diagnosis. All clinical research represents a compromise between what is possible and what is desirable. Clearly the most definitive evidence on the effectiveness of early diagnosis and entry into the health care system would be a randomized control trial in which children with ASD symptoms are randomly assigned to early versus late diagnosis. This is clearly unethical, and so, the field must make do with prospective cohort studies where exposure represents early versus no diagnosis and intervention and subjects are followed for a number of years. Longitudinal cohort studies do provide reasonably good evidence on the effectiveness of interventions as long as they are carried out in a methodologically rigorous fashion. The validity of longitudinal studies depends on assembling an inception cohort (i.e. a group of subjects diagnosed at the same early stage of the disease), keeping missing data to a minimum, and minimizing measurement error for both the exposure and the outcome. In addition, the outcome assessments should be conducted blind to the assessment of predictor variables, clinically relevant outcome assessments should be employed that are independent of each other, and careful adjustment of potential confounding variables must be part of the statistical analysis. The design of the ALSPAC study and the analysis carried out by Russell et al. certainly meet many of these criteria. The cohort was assembled at birth and multiple measurement points were available soon after the diagnosis was provided in the one group. The sample size was large, outcome assessments were blind, the outcome variables were relevant and independent, and careful attention was paid to adjusting for confounding variables such as differences in early behavior and cognitive skills. In addition, the fact that the sample was ascertained from the community eliminates potential sources of bias arising from differential ascertainment via clinical services. Careful readers of the article will be concerned no doubt with two important issues: that of missing data and the measurement of ASD. Missing data are always a problem in longitudinal studies. The issue is not so much whether data are missing but whether they are “missing at random”; whether those who are not available at a particular measurement point are different in ways that might influence the outcome from those who do provide complete data. The standard way of dealing with this is to see whether those with missing data differ at baseline from those who remain in the study. But this comparison should really be done at the point just before data go missing rather than at baseline. Two groups may be similar at baseline but as time goes on, one developmental trajectory might diverge from the other and that particular group might go missing at a greater rate than those who stay in. One could imagine in this circumstance that families with a clinically diagnosed child who was doing exceptionally well and had received a lot of services would no longer wish to participate in an intensive research study. Those who were concerned about their children but had difficulty accessing appropriate diagnoses and services might be more inclined to continue to participate and bias the sample towards a worse outcome for the group as a whole. Another issue has to do with the measurement of ASD. The assessment of ASD in the non-diagnosed group is based on a composite measure consisting of a number of different indicators taken from various instruments and tests. The reliability of the composite measure is unknown and its validity was tested against those who had received a clinical diagnosis. However, the key test of validity is the degree of agreement with true cases of ASD in the community who never received a diagnosis. This comparison was not possible so we do not really know the positive predictive value of those who screened positive by the composite measure. Of that group, some will have true ASD, some will have social-communication “differences” but no impairment, and some will have other diagnoses such as ADHD, learning disability, or anxiety disorder. Some will have no psychopathology at all but their ratings will have been influenced by circumstances or informant characteristics that inflate their scores. So, the proportion of subjects within that sample of 142 with “true” undiagnosed ASD is unknown. If a significant number do not have ASD (or ASD without impairment), then the fact that the outcome of this group is little different to those that received a clinical diagnosis is in fact, an encouraging finding. Perhaps the trajectory of those with the clinical diagnosis is similar to the non-impaired adolescents. The authors argue the opposite however; that the majority do have true disabling ASD. An intensive workup of those 142 children with clinician-administered structured and semi-structured interviews and a best-estimate diagnosis incorporating multiple informants would have been extremely useful to clarify the actual status of those with high scores on the composite measure. However, that was beyond the resources of this group and hence the compromise between what is desirable and what is possible. The policy implications of this study need to be articulated with some precision. The authors are careful not to conclude that putting resources into early diagnosis and early treatment is not worth while. The key implication is that truly efficacious interventions need to be implemented in the community with greater fidelity. There is good evidence that early intervention in this age group does make a difference. Generic services geared to children with developmental delay and non-specific behavior problems do not provide the kind of comprehensive and developmentally sensitive intervention that children with ASD require. The study also clearly suggests that greater health service research into the trajectory of children with ASD is required. Why do some families access services and others not, and once they do access services how are clinical decisions made about efficacious treatments? Health services research is a neglected area in our field, yet it is one that deserves greater attention as the policy implications are more immediate. These are questions that now require answers as we contemplate the rising number of children who receive an ASD diagnosis. The study by Russell et al. provides a firm foundation and rationale for asking pointed questions about the effectiveness of community services and whether the investment in continuing to provide non-evidence based interventions is paying off in the long term. This commentary article was invited by the Editors of JCPP; while it has gone through a process of editorial review, it has not been formally peer reviewed. The author has declared that he has no competing or potential conflicts of interest. Peter Szatmari, Offord Centre for Child Studies and Department of Psychiatry and Behavioural Neurosciences, McMaster University and McMaster Children’s Hospital, Hamilton, ON, Canada Email: szatmar@mcmaster.ca