Incorporating Child Development within Child Development Research

Central to any research in the area of child development is an appreciation that children’s capabilities and behavior change over time as a result of developmental processes, independent of other factors or interventions. Measures of skills or capabilities therefore need to be adjusted and compared with norms for different ages/stages, introducing analytical concerns for cross-sectional studies involving children of different ages or developmental stages. Measurements of the effect of interventions provided over time may also be compromised by analytical concerns inherent in measuring the same domain at different developmental stages, which may necessitate the use of different age/stage-appropriate instruments or, at the very least, correction for age/stage. In addition, measurement of children’s abilities may be confounded by the child’s developmental capacity to understand instructions and communicate comprehension. For example, young children have been described as having difficulty appreciating the perspective of someone else. It is possible that such difficulty may result, at least in part, from limitations in their ability to comprehend the task requested, their language ability to communicate their understanding, or the researcher’s ability to communicate the task required. Research on young children’s understanding of the concepts of human immunodeficiency virus (HIV) and acquired immunodeficiency syndrome (AIDS) initially suggested that young children’s understanding of core concepts of illness was significantly limited developmentally, which seriously constrained their capacity to benefit from educational interventions; however, subsequent research demonstrated that a developmentally based educational intervention could result in dramatic gains in young children’s conceptual understanding in this area.³ In other words, what appeared at first to be a limitation in children’s ability to learn was subsequently found to represent limitations in adults’ understanding of how to teach effectively and/or in researchers’ ability to measure validly children’s underlying comprehension.

Inclusion of Children with Disabilities in Research Protocols

A child with a motor disability may engage in less play activities with peers during a play observation directly because of impairments in motor function. Alternatively, the child’s motor disability may have resulted in fewer opportunities for social interaction in the past, which in turn resulted in less well-developed peer interactions that are subsequently manifested by decreased peer engagement in peer play activities during the observation period. It therefore becomes important to select measures of function (in this example, a measure of peer interaction) that are not confounded by the child’s underlying disability. More subtle influences could be anticipated for the effect of sociocultural or personality factors, which would be difficult to identify and confirm.

The Biopsychosocial Model

The biopsychosocial model emphasizes the complementary influences of genetic predisposition, environmental factors, and experience on development and behavior. DBP research encompasses basic science and social science research methods, allowing the demonstration of causal mechanisms by which environmental and experiential factors alter basic biological processes. For example, to study the influence of early stressful experience on later hyperactivity of neurons that release corticotropin-releasing factor as a cause of adult anxiety or depressive disorders, both stressful experiences and neuronal function must be measured, and the potential confounders of both must be understood.

State versus Trait Measures

In accordance with the biopsychosocial model, many behavioral constructs represent both intrinsic traits of an individual and transient states influenced by recent circumstances and environmental factors. This poses measurement challenges. For example, changes in repeat measures of anxiety may reflect concerns with test-retest reliability or variations in state anxiety over time.

Instrument Development

Many of the outcomes of interest to the field of DBP lack well-validated and standardized measures. Researchers in DBP are therefore often faced with the additional challenge of developing and standardizing new measures and demonstrating their validity. Competing theoretical constructs pose a challenge for achieving construct validity; the absence of well-accepted “gold standards” and objective biological or physical endpoints (e.g., for a measure of coping with the death of a parent) is a major challenge for demonstrating criterion-related concurrent or predictive validity. Researchers must often select measures that were standardized for populations that differ from the current study population in critical ways (e.g., language, culture, social class) and require restandardization. This is highly relevant to testing of certain variables and domains that are central to DBP research, such as children’s intelligence and language. Because the ecological perspective underlies much of DBP research, many instruments measure domains that are intrinsically sensitive to sociocultural influences.

Survey Development

Many of the variables of interest in DBP research relate to perceptions, feelings, or other constructs that rely on self-report. Surveys are often used to assess outcome measures. Researchers need to be able to construct effective and valid questionnaires. Methodological issues relate to sentence or instrument construction, such as question structure (e.g., open-ended vs. close-ended questions), formatting, or wording (e.g., clarity, neutrality, nonleading, reading level, culturally appropriate). Other issues include the construction of scales and summary scores and validation of the instrument, as well as variation introduced by the means of administration of the survey (e.g., self-administered, administered by interviewer, computer, or Web-based administration). Comparable issues of construction and administration are evident for structured and semistructured interviews. Knowing what to ask is the first step; knowing how to ask (and score the responses) is particularly challenging and relevant for DBP research.

Reliance on Parent Report

Young age and cognitive delays, when present, may preclude children’s ability to provide independent reports of experiences, perceptions, or feelings. Self-report by children may also be constrained by their reluctance to disclose sensitive information. In these settings, parental report may be an acceptable proxy. Reliance on parental report often leads to an underestimate of children’s exposure (such as that observed with parental report regarding young children’s exposure to violence) and feelings and internalizing symptoms (e.g., depression, anxiety, fears, pain). The construction of more developmentally appropriate instruments for self-reports of young children (such as utilizing pictures or storytelling) allows even younger children to provide self-reports. In addition, parental reports of children’s symptoms are influenced by the parents’ own perceptions or state (e.g., parents who are themselves depressed may either attribute similar feelings to their children or be less sensitive to or aware of their children’s difficulties). These limitations highlight the value of triangulation of data: the use of multiple instruments to provide reports from different reporters (e.g., child, parent, and teacher) and to compare results across these different perspectives. The validity of individual reports is thereby strengthened through the congruence of data from multiple sources.

Qualitative Methods

Qualitative research methods are most appropriate in situations in which little is known about a phenomenon or when attempts are being made to generate new theories or revise preexisting theories. Qualitative research is inductive rather than deductive and is used to describe phenomena in detail, without answering questions of causality or demonstrating clear relationships among variables. Researchers in DBP should be familiar with common ethnographic methods, such as participant observation (useful for studying interactions and behavior), ethnographic interviewing (useful for studying personal experiences and perspectives), and focus groups (involving moderated discussion to glean information about a specific area of interest relatively rapidly). In comparison with quantitative research, qualitative methods entail different sampling procedures (e.g., purposive rather than random or consecutive sampling; “snow-balling,” which involves identifying cases with connections to other cases), different sample size requirements (e.g., the researcher may sample and analyze in an iterative manner until data saturation occurs, so that no new themes or hypotheses are generated on subsequent analysis), different data management and analytic techniques (e.g., reduction of data to key themes and ideas, which are then coded and organized into domains that yield tentative impressions and hypotheses, which serve as the basis of the next set of data collection, continuing until data saturation occurs and final concepts are generated), and different conventions for writing up and presenting data and analyses. The strength of the findings is maximized through triangulation of data, investigator (e.g., use of researchers from different disciplines and perspectives or several researchers to independently code the same data), theory (i.e., use of multiple perspectives), or method (e.g., use of focus groups and individual interviews to obtain complementary data).

Intervention Fidelity and Treatment Dose

Interventions are often delivered in naturalistic and group settings by individuals who are not part of the research team, such as teachers, parents, and home visitors. Although this allows for the testing of interventions that are much more likely to generalize to the general population, distortions in the delivery of the intervention may occur. Research requires measures of the intervention fidelity (i.e., the degree to which the intervention is delivered in the manner intended by the researcher) and treatment dose (the extent to which the subject participates in or receives the full intervention). A study of a school-based intervention delivered by regular classroom teachers needs not only a strong method for teacher training and monitoring but also explicit measures of how the teachers delivered the intervention and the degree to which students attended and/or received the full intervention. Such monitoring may include a mix of quantitative measures (e.g., curriculum checklists, student attendance records, self-reports of teacher satisfaction with the intervention) and qualitative assessments (e.g., ethnographic observations of classrooms while lessons are being taught, focus groups of teachers, or individual interviews). Other measures (i.e., triangulation) may be used to confirm teacher reports of intervention fidelity or treatment dose, such as asking students to complete a questionnaire about simple concepts or facts from the intervention, to test whether children were exposed to the relevant lessons.

Efficacy verses Effectiveness

Clinical trials involve optimal control in the selection of subjects, and the intervention and the dependent measures are applied so that the most rigorous application and observation of the results can be obtained. The intent is to determine the efficacy of the interventions. However, the evidence may be limited to a narrowly defined group of children, such as those with ADHD without comorbid conditions, or to an application of the intervention that is not practical to use in a real-world setting, such as one requiring too much time or training than is practical for most practicing clinicians. Determining effectiveness involves assessing the effects of an intervention in actual settings. Balancing the need for what is practical and what is rigorous is a creative challenge. Of importance is trying to maintain the elements of randomization, comparative groups, and independent measures. Effectiveness is one of the elements studied in what is now categorized as health services research.

Clustering and Nested Analyses

Interventions delivered in group settings may introduce variability caused by clustering: children are members of classes, which are parts of schools, which are parts of school districts, and so forth. In this manner, the variability in children’s individual responses on outcome measures may be explained in part by variability in some of the group-level variables (e.g., the variability in teacher, school climate, school district practice). Nested analyses are then necessary to attempt to estimate the percentage of the variance attributed to the clustering, which requires larger sample sizes and the collection of group-level measures.

Placebo Effects

The use of a placebo in psychosocial interventions is less clear than in medication trials. Some studies utilize alternative interventions (e.g., the control group for a study of hypnosis to decrease pain in children with a chronic condition might receive education about an unrelated component of their condition). The limited empirical evidence on how “counseling” or “therapy” works contributes to difficulties in controlling for all components that may be involved in a therapeutic or placebo effect. Furthermore, if individuals report less pain or subjective discomfort after the use of an interpersonal placebo intervention, such as the psychoeducational component of the example just described, should the placebo then be considered as a possibly effective, alternative form of treatment (e.g., perhaps informal group support occurred as a result of the psychoeducational sessions that decreased parental anxiety, that in turn decreased the child’s anxiety and perceptions of discomfort or pain)?

Advocacy

Intrinsic to the field of DBP is an explicit value placed on advocacy. Researchers have to be ever mindful of the potential for bias to influence study design or interpretation of results. Ethical concerns may also become evident if interventions valued by members of the professional or lay community (e.g., home visitation, early intervention, supportive psychotherapy, drug prevention curricula, infant daycare) are not supported by the findings of a research project or when “interventions” thought to be damaging to children (e.g., gang enrollment, parental divorce, employment during childhood) yield unexpected positive findings (e.g., increased self-esteem, decreased anxiety, increased problem-solving skills) or neutral results. The inclination to withhold publication or reporting of such “negative” studies may be quite strong, because of the desire to protect the interests of children and families. Researchers must engage in some introspection with regard to their biases, develop methods that limit the likely effects of such biases, and maintain the integrity to report findings, even if the findings run counter to the researchers’ presumptions.

Socially Sensitive Topics

Research in DBP often involves exploring socially sensitive topics, such as bias against individuals with disabilities, stigma related to mental illness, and sexuality among individuals with developmental disabilities. The extreme sensitivity necessary to engage prospective study subjects and their communities may undermine the research team’s willingness to conduct the open and frank discussions necessary to ensure informed consent. Presenting the results of such studies in a public forum is often met with considerable controversy that makes it particularly difficult to present objective findings in an atmosphere of open disclosure.

Prevention of Disclosure of Confidential Information

Inquiry into highly confidential information, such as mental health status or criminal or antisocial beliefs and behaviors, may heighten concerns about the accidental disclosure of highly confidential, personal information. Because the focus of these studies is often to explore the reasons for such outcomes or behaviors, subjects and other informants are typically asked a wide range of highly personal questions, not only about their behaviors but also about the possible reasons for these behaviors. This further increases the amount of highly confidential information obtained for research purposes and results in the need for extreme care to prevent unintended disclosure.

Withholding Explicit Intent of Studies

Informed consent protocols may need to include some element of withholding the explicit purpose of the research study, in order to permit a relatively unbiased collection of data. For example, if researchers are studying reporting bias and aim to compare the validity of self-reports of smoking in the presence or absence of concurrent measurement of salivary nicotine metabolites, it would be very difficult to obtain valid results after informing subjects of the study’s intent. Clarifying the distinction between deception and providing accurate but less than complete information to avoid overly biasing subjects (e.g., in this example, stating that the purpose of the study is simply to measure individuals’ tobacco use) is often difficult but crucial.

The Ability to Provide Consent and Assent

Research in DBP often involves participation of research subjects who have cognitive/developmental limitations that affect their ability to provide consent or assent. Studies may also involve the collection of data from individuals who report on characteristics of the family, community, or other groups. In such situations, ethical concerns may be raised about the ability of the individual (especially when that individual is a minor) to provide potentially sensitive information about others.⁴

The Use of a Placebo

Despite limited evidence for the efficacy of specific psychological or behavioral interventions, ethical concerns may be raised about random assignment of subjects to a nonintervention control group when such interventions are the community norm. For example, despite limited data to support the efficacy of supportive counseling after a traumatic experience, communities are unlikely to accept “sham” or “placebo” mental health interventions to document the efficacy and control for potential biases. The use of a placebo in medication trials is more commonly accepted than the use of a placebo in psychological or behavioral intervention trials.

Training in Quasi-experimental and Observational (Epidemiological) Research Designs

The majority of research in DBP does not involve double-blind, randomized, controlled trials (RCTs). Double-blind RCTs are conducted in studies of psychopharmacological therapy for such conditions as ADHD and autistic spectrum disorders, and single-blind RCTs are conducted for behavioral and educational interventions for the treatment or the prevention of developmental and behavioral disorders. Examples of the latter include the High/Scope Perry preschool study⁵ and the Hawaii home visiting study.⁶ However, many important nonpharmacological interventions in DBP are difficult to randomize. New educational interventions in the school setting, for example, are likely to be offered to whole classes or schools rather than to randomly selected students, with other classes in the same school or other schools in the same district acting as controls after being matched on a variety of characteristics. These “quasi-experimental” designs play an important role in DBP research.

Research training should, therefore, include education in quasi-experimental designs. A quasi-experimental design is similar to an experimental design except that it lacks the important step of randomization.⁷ The most common type of quasi-experimental design involves the use of nonequivalent matching groups. One cohort of children receives an intervention, whereas another matched cohort acts as a control or receives an alternative intervention. Quasi-experimental designs such as this, although frequently the only possible design for some nonpharmacological interventions, suffer from the threat to their validity of selection bias. Education concerning the identification of selection bias and methods to reduce its effect (such as matching, sample stratification, and adjusting for potential confounders) is important for the developmental-behavioral researcher.⁸

Furthermore, researchers in DBP frequently seek to study the effects of harmful and protective factors on childhood outcomes in normative or at-risk populations or to elucidate underlying constructs associated with or causing various outcomes. The study of naturally occurring constructs or of potentially harmful factors leads to choosing observational designs rather than experimental designs.⁸

Observational designs in DBP research are usually single- or double-cohort studies. Single-cohort studies usually involve monitoring a group of children and looking for factors that are predictive of outcomes. For example, the researcher may want to monitor a group of children with intrauterine exposure to cocaine, or with very low birth weight, and determine whether breastfeeding or maternal expectations lead to improved developmental outcomes. Double-cohort studies usually involve a comparison group. For example, a study may compare the incidence of behavior problems in cocaine-exposed infants when they reach preschool with that in a matched group of children. Single-cohort studies may lead to spurious results because of confounding factors. Double-cohort studies, like quasi-experimental designs, may be threatened by selection bias. The DBP researcher should receive in-depth training in these observational study designs and in methods to adjust for confounding and to minimize selection bias.

In addition, the researcher should receive training to understand concerns about inferring causality from results of observational studies and to devise strategies to enhance causal inference (e.g., inclusion and exclusion criteria, matching, and stratification). The principles of strength of evidence for causality—temporality, effect size, dose-response relationships, biological plausibility, reversibility, specificity of results, and consistency of results across studies—should be clearly understood.

Training in Qualitative Methods

Qualitative research designs play an important role in developmental-behavioral research. For example, in one study, investigators sought to understand Latina mothers’ cognition and attitudes concerning stimulant medication for ADHD and how these factors might determine adherence to medication regimens and resistance to starting drug therapy.⁹ Qualitative methods are the best approach to this type of research question, with which investigators seek to understand the perspectives of persons or groups and to develop and revise research hypotheses. Therefore, a strong foundation in qualitative methods is important for DBP researchers. Training should include providing an understanding of the types of research questions that qualitative methods are best suited to answer, skills in the use of common ethnographic methods, facility with methods used for data coding and data reduction, and familiarity with methods used to ensure trustworthiness of qualitative research results.^10–13

Training in the Development and Validation of Testing Instruments and Scales

Many of the outcomes and constructs of DBP lack well-standardized or validated measures. Furthermore, even well-validated and reliable measures developed for a general population may not be appropriate for use in a special population. Therefore, the development and validation of such measures are frequent subjects of a research study or are common components of a larger research project. For example, in one study, investigators sought to measure self-efficacy and expectations for self-management among adolescents with chronic disease (diabetes) and to determine the effects of these factors on disease management outcomes.¹⁴ However, there existed no scale for measuring self-efficacy or measuring expectations for self-management of diabetes. Therefore, the researchers had to develop their own scales and test the reliability and validity of those scales before answering the underlying research question. In another study, researchers wanted to determine whether the Pediatric Symptom Checklist,¹⁵ a commonly used screen for behavior problems in a general population, was valid for use in a chronically ill population.¹⁶ In this case, the researchers sought to test the construct validity of a well-validated scale on a new population.

Therefore, research training should include skills in test development, including testing for reliability (with measures such as test-retest reliability and interrater reliability), internal consistency, and validity. The researcher should be familiar with the different types of validity, including content and sampling validity, construct validity, and criterion-related validity, and be able to use methods that assess them.¹⁷

Training in the Use of Large Secondary Datasets

Research publications increasingly reflect the efforts of researchers to mine large secondary datasets, including federally funded national databases, administrative databases, and electronic medical records, for information related to delivery of care and outcomes in DBP. This type of research has been especially fruitful in the study of ADHD. In one published study, the researchers used the National Ambulatory Medical Care Survey and the National Hospital Ambulatory Medical Care Survey to study regional and ethnic differences in the diagnosis of ADHD and receipt of psychotropic medications for that diagnosis.¹⁸ In another, investigators used health maintenance organization (HMO) data to compare total health care costs for children with and without ADHD.¹⁹ Researchers at the Mayo Clinic have used their computerized medical records database, which has information on 95% of the population in Olmstead County, Minnesota, from 1977 to the present. They have studied ADHD, autism, psychostimulant treatment, and learning disabilities, among other topics.^20–22 Finally, Rappley and colleagues used Medicaid database information to sound an alarm about apparent excessive prescription of psychotropic medication for preschool children.²³

These studies demonstrate that large datasets contain potential answers to clinical, epidemiological, policy, and health finance questions important to DBP. Research training should provide the knowledge and skills to evaluate these large datasets and to use appropriate sampling and statistical methods in subsequent data analysis. Training should also focus on the types of information collected by cross-sectional and longitudinal national survey datasets, as well as the advantages and disadvantages of using national survey data to answer research questions. Familiarity with health plan administrative datasets, electronic medical records, and disease registries may also be useful for the researcher interested in epidemiological questions concerning disease rates and distribution, utilization of resources, and resulting costs.

Training in Multivariate Statistical Analyses

As a consequence of the frequent use of quasi-experimental and observational study designs in DBP research, training in statistical methods that adjust for confounding factors and that determine unique and independent contributions of hypothesized factors to patient outcomes is essential. Research training should focus on such commonly used statistical techniques as multiple linear regression,²⁴ analysis of covariance (ANCOVA), and logistic regression.²⁵ Although training goals may vary in the level of expertise from “educated consumer” of statistical services to independent producer of statistical analyses, the DBP researcher should be skilled in the interpretation of the results of these analyses, knowledgeable about the different types of statistical models, and able to recognize the pitfalls and limitations of these techniques.

Measurement of psychological or behavioral characteristics with scales frequently necessitates an analysis of data for the underlying factors or constructs embodied by the measured variables. Factor analysis is employed to reduce measured variables to a smaller number of underlying “latent” variables. Factor analysis is frequently used to explore and confirm the construct validity of a new scale or a previously validated scale for use in a new population. In addition, factor scores may be used to substitute for variables in other statistical analyses. Because research in DBP often involves the development of new measures for abstract outcomes, factor analysis is a useful statistical method for DBP researchers. For example, factor analysis was used to develop new scales in the previously noted study¹⁴ in which investigators sought to compare self-efficacy and expectations for diabetic management to diabetes self-management outcomes. They then used the scores on these scales, including a separate score for each of two identified factors, in multiple regression analyses of the relationships between these constructs. In another previously cited study,¹⁶ researchers used factor analysis to compare items on the Pediatric Symptom Checklist¹⁵ in a chronically ill population with those in a general population.

Therefore, an understanding of the role of factor analysis in the development and validation of testing instruments and scales is important for the DBP researcher. Familiarity with methods of extraction of factors from the collected data, procedures for keeping and discarding factors, and other aspects of factor analysis are an important part of research training.^26–28

Training in Neurobiological and Genetic Science

The frequently used multidisciplinary approach to DBP, as well as the importance of the biopsychosocial model, requires the DBP researcher to collaborate with other scientists who use research techniques by which they attempt to elucidate the neurobiological and genetic bases for development, behavior, and learning. The researcher needs to be familiar with and understand these basic sciences and scientific methods in order to interpret the scientific literature and to generate new research questions for investigation.

A growing body of research has focused on the use of neuroimaging techniques to elucidate brain function and anatomical location of activation during behavioral and psychological tasks. Functional magnetic resonance imaging (fMRI) has become an important tool in developmental-behavioral research. Unlike positron emission tomography (PET) and single photon emission computed tomography (SPECT), fMRI does not include the injection of radioactive materials. Therefore, children can undergo imaging repeatedly, which allows for study during different disease states or throughout developmental changes. It also allows the researcher to study healthy children at low risk for the disorder under investigation. For example, using fMRI, researchers have sought to localize the areas of brain activation with attention or reading tasks in children with ADHD or dyslexia and compare those activation patterns with patterns in normal children.^29,30 Other researchers have used brain proton magnetic resonance spectroscopy to identify children with brain creatine deficiency who have global developmental delay.³¹ It is likely that functional neuroimaging techniques will play an important role in the future study of the neurobiological basis of developmental and behavioral problems or conditions. Future researchers should receive training in the science of these techniques to enable them to collaborate with neuroradiologists in answering questions that span the spectrum of the biopsychosocial model.

Similarly, training in the genetics of developmental and behavioral disorders is important for researchers. Understanding the role of genetics will empower the researcher to incorporate genetics into research hypotheses and methods. One area that should be considered for inclusion in research training is quantitative behavioral genetics.³² In this field, investigators seek to determine the proportion of variance in behaviors resulting from heritability, shared environment (family, neighborhood, home environment), and unshared environment (e.g., peers, teachers, differential parental treatment, illnesses). For example, one article focused on the quantitative behavioral genetics of child temperament.³³

Another important area for research is the association of behavioral phenotypes with molecular genetic findings and variations. Behavioral and psychoeducational profiles have been elucidated through research for velocardiofacial, Williams, Down, Prader-Willi, fragile X, and Turner syndromes, among others.³⁴ Studies have shown associations of these behavioral and psychological characteristics with specific molecular genetic patterns. The availability of molecular genetic techniques has also allowed scientists to study behavioral phenotypes of subjects with “milder” genetic deficits. For example, in one study, researchers found an association of fragile X premutation (carriers) with autistic spectrum diagnosis through the use of molecular genetic studies.³⁵ Other researchers have been studying polymorphisms in specific genes, such as the dopamine D₄ receptor gene (D4DR) and the serotonin transporter promoter gene (5-HTTLPR), and finding associations with ADHD and temperamental traits.^36,37 Collaboration between DBP researchers and geneticists and the use of molecular genetics technology are likely to yield important findings concerning the etiology of developmental-behavioral disorders and normative behavior patterns.