Why is intervention fidelity so important in the design of a quantitative research study?

Treatment fidelity in reading intervention research

There are three ways to examine treatment fidelity (TF): (1) describing how TF is checked and/or promoted, (2) measuring TF, i.e. reporting the actual level of TF by means of numerical data to make statements about how well interventions were implemented, and/or (3) investigating the contribution of TF to intervention outcomes (Gresham, MacMillan, Beebe-Frankenberger, & Bocian, 2000). Although the importance of investigating TF is widely accepted, only about 20% of studies focusing on learning

Measurement of treatment fidelity

In recent studies, quantitative treatment fidelity was measured by means of exposure/training time, including number of weeks (e.g. Vadasy et al., 2005), or by adherence/program completion (e.g., van Otterloo et al., 2006). With regard to qualitative treatment fidelity, three measures are important: level matching/differentiation, tutor instruction/support quality, and child task orientation. Matching the difficulty of what is taught to the mastery level of the learner contributes to the

The present study

A necessary condition for a study investigating treatment fidelity is an intervention program that has proven its efficacy. In a previous study (Regtvoort et al., 2013), we showed that children at-risk of reading failure who had finished a computer-assisted program delivered by non-professional tutors in Grade 1 and 2 outperformed untrained controls on all reading measures and maintained their benefit one year after finishing the program. It was concluded that this program (Bouw!, see Method)

Design

Table 1 shows occasions of test administration and the two phases of intervention. Children were individually tested by trained graduate students at school. The intervention in Kindergarten was carried out at home by the parents. There was no necessity to tune these activities with what was done at school because initial reading skills (i.e., letters, word segmentation and blending) are not formally taught in Dutch Kindergarten. The period was relatively short – 18 weeks (starting Mid-February)

Discussion

Quantitative treatment fidelity covered an overall level of almost 67%, which can be called sufficient (Durlak & DuPre, 2008). With regard to qualitative treatment fidelity, the overall level was higher: 87% for tutor support and 70% for level matching. The vast majority of the children (around 93%) showed sufficient to high levels of task orientation and levels of task orientation were significantly higher in Grade 1 than in Kindergarten.

The results confirmed that non-professional tutors

Limitations

The present findings should be considered in the context of limitations. First, the sample size was too small to explore the direction of effects through more sophisticated ways of analysis (e.g., structural equation modeling) and to include interaction effects. We therefore suggest that future replications should include larger samples. Second, the study focused on the children who received the intervention, but did not include a measurement of the no-training control group, which prevented

Conclusion

A strength of the study is that TF was investigated in depth over a substantial period of time. Moreover, it included the transition from informal instruction in Kindergarten to formal instruction in Grade 1. To our knowledge, there are no other studies that have investigated this issue. The findings supported the effect of quantitative treatment fidelity of the tutor- and computer assisted intervention program on reading, both initial reading skills and reading fluency. In addition, the

Recommended articles (6)