This site use cookies to provide the best possible experience for you. By closing this message you agree to our use of cookies. Our full privacy policy is available here.
Close disclaimer
RESEARCH AND ANALYSIS

The effectiveness of ICT on literacy learning in English, 5-16

The set of four in-depth sub-reviews follows an initial in-depth sub review on the impact of networked ICT on literacy learning (Andrews et al., 2002). This review is one of those four sub-reviews and addresses a question about the effectiveness of ICT on literacy learning.
Torgerson, C. & Zhu, D. (2003). A Systematic Review and Meta-Analysis of the Effectiveness of ICT on Literacy Learning in English, 5-16. in, Research Evidence in Education Library. London: EPPI-Centre, Social Science Research Unit, Institute of Education.

A range of five different kinds of ICT interventions emerged from the twelve included randomized controlled trials (RCTs) in the review: (1) computer-assisted instruction (CAI), (2) networked computer system (classroom intranet), (3) word-processing software packages, (4) computer-mediated texts (electronic text) and (5) speech synthesis systems. There were also three literacy outcomes: (1) reading, including reading comprehension and phonological awareness (pre-reading understandings), (2) writing and (3) spelling.

In synthesis (1), for five different ICT interventions, overall we included 20 comparisons from the 12 RCTs: 13 were positive and seven were negative. Of the positive ones, three were statistically significant, whilst of the seven negative trials, one was statistically significant. These data would suggest that there is little evidence to support the widespread use of ICT in literacy learning in English.

This also supports the findings from previous systematic reviews that have used data from rigorous study designs. It also supports the most recent observational data from the Impact2 study. These findings support the view that ICT use for literacy learning should be restricted to pupils participating in rigorous, randomised trials of such technology.

In synthesis (2), we undertook three principal meta-analyses: one for each of the three literacy outcomes measures in which we were interested. In two, there was no evidence of benefit or harm; that is, in spelling and reading the small effect sizes were not statistically significant). In writing, there was weak evidence for a positive effect, but it was weak because only 42 children altogether were included in this meta-analysis.

Systematic review
A systematical summary of studies in the current topic based on formal criterias for evalution of related studies.
Published: 11.04.2013
Last updated 27.10.2014
PDF PDF - 716 KB Project summary

A range of five different kinds of ICT interventions emerged from the twelve
included RCTs in the review: (1) computer-assisted instruction (CAI), (2)networked computer system (classroom intranet), (3) word-processing software packages, (4) computer-mediated texts (electronic text) and (5) speech synthesis systems. There were also three literacy outcomes: (1) reading, including reading comprehension and phonological awareness (pre-reading understandings), (2) writing and (3) spelling.

Six RCTs evaluated CAI interventions (Berninger et al., 1998; Heise et al., 1991; Jinkerson and Baggett, 1993; Lin et al., 1991; McArthur et al., 1990; Mitchell and Fox, 2001). The CAI interventions consisted of studies designed to increase spelling abilities, reading abilities or phonological awareness (pre-reading understandings). One RCT evaluated a networked computer system intervention (Golden et al., 1990) and two RCTs evaluated word-processing interventions; three RCTs evaluated computer mediated texts interventions and one RCT evaluated a speech synthesis intervention. 

In synthesis (1), for five different ICT interventions, overall we included 20
comparisons from the 12 RCTs: 13 were positive and seven were negative. Of the positive ones, three were statistically significant, whilst of the seven negative trials, one was statistically significant. These data would suggest that there is little evidence to support the widespread use of ICT in literacy learning in English.


This also supports the findings from previous systematic reviews that have used data from rigorous study designs. It also supports the most recent observational data from the Impact2 study. These findings support the view that ICT use for literacy learning should be restricted to pupils participating in rigorous, randomised trials of such technology.  

In synthesis (2), we undertook three principal meta-analyses: one for each of the three literacy outcomes measures in which we were interested. In two, there was no evidence of benefit or harm; that is, in spelling and reading the small effect sizes were not statistically significant). In writing, there was weak evidence for a positive effect, but it was weak because only 42 children altogether were included in this meta-analysis.

Conclusions: in-depth review

We identified 12 relatively small RCTs for the in-depth review. Some were so
small that they could only really be considered to be pilot studies. This group of tiny trials represent the sum of the most rigorous effectiveness evidence available to date upon which to justify or refute the policy of spending millions of pounds on ICT equipment, software and teacher training. Given that the trials showed little evidence of benefit large, rigorously design, randomised trials are urgently required.

Strengths and limitations

We focused on a robust research design (RCT) appropriate for an effectiveness review. We applied rigorous inclusion and exclusion criteria for including studies in the in-depth review. All the included RCTs were highly relevant to the review and were assessed as being of high or medium overall weight of evidence. We did not include studies of other experimental designs; we did not attempt to combine the results of RCTs with trials of other designs. There was high quality assurance for the review: independent double-screening, data-extraction and quality assessment at each stage.

We did not search for any studies published before 1990. The reason for this is that we felt that ICT of the 1980s and before was relatively unsophisticated compared with current IT provision. Therefore, trying to inform current ICT policy from studies that used 1980s technology could be misleading. We may have missed some studies. Nevertheless, we accept the possibility that our results could have been affected by publication bias. Publication bias is a very real problem for any systematic review.

The fact that we have found and included some negative studies of ICT and literacy is somewhat re-assuring as publication bias tends mainly to affect negative studies. Nevertheless, one interpretation of our data could be that our results are over-optimistic as it is likely that the studies that remain unpublished are more likely to be negative studies than positive ones. If this is true, then the overall effects of ICT could be harmful. All the studies included in the in-depth review were undertaken in the US so they may be of limited generalisability to the UK. All of the participants in the studies were either very young children in the stages of beginning literacy, or slightly older children who were experiencing difficulties or disabilities in learning in literacy.  

Implications for policy, practice and research

Policy-makers should refrain from any further investment in ICT and literacy until at least one large and rigorously designed randomised trial has shown it to be effective in increasing literacy outcomes. 

Teachers should be aware that there is no evidence that non-ICT methods of instruction and non-ICT resources are inferior to the use of ICT to promote literacy learning.

A series of large, rigorously designed RCTs to evaluate ICT and literacy learning across all age ranges is urgently required.