Literature Review on the Impact of Digital Technology on Learning and Teaching

Digital learning and raising attainment

Key findings

There is conclusive evidence that digital equipment, tools and resources can, where effectively used, raise the speed and depth of learning in science and mathematics for primary and secondary age learners. There is indicative evidence that the same can be said for some aspects of literacy, especially writing and comprehension. Digital technologies appear to be appropriate means to improve basic literacy and numeracy skills, especially in primary settings.

The effect sizes are generally similar to other educational interventions that are effective in raising attainment, though the use of digital learning has other benefits. Also, the extent of the effect may be dampened by the level of capability of teachers to use digital learning tools and resources effectively to achieve learning outcomes. More effective use of digital teaching to raise attainment includes the ability of teachers to identify how digital tools and resources can be used to achieve learning outcomes and adapting their approach, as well as having knowledge and understanding of the technology. This applies in all schools.

Where learners use digital learning at home as well as school for formal and non-formal learning activities these have positive effects on their attainment, because they have extended their learning time. This is particularly important for secondary age learners.

The assessment framework, set out in Annex 2, identifies a number of educational benefits that digital learning and teaching has the potential to help learners aged 5 to 18 to realise, through the opportunity to learn in different ways, access more sources of information, and be tested and get feedback differently. In terms of raising attainment, these benefits include short term outcomes, such as having a greater feeling of control over learning and more confidence to practise a skill, through to medium term outcomes such as faster acquisition of knowledge and skills, and improved impacts in terms of learners achieving higher exam or test results where digital technology has been used.

In this section, the impact of digital technology on children's attainment in a range of areas is discussed, followed by the impact on aspects of numeracy, literacy and science learning.

Raising children's attainment

There is a substantial body of research that has examined the impact of digital tools and resources on children's attainment in a range of areas.

Higgins et al (2012) provide a summary of research findings from studies with experimental and quasi-experimental designs, which have been combined in meta-analyses to assess the impact of digital learning in schools. Their search identified 48 studies which synthesised empirical research of the impact of digital tools and resources on the attainment of school age learners (5-18 year olds).

They found consistent but small positive associations between digital learning and educational outcomes. For example, Harrison et al (2004) identified statistically significant findings, positively associating higher levels of ICT use with school achievement at each Key Stage in England, and in English, maths, science, modern foreign languages and design technology. Somekh et al (2007) identified a link between high levels of ICT use and improved school performance. They found that the rate of improvement in tests in English at the end of primary education was faster in ICT Test Bed education authorities in England than in equivalent comparator areas. However, Higgins et al note that while these associations show, on average, schools with higher than average levels of ICT provision also have learners who perform slightly higher than average, it may be the case that high performing schools are more likely to be better equipped or more prepared to invest in technology or more motivated to bring about improvement.

Higgins et al report that in general analyses of the impact of digital technology on learning, the typical overall effect size is between 0.3 and 0.4 - just slightly below the overall average for researched interventions in education (Sipe & Curlette, 1997; Hattie, 2008) and no greater than other researched changes to teaching to raise attainment, such as peer tutoring or more focused feedback to learners. The range of effect sizes is also very wide (-0.03 to 1.05),which suggests that it is essential to take into account the differences between technologies and how they are used.

Table 4: Summary of meta-analyses published between 2000 and 2012 (in Higgins et al 2012)

Focus	No of studies	Overall Study Effect (ES)	Impact on
General	7	0.24-1.05	Academic success; academic outcomes; learner achievement; school achievement; cognitive outcomes
Mathematics	4	0.33-0.71	Mathematics; mathematics performance.
Mathematics and Science	1	0.01-0.38	Mathematics; computer tutorials in science; science simulations; live 'labs'
Science	3	0.19-0.38	Lower order outcomes; higher order outcomes; retention follow up test; science academic achievements
Literacy	12	-0.03-0.55	Reading skills and comprehension; writing quantity and quality; accelerated reader; standardised reading tests; spelling; word processing on writing; ICT on spelling; computer texts on reading
Other Focus	6	0.07-0.46	Academic achievement; individual achievement; learning outcomes; mathematics achievement; cognitive gains.

In an earlier meta-analysis, Liao et al (2007), considered the effects of digital tools and resources on elementary school learners' achievement in Taiwan. Synthesizing research comparing the effects of digital learning (equipment, tools and resources) with traditional instruction on elementary school learners' achievement, they considered quantitative and qualitative information from 48 studies including over 5,000 learners. Of the 48 studies, 44 (92%) showed positive effects in favour of a computer assisted intervention, while four (8%) were negative and favoured a traditional instruction method. Nearly 60% of the studies examined the effects of computer aided instruction for teaching mathematics or science. Another 11% of the studies concentrated on the teaching of reading and language. They found an overall positive effect size across all the studies of 0.45 (study-weighted grand mean), which is considered to be a moderate effect, with a wide range of effect sizes (from 0.25 to 2.67).

No significant differences were found between subject areas, and the authors suggest that digital learning has the potential to be implemented in many different subject areas. They found that the two subjects that showed the highest effects were reading and languages, which had a high positive effect size of 0.7. Studies using computer simulations also had higher effects. The authors suggest this may be because simulations can provide learners with the opportunity to engage in a learning activity which could not be replicated in a classroom.

More qualitative studies have identified how improvements in attainment are achieved. From a wide study of primary and secondary schools in England that were early adopters in using digital learning and teaching, Jewitt et al (2011) concluded that:

• Using digital resources provided learners with more time for active learning in the classroom;
• Digital tools and resources provided more opportunity for active learning outside the classroom, as well as providing self-directed spaces, such as blogs and forums, and access to games with a learning benefit;
• Digital resources provided learners with opportunities to choose the learning resources;
• The resources provided safer spaces for formative assessment and feedback.

The sections below focus on specific key areas of attainment: literacy, numeracy, and science learning.

Literacy

There is a large body of research that has examined the impact of digital equipment, tools and resources on children's literacy. The effects are generally positive, though not as large as the effects found where digital learning is used to improve numeracy, and consistent in finding that ICT helps improve reading and writing skills, as well as developing speaking and listening skills.

Effect of context

Archer and Savage (2014) undertook a meta-analysis to reassess the outcomes presented in three previous meta-analyses considering the impact of digital learning on language and literacy learning: Slavin et al (2008 and 2009) and Torgenson and Zhu (2003). Overall they found a relatively small average positive effect size of 0.18, with a few of the studies having a negative effect and three studies showing moderate to large effect sizes. The authors found that programmes with a small number of participants tended to show larger effect sizes than larger programmes but that not all were statistically significant.

Archer and Savage sought to understand whether the context within which the digital tool or resource was used has an impact on outcomes. In particular, they examined whether training and support given to the teachers or other staff delivering the programme had an impact. The authors found that training and support could be identified in around half of the studies and that it did appear to have a positive impact on the effectiveness of the literacy intervention, with the average effect size rising to 0.57. The authors conclude that this indicates the importance of including implementation factors, such as training and support, when considering the relative effectiveness of digital learning and teaching.

Effect on specific literacy skills

In their meta-analysis, Higgins et al (2012) found that digital learning has a greater impact on writing than on reading or spelling. For example, Torgenson and Zhu (2003) reviewed the impact of using digital technology on the literacy competences of 5-16 year-olds in English and found effect sizes on spelling (0.2) and reading (0.28) much lower than the high effect size for writing (0.89).

In their meta-analysis of studies investigating the effects of digital technology on primary schools in Taiwan, Laio et al (2007) considered studies over a range of curriculum areas; 11 of which addressed the effects of using digital learning in one or more literacy competence. They found no significant differences in effect size between the different subject areas, suggesting the potential for digital technology to raise outcomes is equal across different subjects. However, they did note that the two areas that showed the highest effect sizes (over 0.7) were reading and comprehension.

Effect of specific digital tools and resources

Somekh et al (2007) evaluated the Primary School Whiteboard Expansion (PSWB) project in England. They found that the length of time learners were taught with interactive whiteboards (IWBs) was a major factor in learner attainment at the end of primary schooling, and that there were positive impacts on literacy (and numeracy) once teachers had experienced sustained use and the technology had become embedded in pedagogical practice. This equated to improvements at Key Stage 2 writing (age 11), where boys with low prior attainment made 2.5 months of additional progress.

Hess (2014) investigated the impact of using e-readers and e-books in the classroom, among 9-10 year olds in the USA. The e-books were used in daily teacher-led guided reading groups, replacing traditional print books in these sessions. Teachers also regularly used the e-readers in sessions where the class read aloud, and e-readers were available to learners during the school day for silent reading. The study found a significant difference in reading assessment scores for the group using the e-readers. Scores improved for both male and female learners and the gap between males and females decreased.

The use of digital tools and resources also appears to affect levels of literacy. Lysenko and Abrami (2014) investigated the use of two digital tools on reading comprehension for elementary school children (aged 6-8) in Quebec, Canada. The first was a multimedia tool which linked learning activities to interactive digital stories. The tool included games to engage learners in reading and writing activities, and instructions were provided orally to promote listening comprehension. The second tool was a web-based electronic portfolio in which learners could create a personalised portfolio of their reading and share work with peers, teachers and parents to get feedback. The authors found that in classes where both tools were used together during the whole school year learners performed significantly better both in vocabulary and reading comprehension (with medium-level effect sizes) than learners in classes where the tools were not part of English language instruction.

Rosen and Beck-Hill (2012) reported on a study programme that incorporated an interactive core curriculum and a digital teaching platform. At the time of their report it was available for 9-11 year old learners in English language, arts and mathematics classes in Dallas, Texas. The online platform contained teaching and learning tools. Learners were assessed using standardised tests administered before the programme and after a year's participation. The results of increased achievement scores demonstrated that in each of the two school year groups covered, the experimental learners significantly outperformed the control learners in reading and maths scores. In observations in classrooms that used the programme, the researchers observed higher teacher-learner interaction, a greater number and type of teaching methods per class, more frequent and complex examples of differentiation processes and skills, more frequent opportunities for learner collaboration, and significantly higher learner engagement. The authors report that the teaching pedagogy observed in the classrooms differed significantly from that observed in more traditional classrooms. The teachers following the programme commented that the digital resources made planning and implementing 'differentiation' more feasible. This is differentiation of teaching in terms of content, process, and product, to reflect learners' readiness, interests, and learning profile, through varied instructional and management strategies.

Effect of the amount and quality of digital technology use

The uses of digital technology and access to it appear to be critical factors. Lee et al (2009) analysed how in the US 15-16 year-old learners' school behaviour and standardised test scores in literacy are related to computer use. Learners were asked how many hours a day they typically used a computer for school work and for other activities. The results indicated that the learners who used the computer for one hour a day for both school work and other activities had significantly better reading test scores and more positive teacher evaluations for their classroom behaviours than any other groups^[5]. This was found while controlling for socio-economic status, which has been shown to be a predictor of test scores in other research. The analysis used data from a national 2002 longitudinal study, and it is likely that learners' usage of computers has increased and changed since that time.

Biagi and Loi (2013), using data from the 2009 Programme for International Student Assessment (PISA) and information on how learners used digital technology at school and at home (both for school work and for entertainment), assessed the relationship between the intensity with which learners used digital tools and resources and literacy scores. They examined uses for: gaming activities (playing individual or collective online games), collaboration and communication activities (such as linking with others in on-line chat or discussion forums), information management and technical operations (such as searching for and downloading information) and creating content, knowledge and problem solving activities (such as using computers to do homework or running simulations at school). These were then compared to country specific test scores in reading. The authors found a positive and significant relationship between gaming activity and language attainment in 11 of the 23 countries studied. For the other measures, where relationships existed and were significant, they tended to be negative.

The more recent PISA data study (OECD, 2015, using 2012 results) also found a positive relationship between the use of computers and better results in literacy where it is evident that digital technology is being used by learners to increase study time and practice^[6]. In addition, it found that the effective use of digital tools is related to proficiency in reading.

Numeracy

There is a large body of research which has examined the impact of digital equipment, tools and resources on children's numeracy skills and mathematical competences throughout schooling. Higgins et al (2012) found from their meta-analysis that effect sizes of tested gains in knowledge and understanding tend to be greater in mathematics and science than in literacy. The key benefits found relate to problem solving skills, practising number skills and exploring patterns and relationships (Condie and Monroe, 2007), in addition to increased learner motivation and interest in mathematics.

Effect on specific numeracy skills

Li and Ma's (2010) meta-analysis of the impact of digital learning on school learners' mathematics learning found a generally positive effect. The authors considered 46 primary studies involving a total of over 36,000 learners in primary and secondary schools. About half of the mathematics achievement outcomes were measured by locally-developed or teacher-made instruments, and the other half by standardized tests. Almost all studies were well controlled, employing random assignment of learners to experimental or control conditions.

Overall, the authors found that, on average, there was a high, significantly positive effect of digital technology on mathematics achievement (mean effect size of 0.71), indicating that, in general, learners learning mathematics with the use of digital technology had higher mathematics achievement than those learning without digital technology. The authors found that:

• Although the difference was small, younger school learners (under 13 years old) had higher attainment gains than older secondary school learners;
• Gains were more positive where teaching was more learner-centred than teacher-centred. In this regard, the authors differentiate between traditional models, where the teacher tends to teach to the whole class, and a learner-centred teaching model which is discovery-based (inquiry-oriented) or problem-based (application-oriented) learning;
• Shorter interventions (six months or less) were found to be more effective in promoting mathematics achievement than longer interventions (between six and 12 months). It is suggested that such gains in mathematics achievement are a result of the novelty effects of technology, as suggested in other research, and as learners get familiar with the technology the novelty effects tend to decrease;
• The authors found no significant effects from different types of computer technology on mathematics achievement. Whether it was used as communication media, a tutorial device, or exploratory environment, learners displayed similar results in their mathematics achievement;
• Equally, the authors found no significant relationship between the effect of using digital technology and the characteristics of learners included in the samples for studies, such as gender, ethnicity, or socio-economic characteristics.

Effect of the amount and quality of digital technology use

The studies by Lee et al (2009) and Biagi and Loi (2013) found similar results for mathematics as they did for reading and literacy in relation to the use of digital equipment. Learners who used a computer at least one hour a day for both school work and other activities had significantly better mathematics test scores and more positive teacher evaluations for their classroom behaviour in mathematics classes than those who did not use the computer. Biagi and Loi (2013) found a significant positive relationship between intensity of gaming activity and maths test scores in 15 countries out of the 23 studied. As with language, the authors found that learners' total use of digital technologies was positively and significantly associated with PISA test scores for maths in 18 of the 23 countries studied.

Studies have found that using digital equipment for formal learning is also associated with increases in learners' motivation for learning mathematics. House and Telese (2011 and 2012) found that:

• For learners aged 13 and 14 in South Korea, for example, those who expressed high levels of enjoyment at learning mathematics, more frequently used computers in their mathematics homework. However, learners who more frequently played computer games and used the internet outside of school tended to report that they did not enjoy learning mathematics;
• Learners in the USA and Japan aged 13 and 14 who showed higher levels of algebra achievement also used computers more at home and at school for school work. Those who used computers most for other activities had lower test scores. In each of the USA and Japan they found that overall computer usage which included use for school work was significantly related to improvements in test scores.

Effect of specific digital tools and resources

Somekh et al (2007) found that, once the use of IWBs was embedded, in Key Stage 1 mathematics (age 7) in England, high attaining girls made gains of 4.75 months, enabling them to catch up with high attaining boys. In Key Stage 2 mathematics (age 11), average and high attaining boys and girls who had been taught extensively with the IWB made the equivalent of an extra 2.5 to 5 months' progress over the course of two years.

Digital tools and resources can also increase some learners' confidence in mathematics as well as their engagement in new approaches to learning and their mathematical competences. Overcoming learners' anxieties about mathematics and their competence in specific aspects of the subject are common concerns in teaching mathematics which hampers their ability to learn (reported in Huang et al 2014).

Huang et al (2014) researched the outcomes, in Taiwan, from a computer game simulating the purchase of commodities, from which 7 and 8 year-old primary school learners can learn addition and subtraction, and apply mathematical concepts. The model combined games-based learning with a diagnosis system. When the learner made a mistake, the system could detect the type of mistake and present corresponding instructions to help the learner improve their mathematical comprehension and application. The authors compared two learning groups: both used the game-based model but one without the diagnostic, feedback element. They found that the learning achievement post-test showed a significant difference and also that the mathematics anxiety level of the two learner groups was decreased by about 3.5%.

Passey (2011) found that among over 300 schools in England using Espresso digital resources, those that had been using them over a longer period made significantly greater increases in end of primary school numeracy test results than schools which were recent users.

Science learning

Effects on science knowledge and skills

In their meta-analysis, Laio et al (2007) considered 11 studies looking at the impact of digital technology on science learning. These had a moderate average effect size of 0.38 and generally had positive effects. Condie and Monroe (2007) identified that digital learning made science more interesting, authentic and relevant for learners and provided more time for post-experiment analysis and discussion.

In their study of the PISA data, Biagi and Loi (2013) found a significant positive relationship between learners' total use of digital equipment and science test scores in 21 of the 23 countries they studied. They also found evidence of a significant positive relationship between the intensity of using gaming activity and science scores in 13 of the 23 countries they studied. Somekh et al (2007) found that in primary school science all learners, except high attaining girls, made greater progress when given more exposure to IWBs, with low attaining boys making as much as 7.5 months' additional progress.

Effects of specific digital tools and resources

Digital tools and resources generally have a positive effect on learners' science learning. This can be seen from a number of studies assessing outcomes for learners in different stages of education.

Hung et al (2012) explored the effect of using multi-media tools in science learning in an elementary school's science course in Taiwan. Learners were asked to complete a digital storytelling project by taking pictures with digital cameras, developing the story based on the pictures taken, producing a film based on the pictures by adding subtitles and a background, and presenting the story. From the experimental results, the authors found that this approach improved the learners' motivation to learn science, their attitude, problem-solving capability and learning achievements. In addition, interviews found that the learners in the experimental group enjoyed the project-based learning activity and thought it helpful because of the digital storytelling aspect.

Hsu et al (2012) investigated the effects of incorporating self-explanation principles into a digital tool facilitating learners' conceptual learning about light and shadow with 8-9 year old learners in Taiwan. While they found no difference in the overall test scores of the experimental and control groups, they found a statistically significant difference in retention test scores. Those learners who had paid more attention to the self-explanation prompts tended to outperform those in the control group.

Anderson and Barnett's (2013) study, in the US, examined how a digital game used by learners aged 12-13 increased their understanding of electromagnetic concepts, compared to learners who conducted a more traditional inquiry-based investigation of the same concepts. There was a significant difference between the control and experimental groups in gains in knowledge and understanding of physics concepts. Additionally, learners in the experimental group were able to give more nuanced responses about the descriptions of electric fields and the influence of distance on the forces that change experience because of what they learnt during the game.

Güven and Sülün (2012) considered the effects of computer-enhanced teaching in science and technology courses on the structure and properties of matter, such as the periodical table, chemical bonding, and chemical reactions, for 13-14 year olds in Turkey. Their proposition was that computer-enhanced teaching can instil a greater sense of interest in scientific and technological developments, make abstract concepts concrete through simulation and modelling, and help to carry out some dangerous experiments in the classroom setting. They found a significant difference in achievement tests between the mean scores of the group of learners who were taught with the computer-enhanced teaching method and the control group who were taught with traditional teaching methods.

Belland (2009) investigated the extent to which a digital tool improved US middle school children's ability to form scientific arguments. Taking the premise that being able to construct and test an evidence-based argument is critical to learning science, he studied the impact of using a digital problem based learning tool on 12-14 year olds. Learners worked in small groups and were asked to develop and present proposals for spending a grant to investigate an issue relating to the human genome project. Those in the experimental group used an online system which structured the project into stages of scientific enquiry. The system prompted the learners to structure and organise their thinking in particular ways: by prompting the learners individually, sharing group members' ideas, tasking the group to form a consensus view, and prompting the group to assign specific tasks among themselves.

Using pre- and post- test scores to assess the impact on learners' abilities to evaluate arguments, Belland found a high positive effect size of 0.62 for average-achieving learners compared to their peers in the control group. No significant impacts were found for higher or lower-achieving learners. Belland suggests that for high-achieving learners, this may be because they already have good argument making skills and are already able to successfully structure how they approach an issue and gather evidence. The study also used qualitative information to consider how the learners used the digital tool and compared this to how learners in the control group worked. The author found that in the experimental group they made more progress and were more able to divide tasks up between them, which saved time. They also used the tool more and the teacher less to provide support.

Kucukozer et al (2009) examined the impact of digital tools on teaching basic concepts of astronomy to 11-13 year old school children in Turkey. Learners were asked to make predictions about an astronomical phenomenon such as what causes the seasons or the phases of the moon. A digital tool was used to model the predictions and display their results. The learners were then asked to explain the differences and the similarities between their predictions and their observations. In the prediction and explanation phase the learners worked in groups to discuss their ideas and come to a conclusion. In the observation phase they watched the 3D models presented by their teacher. Thereafter, they were asked to discuss and make conclusions about what they had watched. The authors found that instruction supported by observations and the computer modelling was significantly effective in bringing about better conceptual understanding and learning on the subject.

Ingredients of success

Where studies examine the process that brings about positive results from digital learning and teaching compared to traditional approaches, it is evident that these are more likely to be achieved where digital equipment, tools and resources are used for specific learning outcomes and built into a teaching model from the outset. This broadly supports Higgins et al's (2012) conclusions that:

• Digital technology is best used as a supplement to normal teaching rather than as a replacement for it;
• It is not whether technology is used (or not) which makes the difference, but how well the technology is applied to support teaching and learning by teachers;
• More effective schools and teachers are more likely to use digital technologies effectively than other schools.

Differences in effect sizes and the extent that learners achieve positive gains in attainment are ascribed by most authors of the studies above to:

• The quality of teaching and the ability of teachers to use the digital equipment and tools effectively for lessons;
• The preparation and training teachers are given to use equipment and tools;
• The opportunities teachers have to see how digital resources can be used and pedagogies adapted (Rosen and Beck-Hill, 2012; Belland, 2009).

Teachers have to adapt to learner-centred approaches to learning if they are to use digital tools and resources (Li and Ma, 2010).

As well as ensuring digital tools and resources are supporting learning goals, success appears to also be linked to some other factors:

• The availability of equipment and tools within schools (and at home);
• How learners use digital equipment. Higgins et al (2012) found that collaborative use of technology (in pairs or small groups) is usually more effective than individual use, though some learners - especially younger children - may need guidance in how to collaborate effectively and responsibly;
• The extent that teaching continues to innovate using digital tools and resources (Higgins et al, 2012).

Fullan (2013) suggested four criteria that schools should meet if their use of digital technology to support increased attainment is to be successful. These were that systems should be engaging for learners and teachers; easy to adapt and use; ubiquitous - with access to the technology 24/7; and steeped in real life problem solving.

Fullan and Donnelly (2013) developed these themes further, proposing an evaluation tool to enable educators to systematically evaluate new companies, products and school models, using the context of what they have seen as necessary for success. Questions focus on the three key criteria of pedagogy (clarity and quality of intended outcome, quality of pedagogy and the relationship between teacher and learner, and quality of assessment platform and functioning); system change (implementation support, value for money, and whole system change potential) and technology (quality of user experience/model design, ease of adaptation, and comprehensiveness and integration).