Recent Developments in Technology and Language Learning: A Literature Review and Meta-analysis
Abstract:
The purpose of this study is to assess the potential of technology for improving language education. A review of the effectiveness of past and current practices in the application of information and communication technology(ICT) in language education and the availability as well as capacities of current ICTs was conducted. The review found that existing literature on the effectiveness of technology uses in language education is very limited in four aspects: a) The number of systematic, well-designed empirical evaluative studies of the effects of technology uses in language learning is very small, b) the settings of instruction where the studies were conducted were limited to higher education and adult learners, c) the languages studied were limited to common foreign languages and English as a foreign or second language, and d) the experiments were often short-term and about one or two aspects of language learning (e.g., vocabulary or grammar). However the limited number of available studies shows a pattern of positive effects. They found technology-supported language learning is at least as effective as human teachers, if not more so.
Effectiveness of Technology in Language Education, Limitations of Research Studies, Literature Review, Meta-analysis
This review study is intended to address three related issues in technology and language education. First, policy makers and the general public are interested in learning about the effectiveness of using technology in language education because they need that information to help decide future investment decisions regarding technology (President's Committee of Advisors on Science and Technology (Panel on Educational Technology), 1997). Second, researchers and
developers are interested in knowing what has been done and what we already know about using technology to enhance language learning. Such knowledge will hopefully guide their further explorations and development. Third, language educators want to know what works and what does not so that they can make informed decisions in selecting the appropriate technology to use in their teaching.
Answering these questions is not easy for a number of reasons. First, technology is an ill-defined concept that encompasses a wide range of tools, artifacts, and practices, from multimedia computers to the Internet, from videotapes to online chatrooms, from web pages to interactive audio conferencing. These technologies vary a great deal in their capacity, interface, and accessibility. It is thus misleading to think the effects of videotapes are the same as those of the online chatrooms just because they are all called "technology." Second, the effects of any technology on learning outcomes lie in its uses. A specific technology may hold great educational potential, but, until it is used properly, it may not have any positive impact at all on learning. Thus, assessing the effectiveness of a technology is in reality assessing the effectiveness of its uses rather than the technology itself. Since most information and communication technologies (ICTs) can be used in a variety of ways, some more effective than others, it is inappropriate to overgeneralize the effectiveness (or lack thereof) of one way of using the technology to the technology itself. Third, to further complicate things, the effectiveness of an educational approach is highly mediated by many other variables—the learner, the task, the instructional setting, and of course the assessment tool. Thus, even the same use of a particular technology in different instructional settings may result in different learning outcomes.
Clearly it is unreasonable to expect any single study to tell us to what degree technology is effective in improving language learning. However, a comprehensive review of many studies can get us closer to an answer (e.g., Cavanaugh, 2001; Chapelle, 1997; Lou, Abrami, & d'Apollonia, 2001; Salaberry, 2001). With the help of a research method called meta-analysis (Glass, 1977; Hedges & Olkin, 1985; Lyons, 1995a), we can assess the effectiveness of technology uses in language education by analyzing findings of numerous empirical studies. A carefully conducted review can also help us develop a map of past and current work in the field of technology and language education. The map should reveal what we know, what we have done, what works, and what does not. The study in this paper offers such a review.
Focusing on the issues of effectiveness, this review attempts to achieve three goals: (a) assessing the overall effectiveness of uses of technology in language education through meta-analysis, (b) exploring patterns of recent efforts in using technology to improve language learning, and (c) identifying effective ways to use technology in language education.
There is a long history of using technology to improve language learning (Salaberry, 2001). The review in this paper is limited to research published in referred journals during the last five years, from 1997 to 2001. The decision to limit the review to this period of time was motivated by the concern for relevance. The primary purpose of the review is to seek evidence and ideas that will guide our future work, rather than paint a comprehensive historical picture of research in computer-assisted language learning, which can be found in many existing publications (e.g., Chapelle, 2001; Levy, 1997; Salaberry, 2001). Thus, it is reasonable to focus on studies of technological applications that have the most relevance. Relevance is considered in two areas: technology and pedagogy. As we know, technology changes constantly and rapidly. The technological innovations that we are most interested in and that will most likely have an impact on language education in the future are: (a) multimedia computing; (b) the Internet, especially the web; and (c) speech synthesis and recognition. These innovations were a fairly recent development, and efforts to apply them in language education occurred even later. Focusing on the research publications over the past five years in this way should give us sufficient insight into the applications of these relatively new technologies.
There was also a major paradigm shift in the pedagogical and research focus of technology applications in language education recently (Chapelle, 1997, 2001; Pennington, 1996; Salaberry, 2001)—a shift away from traditional drill-and-skill computer-aided instruction (CAI) models toward multimedia, intelligent CAI, and integration models. Studies about applications of these newer models appeared more recently as well.
Works included in this review were identified from five representative journals devoted to research on second/foreign language education and technology and language learning.
Selecting Representative Journals
A four-step process was followed to identify the representative journals. First, a key word search using "computer assisted language learning" was performed on ERIC through FirstSearch. The search was limited by year, document type, and language. Only journal articles published from 1997 through 2001 in English were included. The search resulted in a total of 389 articles. Second, all articles that did not have the key word "second language" were excluded, which resulted in a total of 355 articles. Third, all articles that appeared in nonpeer-reviewed, irregularly published, or practice-oriented journals or magazines were excluded, resulting in a total of 156 articles. The fourth step was to calculate the distribution of the articles and their sources. These articles were published in 22 different journals. The journals fell into three categories: (a) technology and language learning journals that specifically address issues in applications of
technology in language learning and education; (b) language learning and education journals that address issues in language learning and education in general, of which technology application is a subissue; and (c) educational technology journals that address issues in the application of technology in education in general, of which language learning and education is considered a subarea of study. Figure 1 shows the distribution of articles by journal categories.
Distribution of Articles by Journal Category
However, in many cases, a journal may have had only one or two articles. Nine journals had more than three articles. (see Figure 2).
Distribution of Articles by Journals
As Figure 2 shows, most articles appeared in journals devoted to educational technology, especially language educational technology. The three journals devoted to language learning and technology (System, CALICO Journal, and Language Learning & Technology) published nearly 70% of all the articles.
Further, the articles appeared mostly in two journals: CALICO Journal and Language Learning & Technology (see Figure 3).
Distribution of Articles in Technology Journals
Based on this analysis, the CALICO Journal and Language Learning & Technology were first selected to represent studies published in journals devoted to technology and language learning. The Journal of Educational Computing Research was selected to represent journals devoted to educational technology. The Modern Language Journal and TESOL Quarterly were selected to represent journals in language learning and education. These two journals were selected over Foreign Language Annals (FLA) because they are generally considered more research oriented than FLA. This selection represents 72% of all articles found to be related to computer-assisted language instruction from the ERIC database during the period of 1997 through 2001. Considering that 13% of the articles are scattered in 12 other journals, the selection should be considered as a reasonable representative sample of journals that may publish studies in technology and second language learning and education.
Having selected these five journals, the researcher read the abstracts of all articles published in the journals since 1997 to identify possible studies to include in this review. After identifying the possible articles, the researcher read all identified articles to select the ones suitable for a meta-analysis. The following criteria were used for the selection:
1. The article had to report results of an empirical study or multiple studies on the effectiveness or effects of a technological application aimed at improving language learning. Informational articles describing the process of technological applications or products and opinion articles presenting theories or review the literature were included as part of the review but not for the meta-analysis of effects.
2. Technology was more broadly conceived than just computers because the convergence of media is an emerging trend in language education technology. Thus, studies about the application of video, audio, computer-assisted instruction programs, the web, computer-mediated communication, simulation, speech technologies, word processing, e-books, and grammar checkers were all included.
3. The studies included for the final meta-analysis had to have measures of improvement of language proficiency. Self-assessment of improvement or attitudinal surveys were not acceptable.
A total of 9 articles were found to meet the above criteria for inclusion in the meta-analysis although many more articles published in these journals dealt with technological applications in language learning. While the meta-analysis was conducted on only the 9 studies, the review here draws upon discussions and findings from the other articles as well as a number of books published around the same period of time.
This section is organized into three parts. Part one presents an overview of the literature on technology applications in language education. Part two summarizes how technology has been used to support language learning and the effectiveness of these uses. Part three reports the results from the meta-analysis study intended to assess the overall effectiveness of technology-supported language learning.
Overview of the Literature
The review of recent research on technology-supported language learning reveals a number of interesting points regarding existing research in this area. First, the total number of well designed experimental studies on the effectiveness of technological applications in language learning is very limited. For instance, the four issues in Volume 16 of the CALICO Journal, which were published in 1998 and 1999, contain 10 feature articles. Only two of the 10 met the selection criteria to be included in the meta-analysis. Of all the 51 feature articles published in Language Learning & Technology, one met the criteria. The majority of the articles are either description of cases—uses of technology in language education and processes of software development—or theoretical discussions of principles of technological applications. Some of the limited
empirical studies did not measure learning gains, relied solely on learner self-reports as measures of effectiveness, or were not well designed.
Second, the studies were limited to college level language learners. None of the nine studies was conducted in K-12 settings. Only two of the nine studies were not about college students—their participants were military linguists, also adults.
Third, the target languages under consideration in the studies were also limited. French and Spanish were the most studied (10 out 16 or 63%). Other more commonly studied languages were English as a second language (ESL), German, and Arabic.
Fourth, most of the studies were about the application of a single application instead of systemic large-scale integration of technology. Only two of the 16 studies were about long-term technology integration in the language learning environment. Thus the treatment reported was also short term, lasting from a few hours to a few weeks.
Lastly, the applications of technology to enhance language learning have been wide ranging, both in terms of the types of technology used and the issues language education needs to address. The studies investigated the uses of most available technologies including video, audio, multimedia, communication, network, and speech technologies. These technologies have been applied to support the teaching of various aspects of language learning including vocabulary, grammar, reading, writing, speaking, listening, and culture.
Uses and Effectiveness of Technologies in Language Education
The following paragraphs present detailed discussions of the specific applications that have been studied over the past few years. The discussion is categorized into four groups: access to materials, communication opportunities, feedback, and learner motivation.
Providing Access to Linguistic and Cultural Materials
Access and exposure to engaging, authentic, and comprehensible yet demanding materials in the target language is essential for successful language learning. However for many language learners, whether in classes or self-study settings, such access is often limited. Thus language educators have long looked at information and communication technologies (ICT) as possible solutions to this problem (Egbert, Chao, & Hanson-Smith, 1999; Hanson-Smith, 1999; Salaberry, 2001). The uses of ICT to provide better access and exposure to linguistic and cultural materials fall into the following three areas.
Enhancing access efficiency through digital multimedia technologies
Digital multimedia technologies were used to make access to learning materials more efficient than print media or audio recorders because multimedia (visual, audio, and text) presentations can create stronger memory links than a single
medium alone and digital technology allows instant and accurate playbacks, which helps the learner to access specific segments much more easily without spending time to locate them—a tedious and time consuming process (Hanson-Smith, 1999; Thorton & Dudley, 1996). Shea (2000) compared the time students needed to complete their language learning tasks using captioned video versus interactive videodisc (IVD) and found that the students using IVD completed the tasks significantly faster (p <. 05). Labrie (2000) found that although students spent more time learning a set of French words on paper than those who studied on computer (where they could hear a word pronounced and see a picture about the word), they did not learn more words. In another study, Nutta (1998) examined the learning of students who spent the same amount of time (one hour per day for seven days) learning verb tenses in English in two conditions: attending a regular class and receiving instruction from the teacher versus using a multimedia computer program (audio, video, recording capabilities etc.). Nutta found that the ESL students using the computer program performed as well or significantly better (on three out of six measures, p < .10) than their counterparts attending the class.
Enhancing authenticity using video and the Internet
Video materials can bring natural and context-rich linguistic and cultural materials to the learner, while the Internet enables the learner to access authentic news and literature in the target language, which can reflect current cultural changes more effectively than printed sources (Bacon & Finnemann, 1990; Hanson-Smith, 1999; Herron, Cole, Corrie, & Dubreil, 1999; Herron, Dubreil, Cole, & Corrie, 2000; Kitajima & Lyman-Hager, 1998; Lafford & Lafford, 1997; Lee, 1998; Weyers, 1999). Weyers (1999) studied the effectiveness of authentic video on college Spanish students. He had one class of students watch a Mexican television show as part of a second semester Spanish class that met 60 minutes daily for a total of 8 weeks, while the other class followed the regular curriculum without the video. He found that the video group's performance on both listening comprehension and oral production to be significantly better than the regular group (p < .01). The video group also outperformed their counterparts on other measures of communicative competence. Herron (2000) found that video also helped their first-year college French learners develop significantly better understanding of the target culture. In another study, Green and Youngs (2000) substituted regular classroom instruction with web activities one class period per week for beginning college French and German students. After a semester, they found that
the substitution of one class day for directed, pedagogically sound Web activities seems to have allowed the treatment groups to continue to progress toward their personal and professional goals and allowed them to learn language at a rate similar to that of their peers in the control groups. It also appears, in general, that the students had a positive experience using the web … (p. 108).
Enhancing comprehensibility through learner control and multimedia annotations
Comprehensible input is necessary for language learning, but useful learning materials must also contain enough unfamiliar materials (Krashen, 1985). For language learners, especially beginning and intermediate ones, authentic materials are often beyond their language proficiency and may become incomprehensible without help. To enhance comprehensibility of spoken materials, full caption, keyword caption, or slowing down the speech rate have been found to be effective (Shea, 2000; Zhao, 1997). Zhao (1997) found that the ESL students who were able to flexibly slow down or speed up the rate of speech had significantly better listening comprehension than those who did not (p < .05). For reading materials, glossing or multimedia annotations have been effective means to enhance comprehension (Al-Seghayer, 2001; Chun & Plass, 1997; Johnson, 1999; Lyman-Hager, 2000). Al-Seghayer (2001) compared ESL students' vocabulary learning in different annotation conditions and found that
a video clip in combination with a text definition is more effective in teaching unknown vocabulary than a picture in combination with a text definition … . The variety of modality cues can reinforce each other and are linked together in meaningful ways to provide an in-depth experience (p < .001 ) (p. 225).
Providing Opportunities for Communication
Engaging in authentic communication in the target language is another essential condition for successful language learning yet such opportunities do not exist for most learners. ICT has again been used in many different ways to create opportunities for language learners to communicate in the target language (Hanson-Smith, 1999; Kelm, 1998; Muyskens, 1998; Warschauer & Kern, 2000). Efforts in this area can be summarized into two groups: interaction with the computer and interaction through the computer with remote audiences.
Interactions with the computer
Communicative interactions can occur in either written or spoken language or a combination of both. At the simplest level, a computer program can generate utterances either orally or in writing that require the learner to respond by selecting an answer with a mouse click or providing simple writing responses (Hanson-Smith, 1999). With the advancement of speech synthesis and recognition technologies (Ehsani & Knodt, 1998), the learner can also carry on near natural conversations with a computer program around preselected and programmed topics (Bernstein, Najmi, & Ehsani, 1999; Egan, 1999; Harless, Zier, & Duncan, 1999; LaRocca, Morgan, & Bellinger, 1999; Wachowicz & Scott, 1999). The learner can also give either written or spoken commands to a computer program in a simulation and game environment. The computer program would then perform the command (Holland, Kaplan, & Sabol, 1999; LaRocca
et al., 1999). Harless et al. (1999), for example, tested the effectiveness of a virtual conversation program in Arabic at the Defense Language Institute. The program enabled the students to interview virtual native-speaking characters orally with speech recognition technology. After interacting with these virtual characters for at least 8 hours per day for 4 days, the participants' reading and speaking skills increased significantly (p < .05) while their listening skill increased "convincingly." In another study, Holland (1999) and her colleagues found that a speech-enabled interactive microworld program which allowed the learners of Arabic to construct objects by speaking to the computer improved student motivation and oral output.
Interactions with remote audiences through the computer
Computer-mediated communication (CMC) and teleconferencing technologies have been used to create authentic communication opportunities for language learners since the 1980s (Beauvois, 1997; Pennington, 1996). The uses of CMC technologies, such as electronic mail, bulletin boards, and chatrooms have been found to have many benefits for language learners (Beauvois, 1997; Cahill & Catanzaro, 1997; Kelm, 1998; Salaberry, 2001; Warschauer, 1998). CMC brings the much needed audience to the language learner (Johnson, 1999). It also promotes more equal and better participation, leading to more output in the target language (Beauvois, 1997; Gonzalez-Bueno, 1998). It fosters negotiation and form-focused learning (Pellettieri, 2000). CMC was also found to enhance the writing process and improve student writing (Schultz, 2000). Although CMC communication is, in most cases, conducted in writing, it has been found to improve oral proficiency as well. For instance, Beauvois (1997) found that second-year French learners who held their discussions online achieved better oral proficiency than those who discussed the texts orally in the traditional classroom setting (p < .05).
The capacity for computers to provide instant and individualized feedback has long been recognized by educators, including foreign language educators (Chao, 1999; Salaberry, 2001). While early applications tended to follow the behaviorist tradition by simply assessing the learner's performance and providing simplistic feedback in a correct-or-incorrect fashion, more recent applications are much more contextualized and pedagogically sound (Salaberry, 2001).
Computer-based grammar checkers and spell checkers
Computer-based grammar checkers and spell checkers represent potentially powerful ways to provide feedback to students' written output (Jacobs & Rodgers, 1999). Although the feedback provided by current grammar checkers is not always accurate—albeit immediate—due to its inability to perform semantic analysis and process deep level structures, Burston (2001) found that advanced
students of French benefited tremendously from a French grammar checker. In this study, the students in the treatment group used a French grammar checker while writing their essays, whereas the control group did not. The results suggest that "the effectiveness of the use of Antidote in improving morphosyntatic accuracy in assigned compositions were overwhelmingly positive." (p. 507). The treatment group's first essay scored on average 70%, compared to 20% of the control group. The second essay showed similar results: 85% for the treatment group and 54% for the control.
Automatic speech recognition technology
Automatic speech recognition technology holds the potential to provide feedback that would otherwise be impossible. Pronunciation is a fundamental element of language learning, but providing feedback that can be easily accessible and useful is difficult. In traditional instructional settings, feedback and modeling are often provided by an instructor, who may or may not be good at judging the student pronunciation in the first place. Typical ways to provide feedback often include having students repeat the pronunciation or explaining how the sound should be produced in a very abstract fashion. With the advancement of speech recognition technology, the student can receive feedback in more effective ways (Dalby & Kewley-Port, 1999; Ehsani & Knodt, 1998; Eskenazi, 1999; Mostow & Aist, 1999). Mostow and Aist (1999) have suggested visual, template-based, and model-based feedback. First, a computer program can analyze a student utterance and display the features visually, perhaps with a comparison to that of a native speaker. The program can also display the position and movements of the tongue when a student produces an utterance, which can also be displayed in comparison to that of native speakers. Second, computer programs can compare student pronunciation of individual words or sentences to prerecorded templates. For example, good agreement (r =.81 for high quality speech and r = .76 for telephone-quality speech) was found between automatic and human grading of the pronunciation of English sentences produced by Japanese English learners (Bernstein, Cohen, Murveit, Rtischev, & Weintraub, 1990). More recent studies have found different levels of correlation between machine and human graders: from 0.44 to 0.85 (Bernstein, 1997; Ehsani & Knodt, 1998). Coniam (1998) also found that such high correlation can be achieved at the discourse level. Third, pronunciation can be evaluated against pronunciation models. In this approach, student pronunciation is not limited to preselected words because the model is a generalization of a template.
Tracking and analyzing student errors and behaviors
Tracking and analyzing student errors and behaviors is another approach language educators have experimented with to provide more helpful feedback. Computer programs can store student responses, which can then be analyzed either by a human instructor (Sinyor, 1997) or the computer (Nagata, 1993).
The effectiveness of this approach remains to be determined although Nagata, summarizing her research findings, suggests "traditional feedback may be as good as the intelligent feedback for helping learners to correct word-level errors (e.g., vocabulary and conjugation errors), while the intelligent feedback may be more helpful for understanding and correcting sentence level errors (e.g., particle errors)" (p. 337).
Integrating Technology in the Language Classroom
As mentioned before most of the empirical studies were about a single application used in a few days. We were fortunate to have found two articles that evaluated the effectiveness of more comprehensive uses of technology over a longer period of time (Adair-Hauk, Willingham-McLain, & Youngs, 2000; Green & Youngs, 2001). These efforts were all carried out at Carnegie Mellon University. The first study (Adair-Hauk et al., 2000) was conducted in 1996, and the second study took place in Fall 1998 and Spring 1999 (Green & Youngs, 2001). Participants of the first study were second-semester French students and those of the second study were first-semester French students and first- and second-semester German students. Both studies followed the same format: the treatment group participated in technology-enhanced language learning activities, while the control group attended a regular class for one of the class periods each week. The technological applications included computerized multimedia grammar and vocabulary exercises, instructional video, online spell checker, French-English glossary, and the web. Measures of listening, reading, writing, cultural knowledge, and student attitudes were taken during the course of both studies. Speaking was assessed in the first study. The findings are summarized below.
1. For study 1, when change over time was considered, there was no significant difference between the treatment group and the control group in cultural knowledge, speaking or listening. For study 2, there was no significant difference in any of the skills measured (cultural knowledge, listening, reading, and writing) (p < .05).
2. However, the difference in writing was significant in study 1. The control group's homework writing scores decreased, while the treatment group's increased. Writing test scores also indicate a significant difference between the two groups favoring the treatment group (p < .001). The treatment group also scored significantly better than the control group in reading (p < .001).
3. Both studies found that students in the treatment group spent about the same amount time completing the tasks as their peers in the control group.
Both studies concluded that technology-supported independent language learning is as effective as classroom instruction, if not more.
Assessing the Overall Effectiveness: A Preliminary Meta-analysis
In order to gain a better sense of the overall effectiveness of technology applications in language learning, a meta-analysis was conducted of the studies that included enough data for such an analysis. Meta-analysis is the analysis of analyses—a statistical technique for aggregating the results of multiple experimental studies (Glass, 1976, 1977; Hedges & Olkin, 1985; Lyons, 1995a). The result of each identified study is converted into a measure called effect size. An effect size is obtained by transforming the findings from each study into a standard deviation unit. The effect size indicates the extent to which experimental and control groups differ in the means of a dependent variable at the end of a treatment phase. An effect size (d) is calculated as the difference between the means of the treatment group and the control group divided by the pooled standard deviation.
For the meta-analysis presented here, more than one effect sizes was calculated for several studies because they had more than one measure (e.g., listening, reading, and writing). But in order to satisfy the independence assumption of meta-analysis (Hedges & Olkin, 1985), only one effect size per study was entered into the study. When two or more effect sizes were calculated, they were averaged. The effect sizes used in this analysis are weighted ds, which corrected sample size biases (Hedges & Olkin, 1985). The calculation was performed using Meta-analysis Calculator (Lyons, 1995b), a computer program designed for meta-analyses. Table 1 summarizes the results of the meta-analysis study.
Overall Effect of Technology Applications in Language Learning
Notes: Averaged shows the result when only one averaged effect size was included per study, while All shows the result when all effect sizes are included. K = number of effect sizes. N = number of subjects. The number of subjects was repeatedly counted for each effect size for studies that used multiple measures.
As Table 1 shows, the mean effect size of the 9 studies is quite large, indicating an overwhelmingly positive effect of technology applications on language learning. The confidence interval at the .05 level further confirms this finding. Thus judging from this analysis, it is reasonable to conclude that technology has been shown by the published empirical studies to be very effective in improving student language learning. What is worth mentioning is that this analysis put all technologies and their various applications in language learning
together. It did not differentiate among the areas of improvement in the target language either. In other words, this is a summary of the empirical findings of the effects of a variety of technological applications on virtually all aspects of language learning (e.g., vocabulary, grammar, reading, listening, writing, speaking, and cultural understanding.). Table 2 summarizes the technological applications and measures under investigation in the 9 studies comprising the meta-analysis sample.
Summary of Technology and Content of Studies
DISCUSSION AND CONCLUSIONS
This review study was conducted to achieve three goals: (a) to assess the overall effectiveness of uses of technology in language education through meta-analysis, (b) to explore patterns of recent efforts in using technology to improve language learning, and (c) to identify effective ways to use technology in language education. In this final section, the findings of the study are summarized and their implications are discussed for future research and development efforts in technology supported language education.
In terms of overall effectiveness of technology on language learning, there is evidence suggesting that technology-based language instruction can be as effective as teacher-delivered instruction. Although the number of available experimental studies is limited, a consistent pattern of positive effects is found across the studies. However, this finding should be interpreted with extreme caution for a number of reasons, in addition to the limited number of studies. First, there may be a tendency for journals to publish studies that report significant positive gains. In other words, studies that found less significant or even
negative effect of technology may not have been published. While there is no simple way to verify this assumption, it is to some extent supported by a recent meta-analysis study about the effects of social contexts on computer uses in learning, which found that published studies have a bigger effect size than unpublished studies (Lou et al., 2001). Second, most of the studies had fairly small sample sizes and rarely employed random sampling. Third, the fact that all studies were conducted on college students and adult learners raises questions about the generalizability of the finding to other language learners who may differ in motivation, language background, learning style and ability, and instructional context. For instance, it is very likely that college students are generally more motivated and better learners than K-12 students as a whole. Lastly, in most cases, the researchers of these studies were also the instructors who designed, implemented, and evaluated the technology uses. It is conceivable that the classical "Pygmalion effect" (Rosenthal, 1973) could affect the results. It is also the case that most of the instruments were designed by the researchers, who were also the instructors, instead of independent standardized instruments. It is possible that these measures might have a bias in favor of conditions where technology was applied.
This review found that recent efforts in applying technology in language education share three interesting characteristics. First, many of these efforts were carried out by individual instructors or small groups of individual instructors with limited resources. Consequently, efforts were of smaller scale. Very often only individual technology was used to affect a very specific part of language education. The review found a very limited number of efforts that attempted comprehensive applications of multiple technologies to the whole process of language education. Second, most efforts involved the development of products which then were used in language teaching. Commercially available language software or tools were rarely used in these studies. As a result, the review found many publications describing the development process, while only few articles reported the effectiveness of these products. In the meantime, the numerous commercial products readily available and widely used in classrooms were not studied. Third, the review found that in general current attempts to use technology in language education were not connected and ignored the language learning at the precollege level. This finding is very surprising, especially viewed in the context of technology applications on other content areas, such as mathematics and science, where a large number of studies of technology applications were conducted at the K-12 level.
What are effective uses of technology in language education? This review shows that the application of technologies can be effective in almost all areas of language education. Modern technology can help enhance the quality of input, authenticity of communication, and provide more relevant and useful feedback. In particular, communication technologies such as the Internet and satellite television have been found to be widely used as a way to bring authentic materials into the classroom, involve learners in more authentic communications with
distant audiences, and provide researchers the opportunity to better examine the language learning process. Additionally speech technology, while still not quite ready for full implementation for language education, has already been shown its potential for supporting language learning.
Findings from this review study have significant implications for future work. It is apparent from the literature review that technology, when used properly, can have a positive effect on language learning. It is also apparent that the availability and capacities of information technologies have not been fully taken advantage of by language students or educators. To truly capitalize on modern information and communication technologies to significantly improve language learning, a number of issues must be addressed.
Issue 1: Comprehensive and systematic development of curriculum and content
Technology capacities need to be translated into pedagogical solutions and realized in the forms of curriculum and content for language learners. Current uses of technology, as revealed by the literature review, are fragmented and isolated. There are very few comprehensive technology-based curricula that fully take advantage of the power of available technologies. Thus, in the future, what is needed is the development of full curricula that are supported by available technologies instead of individual tools that are only used infrequently or as a supplement to a primarily print-material-based curriculum.
Issue 2: Basic research to explore effective ways of using technology
The effectiveness of technology on language learning is dependent on how it is used. Certain technologies are more suitable than others for certain learning tasks for certain learners. Therefore research about appropriate ways and contexts of technology use is much needed (Salaberry, 2001).
Issue 3: Technology uses in the classroom
Technology is underutilized in classrooms (Cuban, 2001). The finding that none of the studies found in the major language education and technology journals is about technology use in K-12 classrooms is shocking because studies of technology applications in other subject areas (e.g., mathematics, science, social studies, and language arts) have taken place in mainly K-12 classrooms. The fact that almost all the authors of the reported studies were also the instructors in the experiments suggests a possible explanation: K-12 teachers are not using technology in their teaching and there is a lack of interest among university researchers in studying technology applications in K-12 language classrooms. This finding raises two issues: how to promote technology use in K-12 classrooms and how to encourage more research about technology use in K-12 language classes.
Issue 4: Large scale systematic empirical evaluation of technology uses in schools
There is a clear lack of systematic empirical evaluation efforts to assess the effectiveness of large scale comprehensive uses of technology to support language learning. This, of course, may be accounted for by the lack of large-scale implementation efforts. However, it may also be the result of an overall emphasis on the process rather than result of using technology in language learning. As mentioned earlier in this paper, there was a shift among researchers of language education in the mid 1980s from product-oriented research to more process-oriented research which focuses on understanding how students learn instead of what and how much they learn. For example, many studies on CMC have been about the nature and patterns of student participation in online interactions rather than how much their language skills have improved. While such research is necessary and important, we cannot ignore the practical question of how and in what ways technology uses are effective in improving language learning.
