The Effects of CALL Versus Traditional L1 Glosses on L2 Reading Comprehension

Abstract:
Meta-analytic research of experiments conducted on the effects of native-language (L1) glosses on second-language (L2) reading comprehension have revealed a significant difference between groups of studies with traditional and computer-assisted L1 glosses. This means that learners provided with L1 glosses comprehend significantly more text�through the medium of a computer�than learners with traditional, paper-based L1 glossing aids. The average weighted effect size of studies with computer-assisted L1 glosses is large (g = 1.09) and differs significantly (p < .001) when compared to studies with traditional L1 glosses (g = .39). This article explores factors explaining how and when computer-assisted L1 glosses can be effective in L2 reading comprehension.

KEYWORDS

L2 Reading Comprehension, L1 Glossing, CALL Glossing, CALL Reading, Meta-analysis

INTRODUCTION

I just did the French, I mean not the French, the English definition, `cause that'd be the easiest for me to understand. I mean I don't speak French all that well, and I'd hadda look-up French, I'd a had more French to read, which I might not understand either, so I just used the English `cause it usually summarized it in just a coupla words. - Dave. (Hayden, 1997, pp. 146-147)

Computer-assisted language learning (CALL) studies have found that native-language (L1) glosses are used by readers because they can be an efficient tool for understanding a second-language (L2) text and they are easily tailored to fit learner lexical needs (Bell & Leblanc, 2000; Hayden, 1997). CALL L1 glossing is defined in the present study as an instant look-up capability—dictionary or linked—to which readers have immediate access to meaning in their native language. This meta-analytic study focuses on such glosses and their effectiveness with or without a computer.

309

It is important to consider studies conducted on glossing in light of research concerning the extent to which the L1 can be used in L2 reading and the fairly recent debate concerning general L1 use in L2 learning (e.g., Cook, 2001; Turnbull, 2001; Wells, 1998). This debate continues for several reasons. First, it is unclear how much the L1 should be used in the CALL classroom, given the dynamic social nature of CALL classrooms and that different social contexts may require differing amounts of L1 use (see Taylor, 2002b). Second, much experimental research, especially in the field of reading comprehension, suggests that the L1 can have a significant effect on L2 learning. Yet, this seems contrary to the goal of using the L2 in the classroom as much as possible.

This meta-analysis has found that CALL L1 glosses have a significantly stronger effect on L2 reading comprehension than traditional glosses. Its rationale is described in a series of questions: “Why glosses?”, “Why CALL glosses?”, and, finally, “Why L1 glosses?”

Why glosses?

Bernhardt and Kamil (1995) suggested that linguistic knowledge (linguistic threshold) influences L2 reading comprehension to a greater degree than does L1 reading skill competence (linguistic interdependence). Thus, general background knowledge may not have as essential a role in L2 reading comprehension as linguistic knowledge. Some researchers claim the lexicon plays a central role in second language acquisition (see Gettys, Imhof, & Kautz, 2001). From a bottom-up perspective of reading, such a point of view seems valid. From an interactive reading standpoint, top-down strategies such as activating background knowledge and noticing text structure both are important but are often dependent on lexical knowledge (Eskey, 1988). Glossing is essentially bottom-up lexical help providing direct support for L2 lexical acquisition and perhaps more generally for reading comprehension.

Why CALL glosses?

CALL glosses are easily accessible. Further, using software that provides immediate access to L1 glosses is not difficult. Software, such as Hachette dictionnaire Oxford iFinger (1997), can be easily used while consulting online texts. Instead of looking up a word, all learners need to do is either type in a word or simply click on it. There are several studies that have compared traditional, paper-based L1 glosses with CALL L1 glosses (e.g., Goyette, 1995; Stoehr, 1999). With regard to CALL effectiveness, Chapelle and Jamieson (1986) wrote, “Clearly, CALL effectiveness cannot be looked at as though CALL represented one form of instruction and all students were in need of that kind of instruction” (p. 42). The present study does not claim that CALL L1 glosses are the one factor that will make all the difference in L2 learning. However, we do claim that CALL L1 glosses can make an important difference in L2 reading comprehension.

Given the proliferation of computer technology, use, and skills, there is increasing access to authentic L2 texts. Thus, more research is needed on how computer-

310

mediated reading comprehension of L2 texts can be enhanced by certain kinds of help such as L1 glosses.

Why L1 glosses?

CALL studies have shown that the L2 is seldom used when put next to the L1. Researchers of CALL studies (e.g., David & Lyman-Hager, 1997; Goyette, 1995; Hayden, 1997; Bell & Leblanc, 2000) all report quantitative results suggesting negligible usage of other types of glosses on L2 reading comprehension by L2 readers. First and second-year learners who have the option of choosing between L1 and L2 glosses generally choose the L1.

THE LITERATURE

Aweiss (1994) investigated whether there was a causal relationship between L1 CALL reading supports and L2 reading comprehension in English-speaking learners of Arabic. Aweiss' results revealed that those with CALL L1 glosses recalled significantly more pausal units that those without glosses. Stoehr (1999) found that participants with L1 glosses recalled a significantly higher amount of L2 text than those without glosses. Hayden (1997) observed that CALL L1 glossing, when presented to readers along with several other options of glossing had a significant effect on L2 reading comprehension. Hayden also found that the other glossing options were hardly consulted when placed next to L1 glosses. Moreover, Hayden's results suggested that CALL L1 glosses encouraged more look-ups of words.

The ease and rapidity of access of helps made possible by computer-mediated text may actually promote a greater amount of lookup behaviors, as is associated with bottom-up processing. It is far easier to simply “click” on a word or expression and instantly see a definition than to locate each expression in a printed dictionary. (p. 215)

Goyette (1995) observed that learners with L1 glosses did better than those without glosses, although the difference was not significant. Goyette also found that there was no significant difference in amount of reading time in comparing the online with the hard-copy dictionary treatments. Moreover, even though the reading times were comparable, the online dictionary was used significantly more than the hard-copy dictionary. Goyette (1995) concluded

The large differences in the number of words accessed by dictionary condition and by language indicate that, as expected, the speed and ease of access of the on-line dictionary do increase the frequency of dictionary consultation. On-line dictionary look-ups were found to be particularly prevalent in second-language reading. (p. 96)

Goyette suggested that online dictionaries encouraged readers to look up significantly more words, stating that “speed and ease of access of electronic dictionaries do encourage readers to consult dictionaries more often” (p. 110). It is

311

interesting to note that, with the exception of Goyette's study, glossing had an enormous effect on L2 reading comprehension. Nonsignificant results can easily be explained by confounding variables and Type II error.

THE META-ANALYSIS

Method

Studies relevant and included in the present meta-analysis had to meet four criteria: (a) all studies up to the year 2002 were included, (b) the study had to be either an experimental or a quasi-experimental, (c) at least one of the dependent variables of the study had to be reading comprehension, and (d) the effect of access to glosses—whether traditional or CALL—versus no access to glosses had to be tested. Studies failing to meet any of these criteria were not included in the meta-analysis. Lomicka's (1998) classic study, for example, mixed L1 with L2 glosses and thus did not fit the criteria.

A variety of electronic sources were searched to find relevant studies, including theses and dissertations, for the meta-analysis. The most important of these were Dissertation Abstracts International (DAI), Languages and Literatures Behavior Abstracts (LLBA), The Educational Resources Information Center (ERIC), Psychology Information (Psych INFO), and the World Wide Web (WWW). A total of 18 study reports (outcomes) met the four criteria for inclusion in the present meta-analysis (see Table 1 below).

In order to provide a more comprehensive explanation of what the studies suggest, quantitative meta-analysis is an effective means by which statistical results can be combined and the overall effectiveness of treatments ascertained. Quantitative meta-analysis makes use of a statistic called the “effect size,” which is characterized as large (g = .80 or above), medium (g = .50-.79), small (g = .20- 49), or of no practical importance (less than .20). An effect size of .20 means that on average, the learners provided with the experimental treatment (independent variable, in this case, the CALL L1 glosses) perform two-tenths of a standard deviation (on a dependent variable, e.g., a reading comprehension test) above those participants who did not receive the experimental treatment. The effects size, g, is calculated using the formula

where is the mean of the experimental group, is the mean of the control group, and Sp is the pooled standard deviation of the control and experimental groups. The effect size is a more accurate measure of the effect of the degree of effect of a treatment on participants as measured by the testing instrument than a statistical test of significance because such tests are strongly dependent on sample size and do not provide a standardized indication of the impact of the independent (treatment) variable on the dependent variable (test instrument). Analyzing the literature according to whether or not statistical significance was obtained can often lead to the wrong overall conclusion, as demonstrated by measurement experts Hedges and Olkin (1980).

312

RESULTS

Taylor (2002a) found that effect sizes for reading comprehension are significantly higher when L2 texts are accompanied by L1 glosses than when they are not accompanied by glosses, overall effect size of .49 (small). Similarly, the present study has found that the effect of L1 glosses on L2 reading comprehension is even more pronounced than reported in previous research. As shown in Table 1, the overall effect size was .56 (medium), which means that L2 readers who have L1 glosses will perform on average over half a standard deviation higher than those who do not have access to such glosses. This is the underlying question upon which a comparison between CALL and non-CALL L1 glossing can be based. These results simply indicate that, in general, learners provided with L1 glosses perform consistently better than those without such aids.

Table 1

Descriptive Statistics

The main question of the present meta-analysis is whether a significant difference exists between studies that have CALL L1 glosses versus those that have traditional L1 glosses. A large effect size was observed for CALL L1 outcomes

313

(g = 1.09), and a small effect size was obtained for the traditional glossing group (g = .39) with a significant difference between the two groups (p = .0001) (see Table 2). It is evident that CALL L1 glosses have a much stronger effect on L2 reading comprehension than traditional L1 glosses.

Table 2

Meta-analysis for CALL

	Effect size and 95% confidence interval
Group	Point estimate	Standard error	Lower limit	Upper limit
CALL (k = 4)	1.0946	.1576	.7837	1.4055
Classroom (k = 14)	.3868	.0786	.2325	.5410
Combined	.5277	.0703	.3897	.6657
			Summary of fit statistics
Source		Q value	DF	P value
Between classes		16.1550	1.0000	.0001

DISCUSSION

CALL L1 glossing is the most effective means by which L2 learners can access the greatest amount of text. Although some studies have not obtained significant results for CALL L1 glossing (e.g., Goyette, 1995), nonsignificant findings are largely the result of Type II errors. As Taylor (2002a) and the analysis presented here show, CALL L1 glossing in L2 reading comprehension significantly enhances L2 reading comprehension. However, despite these robust results, CALL L1 glosses may not always be effective in every L2 learning situation. There are both advantages and disadvantages to using them. Arguments for and against the use of CALL L1 glosses can be summarized as in Table3.

Table 3

Arguments for and against Using CALL L1 Glosses

Arguments for using CALL L1 glosses	Arguments against using CALL L1 glosses
1. Like a bilingual dictionary	1. Naive lexical hypothesis
2. Faster access	2. Too little effort; not enough deep processing
3. Motivational; encourages look-up behavior	3. Stoehr (1999), what about paraphrases?
4. Better comprehension in short term	4. L1 to be avoided
5. Attentional resources can be used elsewhere	5. Creates bad habits
6. Pointless to use other glosses	6. Long term not conclusive

314

Arguments for the Use of CALL L1 Glosses

Most L2 learners use bilingual dictionaries. The process of looking up words is simply made much easier and more time efficient with computer assistance. Because of such fast access, readers may look up more words. Faster access may also motivate readers to feel more capable of tackling L2 texts. Bell and LeBlanc (2000) suggested that L1 glossing would “encourage more positive anticipation of reading in the L2 or make students more comfortable when facing a reading selection, teachers will want a text that provides vocabulary helps in the students' L1” (p. 282). Such positive anticipation can be important, especially for beginning-level courses in which retention is important for learners to be able to continue their study of the L2, not to mention the survival of the L2 program itself. This is not to say that CALL L1 glossing will solve all retention problems. It can, however, be an important motivational component in the CALL lab environment by providing tailored and rapid assistance. Bottom-up support clearly complements the functioning of top-down processing (Birch, 2002; Eskey, 1987). Because L2 readers understand more words with L1 glosses, they should be able to focus more on other reading strategies such as activating background knowledge, analyzing text structure, and using semantic-mapping strategies.

Learners generally want to access meaning as quickly as possible. For this reason, it is arguably pointless to use other glosses along with the L1 (including those in the L2). Why take time programming or creating other ancillary aids if students will not use them? Why not provide the L2 reader with what is desired, aids that provide the quickest access to meaning? Of course, giving learners what they want may not always be good practice. Also, obtaining the meaning of a word quickly will not always ensure that the word has been understood, as is discussed below.

Arguments against the Use of CALL L1 Glosses

A pedagogically unhelpful phenomenon can occur with glossing, in which the “naive lexical hypothesis” (Bland, Noblitt, Armington & Gay, 1990) is reinforced, that is, learners assume that there is a simple one-to-one correspondence between words in the L1 and L2. Learners may well develop semantic comprehension of a text, but their understanding of the meaning of the text could be quite different from that of an L2 native speaker. To the extent that L1 glossing is used to make authentic (i.e., semantically rich and subtle) texts accessible to readers, this could in the long term prove counterproductive. Of course, knowing whether L2 learners really have understood the text and lexical items in the text is also an important issue, partially resolved by quality assessment measures of comprehension.

In addition, the availability of rapid, tailored access to CALL L1 glosses can minimize learners' efforts and, as a consequence, hamper their deeper processing of the text. If learners can access L2 paraphrases only, perhaps they will have to think a bit harder about the meaning and therefore remember words better in the long term (Stoehr, 1999). Of course, all the studies in this meta-analysis

315

were short-term studies. It is possible that other forms of help, such as L2 paraphrases, are more effective in facilitating long-term comprehension.

Another issue to be addressed is the use of the L1. The use of the L1 in software is generally viewed as something to be avoided because it can potentially confuse L2 learners as to which language they are expected to use in the computer lab (Taylor, 2002b). Poor perceptions of L2 use can develop and generalize to other contexts besides CALL, such as the classroom in which the L2 should of course be emphasized. Thus, CALL L1 glosses may be useful only when restricted to individualized, out-of-classroom settings.

The last issue is the absence of long-term studies in the literature. We do not know the long-term effects of CALL L1 glosses on L2 reading comprehension. Thus, it is difficult to become fully committed to the use of L1 glosses because the question of their long-term effectiveness remains unanswered. It is possible that the use of CALL L1 glosses has many short-term benefits but few long-term rewards.

PEDAGOGICAL IMPLICATIONS AND FUTURE RESEARCH

The data from the studies we have discussed suggest that other ancillary online aids placed next to L1 glosses are essentially unused for L2 reading comprehension; they may be interesting and enjoyable, but readers simply do not use them if comprehension is their aim.

It is apparent that learners prefer CALL L1 glosses, but the question remains: Are they really helpful for learners in the long term? As Chapelle and Jamieson (1986) suggest, there are many variables involved in CALL such as length of study, level of reader, and level of text. It is quite possible that L1 glosses can become a crutch upon which L2 learners lean too heavily—especially at more advanced levels. Glossing could and probably should be phased out at higher levels, although this is largely untested. Furthermore, most of the studies we have reviewed are first- and second-year-level studies. More research should be conducted on the usefulness of L1 glosses in L2 reading for more advanced L2 learners. Future studies should determine the level at which L1 glosses can be considered to have the greatest effect on L2 reading comprehension. More studies on a text `fit' should also be conducted (for a traditional glossing example, see Joyce, 1997). These research directions could provide more information about which variables are more influential than others for efficient L2 reading comprehension.