Hajime Umeda
Suzuka International University
The purpose of this paper is two-fold: (1) to determine whether Japanese Teachers of English (JTs) and American Teachers of English (ATs) differ in their judgments of the acceptability of some ungrammatical English sentences; and (2) to what degree and in what respect they differ, if at all.
In order to determine this, a questionnaire has been prepared and data has been collected from 52 JTs and 32 ATs.
This questionnaire has 15 sentences which have been selected from several junior high school English textbooks currently in use in Japanese schools. The sentences include wh-questions, infinitive constructions, and passive structures among others. For each of the fifteen correct sentences, four ungrammatical structures have been supplied as alternatives. These alternatives are intended to have the same meanings as their correct corollaries but may vary in degree of acceptability. The subjects were asked to rate the alternatives on a scale of one to four (best, second best, third best, and worst). The data collected thus far have been statistically analyzed and investigated.
The 15 sets of questions were classified in five categories, depending on how many ungrammatical alternatives showed JT-AT correlation in terms of acceptability.
0. Introduction
Today the development of communicative competence is one of the most discussed topics in English teaching. In fact, it is hard to avoid articles on this theme or related topics in current English teaching journals and magazines. The profession is most certainly headed in a desirable direction, but one must not forget that a sound knowledge of grammar is vital in order to develop communicative competence. This paper deals with the evaluation of some ungrammatical English sentences that might be produced by Japanese students.
There has been similar research done in the past on this topic. Kanaya and Takanashi (1978), for example, claim that in grading ungrammatical English sentences, American students are more likely to emphasize meaning than form (grammar) while Japanese teachers of English tend to lend more importance to form. Schmitt (1993) has also proven through his experiments that Japanese teachers of English place greater emphasis on formal accuracy than Assistant English Teachers (AETs). That is to say, they pay more attention to grammatical errors than do their native-speaking counterparts.
1. Purposes
The purpose of this paper is twofold: (1) to investigate whether there is any difference in the judgment of acceptability of some ungrammatical English sentences between Japanese Teachers of English (JTs) and American Teachers of English (ATs); and (2) to investigate to what degree and in what respect they are different (if indeed there are any differences).
2. Subjects
In order to achieve the stated purpose, a questionnaire has been prepared (See Appendix.) and data has been collected from 52 JTs and 32 ATs. All of the JTs teach in either junior or senior high schools, while almost all of the ATs teach in Japanese or American universities.
3. Source of Questionnaire
Fifteen sentences were selected from the following seven junior high school English textbooks currently in use. The abbreviations and the grades are given in parentheses:
New Total English Course (NT 1, 2)
Everyday English (EE 2, 3)
New Horizon (NH 1, 2, 3)
New Crown (NC 2, 3)
One World (OW 1, 3)
Columbus English Course (CE 3)
Sunshine English Course (SE 1, 2)
4. Testing Procedure
For each of the fifteen correct sentences, we have supplied four alternative sentences. (See Appendix.) These alternatives are intended to mean the same as the correct sentences but may vary in degree of acceptability. The subjects are to rate "1" through "4" in the order of "best," "second best," "third best," and "worst" ("1" being "best" and "4" being "worst.").
5. Statistical Analysis
In order to find out how similarly or differently the two groups answer each question, Chi-square (x2) test was used. The critical value of x2 is 7.8147 at the level of 0.050, and one can conclude that if the value is smaller than 7.8147, there is a relationship between the two groups in their evaluation. To the right of each alternative sentence is shown the value of the x2 test. (See Appendix.)
6. Results and Observations
It is possible to observe the results by classifying the 15 items into five categories based on the number of alternatives and by showing correlation for each item.
Category 1
Each of the four alternatives has a relationship between the JTs and ATs. Items 1, 2, and 11 belong to this category. Grammatical structures involved are yes-no question, comparison, and relative clause, respectively.
49 JTs (94.2%) and 31 ATs (96.8%) rated (1a) as best, while 37 JTs (71.1%) and 26 ATs (81.2%) rated (1c) worst. In Item 2, 30 JTs (57.7%) and 21 ATs (65.6%) rated (2a) best in spite of the fact that "is" is eliminated from this sentence. Both groups rated (11c) best, whereas (11d) received the worst rating. 35 JTs (67.3%) and 26 ATs (81.2%) are in the former category as opposed to 43 JTs (82.7%) and 29 ATs (90.6%) in the latter category, respectively.
Category 2
Three out of four alternatives have a relationship when comparing the two groups. A tag question (Item 3) and a participial structure (Item 8) are in this category.
A majority of both JTs and ATs rated (3a) best and (3d) worst. However, they showed a difference in grading (3c). While 9 and 14 JTs (17.3% and 26.9%) rated the sentence as best and the second best respectively, none of the ATs rated it as best and only 2 of them (6.2%) said that it was second best.
It is not difficult to understand why many in each group considered (8d) best since this type of word order is often used in colloquial, conversational English, although there was some difference in the percentages (JTs 55.7%, ATs 78.1%). It is also interesting that 17 JTs (32.6%) rated (8b) as best and 26 of them (50%) rated it second best, but none of the ATs chose it as the best, and only 9 of them (28.1%) rated it as second best. It may be that native speakers of English see a greater difference in meaning between sentences with the -ing suffix and those without it [as in Item 8 and (8b)] than do many Japanese teachers.
Category 3
The x2 values are greater than the critical value of 7.8147 in (6a) and (6b); (7a) and (7b); and (9a) and (9d). This means that two alternatives are related, and the other two are not. Wh-question, to-infinitive construction and "It...to-infinitive" structure are involved in these items, respectively.
A great majority of both groups chose (6d) as best and (6c) as worst, but there were differences in (6a) and (6b). (6b) is an echo question which is often used colloquially. However, only 4 JTs (7.6%) chose this as best as opposed to 8 ATs (25%). On the other hand, JTs tended to rate higher those sentences beginning with wh-words such as (6d).
Although the two groups showed a definite correlation in the evaluation of (7d), their evaluation varied in (7a) and (7b). 25 JTs (48.1%) chose (7a) as best, but only 5 ATs (15.6%) joined them. Conversely, 17 ATs (53.1%) evaluated (7b) as best, and only 11 JTs (21.1%) agreed with them. Here again, one observes that ATs tended to emphasize meaning over structure.
In Item 9, a great majority of both JTs and ATs chose (9b) as best, and (9c) as worst, but their grade points showed a somewhat unbalanced distribution between the two groups in (9a) and (9d).
Category 4
Only one alternative sentence is correlated in Items 4, 5, 12, and 14, and they contain passive structure, contact clause, present perfect tense, and gerund, respectively.
In Item 4, 29 JTs (55.7%) rated (4b) as best, but only 5 ATs (15.6%) agreed. Conversely, 17 ATs (53.1%) said that (4a) was most acceptable while only 8 JTs (15.3%) did.
39 JTs (75.0%) and 25 ATs (78.1%) agreed that (5c) was the most acceptable choice, and thus they show a very high correlation. However, they disagree in three other items. Specifically, the difference is more conspicuous in (5b) and (5d). For example, 15 JTs (28.8%) chose (5b) as third best, and 29 of them (55.7%) as worst, while as many as 25 ATs (78.1%) rated it as third best and only 4 of them (12.5%) as worst. A similar difference is observed in (5d) between the two groups.
Sentence (12c) has a good correlation between the two groups. However, 19 JTs (36.5%) rated (12a) as best, and only 3 ATs (9.3%) joined in this judgment. Disparities in (12b) and (12d) are even greater between the two groups.
In Item 14, many subjects in both groups seemed to have difficulty judging which alternative was best and which was worst, judging from the considerable disparity between the two groups. One interesting fact is that 17 JTs (32.6%) and 6 ATs (18.7%) rated (14c) as best, although the use of to-infinitive form is wrong in this context.
Category 5
Items 10, 13, and 15 belong to this category, two of which are wh-questions. JTs tended to prefer placing wh-words at the beginnings of sentences, while ATs did not mind putting them at the ends as long as the sentences sounded more natural. Item 10 is a typical example of this. 33 JTs (63.4%) thought (10a) best as opposed to 8 ATs (25%). On the other hand, 20 ATs (62.5%) rated (10b) as best in contrast to 11 JTs (21.1%). Furthermore, (10c) is the least acceptable sentence to 44 JTs (84.6%), but only 2 ATs (6.2%) chose this as worst, a great difference in judgment between the two groups.
The same tendency is observed in Item 15. While 46 JTs (88.4%) judged (15a) most acceptable, 12 ATs (37.5%) chose it, while the remaining 20 ATs (62.5%) preferred the echo question (15b). The worst sentence was (15c) for 35 JTs (67.3%), but (15d) was worst for 25 ATs (78.1%).
Item 13 does not show as much difference as the other two items. But similar distributional disparities are observed among all the four alternative sentences.
7. Conclusion
A noticeable difference was found in the acceptability judgment between the JTs and ATs as observed in the values of Chi-square test. Mathematically speaking, the two groups disagreed in judgment in 32 sentences (54.4%), which is more than half of the 60 alternative sentences, although it must be admitted that the other 28 sentences showed good correlation.
It was also discovered that JTs tended to emphasize grammar accuracy over meaning as observed most typically in the structures of the wh-questions.
This paper has compared similarities and differences in acceptability judgments between Japanese teachers of English and their American counterparts. A follow- up study will investigate whether non-native speakers of English are more critical of non-standard language among their students than are native speaking teachers as reported, for example, by Fayer and Krasinski (1987).
References
Fayer, J. and E. Krasinski. (1987). "Native and Non-native Judgments of Intelligibility and Irritation," Language Learning, 37, 3.
Hatch, E. and A. Lazaraton. (1991). The Research Manual: Design and Statistics for Applied Linguistics. Newbury House.
Kanatani, K. and Y. Takanashi. (1978). "Error Analysis: Eigo kyouiku ni okeru ayamari no hyouka (An Evaluation of errors in English Teaching)," Tokyo Daigaku Kyouiku-gakubu Kiyou.
Kiyokawa, Hideo. (1990). Eigo kyouiku kenkyuu nyuumon. (An Introduction to Research in English Teaching.) Taishuukan.
Schmitt, Norbert. (1993). "Comparing Native and Nonnative Teachers' Evaluations of Error Seriousness," JALT Journal 15, 2.
Umeda, Hajime (1994). ﾒWh- gimon-bun no hyouka: nichi-bei eigo kyoushi-kan no hikaku (A Comparative Study of Acceptability Judgment on Wh- Questions between Japanese Teachers of English and their American Counterparts),ﾓ Suzuka International Forum 1.