|
The measurement of change in English language proficiency
Sasikala Nallaya
The University of South Australia
This study investigated the measurement of change in English language proficiency levels of pre-service and in-service teachers enrolled in a public university in Malaysia. Path analysis was used to measure change on three occasions by examining the effects of student characteristics, learning needs and multimodal technology on proficiency levels. An English Proficiency Test was used as the outcome variable. The results indicated that two direct factors influenced this change, namely, the Malaysian University English Test (MUET) and Multimodal and Language Proficiency. On the third occasion of this study, learning needs and students' State of origin reflected a mediated effect on the English Proficiency Test through Multimodal and Language Proficiency. The findings also indicated that students' learning needs indirectly influenced their English language proficiency test scores on the third occasion. This study confirms that the greater use of the procedures of Multimodal and Language Proficiency facilitated the development of English language proficiency.
The learning of English language, however, is perceived to be a difficult process. Factors such as motivation, aptitude, age and socio-cultural background are known to influence learning outcomes (Balderrama & Diaz-Rico, 2006). Furthermore, the views of the most important stakeholders, namely students, are often not considered when courses are designed thus resulting in the needs of the learner not complementing course aims and objectives (Dudley-Evans, 2001; Leki, 2000). It is widely perceived that most English language programs merely expect learners to integrate into existing academic discourses without taking into consideration their prior experience and background. It is important to make the teaching and learning process simpler for English language learners. Creating a language learning environment that is authentic, flexible, interactive, systematic and grounded on learners' previous knowledge and experience will facilitate the learning process. This is especially important as communication in everyday life is increasingly becoming visual or multimodal in nature (Thesen, 2001).
The measurement of change in educational research is important. When students learn, they acquire new skills that result in their intellectual growth and change. By measuring this change over time, it is possible to identify the trends in education. Conversely, it is only by measuring individual change that a student's progress and the effectiveness of the educational system can be identified (Willett, 1994). Although, the measurement of tertiary students' English language proficiency has received much attention, not many studies have investigated the relationship between learning needs, technology and English language proficiency, especially in courses such as English for Academic Purposes (EAP). In view of this, the present study was undertaken to investigate, 1) the direct factors that influence change in English language proficiency (such as age, gender, ethnicity, state of origin, faculty, Malaysian University English Test and learning needs); 2) the mediated factors that operate through technology to influence change in English language proficiency (such as learning needs and state of origin); 3) the influence of learning needs on other variables; and 4) the factors that influence change in Multimodal and Language proficiency.
This study was conducted in the context of low English language proficiency of Malaysian students. Limited English language proficiency contributed to graduate unemployment and concern among politicians and academicians. The seriousness of these issues was clear when it was highlighted that 40,000 graduates from public universities in Malaysia were unemployed in 2002 due to their limited English language proficiency (Gill, 2004). Malaysian students have no choice but to become proficient in English language usage in order to be included in the academic discourse community while in the university and for employment once outside.
In an academic setting, English language proficiency is almost always assessed based on listening, speaking, reading and writing skills. A learner has no choice but to display competence in these skills in order to be perceived as being proficient by course administrators. An English language proficiency test is usually the standard instrument that is used to identify the level of proficiency at which a learner is located. Thus in this study, an English language proficiency test adapted from two tests labelled 'Test for English Proficiency' designed by the Educational Testing Service in the United Kingdom and the 'The Assessment Test' (2002) employed in 'The Assessment of Pupils' Skills in English in Eight European Countries' was employed. The test was administered on three occasions (at the beginning, at the end, and three months after the students had completed the course) to identify the levels of proficiency at these successive stages.
It is possible that students' are unable to obtain a competent level of English language proficiency due to a mismatch between course objectives and learning needs. It is argued that the effectiveness of an educational program was dependent on "planning, implementation, evaluation and change" (Burgess & Owens, 2003, p. 3). For this process to unfold successfully, a needs analysis would be necessary. Needs analysis has been defined as a systematic set of procedures for identifying or the gathering of specific knowledge regarding learner needs. There has been much debate on why a needs analysis should be the initial step in course design. There appeared to be no evidence that a needs analysis was conducted in the research setting. Thus, the views of the most important stakeholders, namely, the students, were not taken into consideration in designing the course.
Xiao's (2003) Needs Analysis Questionnaire was adapted and validated to collect data in the current study because it asked a comprehensive list of questions with regard to learners' perceptions of their needs. Furthermore, the themes included in the questionnaire were highly relevant for the current study. The questionnaire was validated through Confirmatory Factor Analysis (CFA). Nallaya (2010) provides a detailed description of the processes undertaken.
Technology was able to create a language learning environment that was interactive, collaborative as well as highly situated (Clovis, 1997; Kasper, 2000; New London Group, 1996; Tan, 2006; Warschauer, 2000). For example, the email, chat room and discussion board provided English language learners with the opportunities to interact with other speakers of the target language in a stress-free environment. Technology also allowed learners to write in real life contexts. The existence of 'virtual tours' and 'Webquest' web sites, for example, allowed for language learning to take place through simulated activities (Lacina, 2005). Research had shown that learners had no problems moving from one medium of communication to another for meaning, but teachers had tended to privilege "continuous, uninterrupted prose texts" (Millard & Marsh, 2002, p. 55). It was thus important to investigate whether technology and multimodal texts impacted the development of English language proficiency in the research context under investigation.
The Language Centre in the Hong Kong University of Science and Technology's (LTCFIT) Placement Facility and Needs Analysis (2006) Questionnaire was adapted to collect data for the present study and was named the Multimodal and Language Proficiency Questionnaire (refer to Nallaya, 2010 for a detailed description of the validation processes undertaken). The questionnaire was divided into eight sections. Two of the sections were used to collect personal information and academic qualifications. The other six sections collected information with regard to, (a) use of technology, (b) frequency of multimodal texts use in English for activities in daily life, (c) frequency of English use for university activities, (d) perception of proficiency for the use of multimodal texts in English for activities in daily life, (e) perception of English proficiency for university activities, and (f) perception of English proficiency for every day communication. Confirmatory Factor Analysis (CFA) was conducted to investigate the way in which observed variables were related to particular factors.
Bereiter (2002, p. 64) refers to the ideas and relationships of World 3 as abstract objects that are "conceptual artefacts". The formation and testing of these "conceptual artefacts" have been traced back to the writings of Archimedes (see Netz & Noel, 2007, p. 291) and are now widely referred to as "models" or "hypotheses" and sometimes as "laws". The analytical procedures involved in the examination of these models and hypotheses in the research study presented in this paper, arise from the work of R. Fisher and are involved in "a modelling approach" that is employed in the social and behavioural sciences. Different empirical strategies are employed in different disciplinary areas, alongside rational argument, and controlled experimentation in order to examine the strength of models and hypotheses. These models are "conceptual artefacts" from the basis of knowledge and are human creations that can be subjected to examination either empirically or rationally. They are fallible but improvable, and are independent of their creation. Vygotsky (1978) contended that cognitive structures were formed by individuals that arose from the context in which they were located and were socially and culturally situated. The idea of 'constructivism' arose from the studies of Piaget and developed into social constructivism.
The study reported in this paper was an exercise in the field of social constructivism in which students were advancing and developing their communication skills in English. Vygotsky (1978) argued that all cognitive functions originated as social functions and that learning was not simply the assimilation and accommodation of new knowledge by learners; it was the process by which learners were integrated into a 'knowledge community'. The knowledge community in this study is characterised by 'Academic Discourse'. Central to the processes of teaching and learning is Vygotsky's concept of Zone of Proximal Development (ZPD) which he used to refer to the difference between a currently achieved developmental level and the potential development of an individual that was considered in the design of instruction. Scaffolding was defined as a process "of negotiated interaction in which experts first assess the learner's level of competence and determine the types of assistance they need to accomplish a particular task" (Hall, 2001, p. 31). These concepts were acknowledged and examined in the current study. They are graphically illustrated in Figure 1.
Figure 1: The nature and processes of the current study
Adapted from Mansheng and Keeves (2003, p. 1033)
Although the theory of social constructivism was relevant to the research study, it was acknowledged that this theory is limited in that it does not recognise the need to test ideas and relationships in the real world. Knowledge must always be examined and tested against evidence obtained from the real world. Consequently, Keeves (2002) asserted that a 'modelling approach' in both the advancement of knowledge and in learning was necessary so that the constructed models were tested. Keeves (2002, p. 115) emphasised that a "modelling approach requires that both individual and corporate knowledge must be tested not only for coherence but also against evidence obtained from the real world in which human beings are living and undertaking their inquiries". The present research study sought to accomplish the procedures involved in a 'modelling approach' and employ a social constructivist approach in the teaching and learning undertaken with students in a situation associated with the acquisition of communicative skills or the English language in the Real World of World 1. The processes of learning took place in World 2 and in interaction with the Social World. Multimodal technology and multimodal texts extended the domain of the Social World and through its interaction with the Human Mind of World 2.
The respondents of this study were selected using the procedures and criteria of purposeful sampling. Patton (1990) suggested that the power of purposeful sampling lay in selecting information-rich cases for in-depth study. Information-rich cases were those from which one could learn a great deal about issues of central importance to the study. Purposeful sampling was used for the context of this study as it was a sampling strategy that was commonly used in exploratory research (Neuman, 2006). A total of 316 students enrolled in the CE1 course representing a little below 80 per cent of the chosen sample, were respondents in this study. These respondents displayed a common characteristic, namely, low level of English language proficiency. They were selected using the Course coordinator's judgement.
Out of the 316 CE1 students under survey in this study, 70 were male and 246 were female. They were between 19 years and 29 years of age. The respondents in this study can be classified into different ethnic groups. The majority of the respondents were Malays, totalling 294. The second largest ethnic group were the Indigenous students who total 18. In Malaysia, the Indigenous students come from the East Coast States of Sabah and Sarawak. The composition of the Indigenous people is made up of the following ethnic backgrounds, namely, Bajau, Bidayuh, Iban, Kadazan-Dusun, Dayak, Melanau. Like Malays, the Indigenous people are referred to as 'Bumiputera' or 'Sons of the land' to distinguish them from the other racial groups, such as the Chinese and Indian people. The Malays and Indigenous people in Malaysia are different in that the former embrace Islamic beliefs while the latter practise Islam, Christianity or Buddhism. Chinese and Indian students are not well represented in this study. However, the composition of these ethnic groups is representative of the students enrolled in the CE1 course. The students who enrol in this course are heavily represented by the Malay ethnic group. Chinese and Indian students do not enrol in large numbers. This pattern of representation also reflects the overall enrolment of students in the University, as well as the composition of ethnic groups in Malaysia. The estimated population for Malaysia in 2007 was 27.17 million people of whom 93 per cent were citizens and 7 per cent were non-citizens. Of this population, the Bumiputeras made up 66.4 per cent, Chinese - 24.8 per cent, Indians - 7.5 per cent and others - 1.3 per cent.
Out of the 316 respondents in this study, 112 listed the Eastern States as their State of origin, with 87 respondents from the Northern States and 54 from the East Coast States. The Southern and Central States were both represented by 33 and 30 students respectively. The statistics for respondents' State of origin reflected the overall student population in the research setting. The majority of the students studying in this research setting generally came from the Eastern and Northern States as well as the East Coast States. The Social Sciences and Humanities faculty had the most respondents totalling 53 per cent. The faculty that had the second highest number of respondents was Cognitive Science and Human Development with 19 per cent. This was followed by Business and Commerce with 14 per cent and Arts and Music with 12 per cent respondents. The Faculty of IT and Communication had the least representation of 2 per cent. Since 95 per cent of the respondents were from non-English-speaking backgrounds and only 5 per cent came from English-speaking homes, this composition of the sample was likely to place some restrictions on the analyses that could be carried out in this study. However, as this study was focused on the development of English language proficiency and was longitudinal in nature, it was necessary to consider the family language backgrounds of the small group of English-speaking students.
PLSPath analysis is also referred to as 'soft modelling', as it is an appropriate statistical technique for research conditions where mild supplementary assumptions were required (Sellin, 1989). The PLSPath analysis technique was also suitable for smaller models and smaller data sets and for the explanatory analysis of the situation under survey. PLSPath models involved both observed or manifest variables (MVs) and unobservable or latent variables (LVs). The inner (structural) model specified relationships between the LVs. The outer (measurement) model specified the relationships between MVs and LVs. The basic PLS design assumed "recursive inner structures. In other words, each inner model relation formally constituted a multiple regression equation, and the inner model coefficients were estimated by Ordinary Least Squares (OLS) regression applied to each inner model equation separately (Sellin, 1989, p. 21).
The tested path analysis models included both direct and indirect or mediating effects. The models of individual factors influencing the impact of multimodal texts on the development of English language proficiency indicated that there were 21 latent variables (LVs). They are listed in Appendix 1 in which the acronym of each latent variable is given, followed by its descriptive name and the manifest variables and their descriptors that are associated with each latent variable. In addition, information is given on whether the latent (unobservable) variable is formed in the 'inward mode', or reflected in the 'outward mode', or involves the 'unity mode' where there is only one manifest (observed) variable. The outcome or criterion variable is the English Proficiency Test score on Occasion 3 (EPT3).
In the inner (Structural) model, 20 latent variables are hypothesised with each of them involving one or more MVs. Therefore, in the examination of the final causal model, there are 20 LVs and 48 MVs. The hypothesised path model for estimating the impact of multimodal texts on the development of English language proficiency is presented in Figure 2. The latent (unobservable) variables in the path diagram can be grouped into two classes: (a) those that do not receive causal inputs from any other variables in the path diagram referred to as exogenous (independent) variables, and (b) those that receive one or more of such causal inputs referred to as endogenous (dependent) variables. In the path diagram presented in Figure 2, there are five exogenous variables and 15 endogenous variables. The exogenous (independent) variables are, namely, Age, Ethnicity, State, Faculty, and Gender. The endogenous (dependent) variables are, Malaysian University English Test, Learning Needs, Student-Centred Teaching Approach, Use of Technology on Occasions 1, 2 and 3, Multimodal and Language Proficiency on Occasions 1, 2 and 3, Frequency of English Use on Occasions 1, 2 and 3, English Proficiency Test Scores on Occasions 1, 2 and 3 (see Appendix 1 for the acronyms). One of the latent (unobservable) variables 'ENGNENG' (acronym for English or non-English Speaking Background) was dropped from the analysis because no causal paths could be initiated due to the low fit indices. There were initially 49 manifest (observed) variables but the variable 'Background' was dropped when 'ENGNENG' (representing students' family language background) was dropped from the analysis.
Several variables were assessed on three occasions, namely, Use of Technology, Multimodal and Language Proficiency, Frequency of English Language Use and English Proficiency Test. The English Proficiency Test on Occasion 3 is the criterion variable. Of particular interest were the variables associated with respondents' Learning Needs (NEEDS), the Malaysian University English Test (MUET) and Attitudes towards Student-centred Teaching Approach (STCAP), although some of these variables are only examined at the initial stages of the analyses. In Figure 2, the manifest (observed) variables are displayed with small rectangular boxes and the latent (unobservable) variables are shown with oval shapes. While the path diagram presented in Figure 2 has a complex structure, the situation under investigation in the real world is a highly complex one, in part because it is examining change between occasions. The results of the analyses carried out employing PLSPath Version 3.01 (Sellin, 1989) are reported in two parts: (a) the measurement (outer) model; and (b) the structural (inner) model.
One of the limitations of the PLSPath program is that there is no formal testing of statistical significance (Darmawan, 2003). This is because it cannot be assumed that the conditions for the formal testing of statistical significance apply in this situation, since the students are nested within classroom groups. Moreover, the normal distribution of residuals may not apply in this exploratory analysis. However, Sellin (1989) suggested that the estimates of the standard errors through the jack-knife estimates could be used in the examination of the significance of the results. For the statistical evaluation of PLS models, Sellin (1989) proposed the use of the 'Stone-Geisser' test of predictive relevance (see also Wold, 1982). According to Sellin (1989, p. 20) the Stone-Geisser test basically produces jack-knife estimates of residual variances which could be used for evaluating the predictive power of the inner (structural) model and the outer (measurement) model relationships for estimating jack-knife standard errors of point estimates. The basic idea of the Stone-Geisser test is
to omit or 'blindfold' one case at a time, to re-estimate model parameters on the basis of the remaining n-1 cases, and to reconstruct or predict the omitted case value of the dependent variables using the re-estimated parameters. (Sellin, 1989, p. 22)For the outer model there are five indices that can be used to reject or delete a manifest (observed) variable from a latent (unobservable) variable, it reflects or forms (Nallaya, 2010: Table 9.2, p.177). The five indices are: (a) weight; (b) loading, (c) communality, (d) redundancy, and (e) tolerance. Weight indicates the strength of the regression type relationship between manifest (observed) variables and latent (unobservable) variables with the inward or formative mode. The relationship between a manifest variable and its corresponding latent variable with the inward mode can be considered adequate if the value of the weight is equal to or larger than 0.10, with the assumption of a sample size of approximately 316 cases (Sellin & Keeves, 1997).
A loading indicates the strength of the factor analytic type relationship between manifest (observed) variables and latent (unobservable) variables with the outward or reflective mode. It is commonly accepted (see Sellin & Keeves, 1997), that a loading of 0.40 or larger can be considered adequate. The outward mode type relationships between the manifest variables and latent variables in the path diagram indicated loadings larger than 0.40 for Learning Needs, Use of Technology on Occasions 1, 2 and 3, Multimodal and Language Proficiency on Occasions 1, 2 and 3 (Nallaya, 2010: Table 9.2, p.177).
Where a weight of zero is recorded, the manifest (observed) variable acts as the reference variable in the use of dummy variable regression analysis. However, for the latent variable Faculty, the Information and Communication Technology (ICT) group was very small. Moreover, for the variable Ethnicity (ETHNIC), only the Malay group was of a substantial size. The weights for the hypothesised outer (measurement) model in the present research study, for variables ETHNIC, STATE and FACULTY that had inward modes, were all higher than 0.10 (Nallaya, 2010: Table 9.2, p.177).
Communality indices indicate the strength of the outer (measurement) model and are calculated as the squared correlations between the manifest (observed) variables and their corresponding latent (unobservable) variables. The strength of the outer model is measured by the average communalities (see Falk, 1987) which is considered to be adequately strong if the average communality value for each latent variable is greater than 0.30. The outer model communality in the present research study was adequate and the average communality for all the variables was greater than 0.30 (Nallaya, 2010: Table 9.2, p.177).
Redundancy indicates "the joint predictive power of inner model and outer model relationships as estimated for a given data set" (Sellin, 1989, p. 12). Tolerance indicates the possibility of seriously damaging multicollinearity (correlation between independent variables) within the corresponding block of manifest variables (Darmawan, 2003). A tolerance value of 0.50 or higher indicates possible multicollinearity within a block of MVs. However, these effects can be considered damaging for the formative (inward) mode but not for the reflective (outward) mode. Consequently, although some of the tolerance indices in the hypothesised model for latent variables in the outward or reflective mode were higher than 0.50, this was of no concern if the latent (unobservable) variable were not in the formative or inward mode.
Figure 2: Path model
(a higher resolution version is available in Appendix 2)
The discussion in the following section is carried out with reference to the path diagram in Figure 2. The path diagram in Figure 2 was developed based on the causal framework that contained the following antecedent variables that were exogenous (independent), namely, (a) AGE, (b) GENDER, (c) ETHNICITY, (d) FACULTY, and (e) STATE. In addition there were two other antecedent variables that were endogenous (dependent), namely, (a) MUET, and (b) NEEDS. There were also several variables that were measured on three occasions: (a) Multimodal and Language Proficiency (MMLP), (b) Use of Technology (USETEC), (c) Frequency of English Language Use for University Activities (FRENGUS), and (d) English Proficiency Test (EPT). The crux of this research study was to investigate the effects of all these variables on English language proficiency on the three occasions with EPT3 as the criterion variable, since EPT3 involved students' EPT scores on Occasion 3. In order for the investigation of change between occasions to take place, it was necessary to adhere to a system that enabled the analysis of the effects of the latent variables on the criterion variable.
The discussion that follows seeks to explain the effects of the variables under study with strict reference to the three main facets of this study: (a) language proficiency, (b) needs assessment, and (c) the effectiveness of learning language communication skills through the use of multimodal texts. These analyses were based on the causal relationships of (b) −> (c) −> (a). While there were very many causal path relationships involved in the path diagram shown in Figure 2, the discussion that follows is specifically interested in the effects of the latent (unobservable) variables on change with respect to those latent variables that were measured on the three occasions.
The results from the analyses recorded that the beta coefficients were numerically greater than or equal to 0.10 for all the variables except for the effect of MUET on MMLP2 and FRENGUS1 on FRENGUS2. Moreover, each beta coefficient retained in the final model was at least 1.66 times its jack-knife standard error, which is the cut-off value for retaining or removing a path. The next section discusses the analysis with reference to Figure 2 and the estimated effects (see Nallaya, 2010: Table 9.3, p.181). The discussion focuses on the effects of the latent variables included in the path model on the endogenous (dependent) variables in the model. The discussion is categorised into four sections: (a) What are the direct factors that influence change in EPT? (b) What are the mediated factors that operate through MMLP3 to influence change in EPT3? (c) What is the influence of learning needs on other variables? and (d) What are the factors that influence change in Multimodal and Language Proficiency? The numbers in parentheses associated with the acronyms indicate the position of a latent variable in the path diagram recorded in Figure 2 and are provided to assist in reading the diagram.
Figure 3: Path diagram illustrating the direct factors influencing change in EPT3
(JknStd errors are recorded in parentheses)
The variable Multimodal and Language Proficiency at Occasion 3 (18)MMLP3 was found to have a positive path relationship (beta = 0.13) with English Proficiency Test Performance on Occasion 3 (20)EPT3 in contrast to the lack of such relationships on both Occasions 1 and 2. This suggested that the continuous use of the procedures of Multimodal and Language Proficiency appeared to have facilitated the development of English Language Proficiency during the course.
Moreover, the relationship recorded for the effects of performance on the Malaysian University English Test (6)MUET on EPT1 prior to the commencement of the course that was estimated by the path coefficient to be small and marginally significant (( = 0.13) indicated that students who performed well on the Malaysian University English Test (MUET) were the students who were showing the higher levels of English Language Proficiency on Occasion 1, after allowance for the influences of the other factors involved in the causal path model. Moreover, a similar effect operated between the Malaysian University English Test and performance on English Proficiency Test at Occasion 2 (beta = 0.15). However, on Occasion 3 the effect of MUET on English Proficiency Test (20)EPT3 was negative (beta = -0.10). This indicated that after the three month semester break, the students who did well on the EPT on Occasions 1 and 2, were no longer the higher performers on Occasion 3, while the students who initially were lower on MUET, gained in performance after the holidays. This suggested that the lower performers maintained their efforts to improve their performance after completing the course.
Figure 4: Path diagram illustrating the mediated factors that operate through MMLP3 to influence change in EPT3
(JknStd errors are recorded in parentheses)
In order to understand these two mediated effects, it is useful to answer two questions.
The learning needs of the Communicative English One (CE1) students influenced:
Figure 5: Path diagram illustrating the influence of learning needs on other variables
(JknStd errors are recorded in parentheses)
Students' Learning Needs (7)NEEDS influenced Use of Technology (13)USETEC2 on Occasion 2 (beta = 0.17). Students with a higher level of Learning Needs, used technology more for university activities (uni2) and for personal use (persnl2) on Occasion 2 of this research study.
This is understandable because on Occasion 2 of this research study, students had just completed their CE1 course. The regular use of technology to search for resources to complete assignments and communicate with family and friends who were back in their respective home States during the CE1 course, confirmed that the students were likely to continue using technology until they left for their semester break.
Similarly, students' Learning Needs (7)NEEDS also influenced the Frequency of English Language Use for University Activities (15)FRENGUS2 on Occasion 2 (beta = -0.13). It is interesting to note that students with higher Learning Needs used English less frequently for University Activities on Occasion 2. It was probable that even though students used technology on Occasion 2, they used Bahasa Melayu and not English because of their study program requirements. However, it was also probable that students did not use English for University Activities frequently but did so for other recreational activities such as chatting in chat rooms and watching movies. Moreover, even though students with higher Learning Needs did not Use English frequently for University Activities on Occasion 2, their Multimodal and Language Proficiency (18)MMLP3 improved on Occasion 3 (beta = 0.13), suggesting that these students continued to use technology as well as multimodal texts in the English language during the holidays. In this way, Learning Needs had a mediating effect on EPT3 operating through MMLP3. This indicated that the CE1 students' English language proficiency improved in line with their learning needs.
Respondents' State of origin and students' Use of Technology both directly influenced Multimodal and Language Proficiency. In addition change was recorded over the three occasions. The variable (4)STATE, indicated a reversal on Occasion 3. The Northern and Central States performed better on Multimodal and Language Proficiency compared to the East Coast and Southern States on Occasion 3. On Occasion 1, it was the East Coast and Southern States who performed better and there was no effect recorded on Occasion 2. Similarly, there was also a reversal for USETEC on Occasion 2. On Occasion 1, the students who used technology less had higher Multimodal and Language Proficiency. A possible explanation for this could be that technology and multimodal texts were popularly used before students enrolled for the course and this impacted on their Multimodal and Language Proficiency when data was collected on Occasion 1.
However, this changed on Occasion 2. On Occasion 2, the students who used technology less, had a lower level of Multimodal and Language Proficiency compared to students who used technology more, who had higher Multimodal and Language Proficiency scores. As students used technology less while enrolled for the course, their Multimodal and Language Proficiency declined. The variable (3)ETHNIC did not appear to influence MMLP after Occasion 1 and the influence of USETEC disappeared after Occasion 2. The effect of the variable (7)NEEDS only appeared after the completion of the CE1 course on Occasion 3, but it did influence USETEC on Occasion 2. This suggested that students, who during the CE1 course and during the vacation, recognised the role that technology can play in meeting their needs to raise their level of English Language Proficiency, made efforts to increase their performance through the increased use of multimodal technology. As a consequence they recorded a higher level of achievement on the English Proficiency Test on Occasion 3.
For the second question, 'What are the mediated factors that operate through technology to influence change in English language proficiency?', it was found that NEEDS and STATE reflected mediated effects on EPT3 operating through technology to influence change in English language proficiency.
For the question 'What is the influence of Learning Needs on Other Variables?', it was found that the Use of Technology on Occasion 2 (USETEC2), Frequency of English Language Use for University Activities on Occasion 2 (FRENGUS2) and Multimodal and Language Proficiency on Occasion 3 (MMLP3) were influenced by the CE1 students' learning needs.
For the question, 'What were the factors that were influencing change in Multimodal and Language Proficiency, it was found that respondents' State of origin and students' Use of Technology both directly influenced Multimodal and Language Proficiency. In addition change was recorded over the three occasions. The variable (4)STATE, indicated a reversal on Occasion 3. The Northern and Central States performed better on Multimodal and Language Proficiency compared to the East Coast and Southern States on Occasion 3. On Occasion 1, it was the East Coast and Southern States who performed better and there was no effect recorded on Occasion 2. Similarly, there was also a reversal for USETEC on Occasion 2. On Occasion 1, the students who used technology less had higher Multimodal and Language Proficiency. However, this changed on Occasion 2. On Occasion 2, the students who used technology less, had a lower level of Multimodal and Language Proficiency compared to students who used technology more, who had higher Multimodal and Language Proficiency scores. The variable (3)ETHNIC did not appear to influence MMLP after Occasion 1 and the influence of USETEC disappeared after Occasion 2. The effect of the variable (7)NEEDS only appeared after the completion of the CE1 course on Occasion 3, but it did influence USETEC on Occasion 2.
This suggested that those students who, during the CE1 course and during the vacation, recognised the role that technology could play in meeting their needs to raise their level of English Language Proficiency, made efforts to increase their performance through the increased use of multimodal technology. As a consequence they recorded a higher level of achievement on the English Proficiency Test on Occasion 3.
Students can experience extensive informal learning through the use of computer-based technology. Multimodal technology increases the informal use of learning English as a foreign language alongside formal instruction in a classroom. This leads to the idea that the use of technology can supplement the learning of a foreign language in a way similar to the learning that currently takes place within the community in the learning of a second language. This research study indicates that there are important effects arising from the use of multimodal technologies that can be employed in foreign language learning as well as for broadening second language learning. An acknowledgement of this feature in the English language learning environment can facilitate learning and make the learning and teaching process more interesting as well as more meaningful for the students engaged in the learning of either a foreign or a second language.
Appendices
Appendix 1: Variables in the Individual Level Model. This is available in the file:
http://www.iier.org.au/iier22/nallaya-appendix1.pdf
Appendix 2: A higher resolution version of Figure 2: Path model is available in the file:
http://www.iier.org.au/iier22/nallaya-appendix2.jpg
(356 kB; widen window to view this file)
Bereiter, C. (2002). Education and mind in the knowledge age. Mahwah, NJ: Lawrence Erlbaum Associates.
Burgess, R. & Owens, C. (2003). Needs analysis: Preliminary report. [viewed 17 Sep 2007, verified 20 Aug 2012] http://www.aui.ma/old/VPAA/cads/research/cad-research-faculty-needsanalysis.pdf
Chomsky, N. (1965). Aspects of theory of syntax. Cambridge, Massachusetts: MIT Press.
Clovis, D. L. (1997). Lights, television, action! Educational Leadership, 55(3), 38-40.
Darmawan, I. G. N. (2003). Implementation of information technology in local government in Bali, Indonesia. Studies in Comparative and International Education, 9. Flinders University: Institute of International Education.
Department of Multicultural Education (2003). Academic and English language acquisition. [viewed 8 Oct 2007, not found 20 Aug 2012]. http://www.cde.state.co.us/cdesped/download/pdf/ELL_LangAcqArticle.pdf
Dudley-Evans, T. (2001). Critical English for academic purposes: Theory, politics and practice. Mahwah, New Jersey: Lawrence Erlbaum.
Falk, R. F. (1987). A primer for soft modelling. University of California, Berkeley: Institute of Human Development.
Fisher, R. A. (1932). Statistical methods for research workers. London: Oliver & Boyd.
Gill, S. K. (2004). Language policy and planning in higher education in Malaysia: A nation in linguistic transition. Asian Englishes, 6(2), 10-25. [viewed 8 Mar 2007, verified 20 Aug 2012] http://ccat.sas.upenn.edu/plc/clpp/proposal/SaranMalaysia.htm
Hall, J. K. (2001). Methods for teaching foreign languages. New Jersey: Prentice-Hall.
Kasper, L.F. (2000). New technologies, new literacies: Focus discipline research and ESL learning communities. Language Learning & Technology, 4(2), 105-128. http://llt.msu.edu/vol4num2/kasper/default.html
Keeves, J. (2002). Learning in schools: A modelling approach. International Education Journal, 3(2), 114-125. http://ehlt.flinders.edu.au/education/iej/articles/v3n2/keeves/paper.pdf
Lacina, J. (2005). Promoting language acquisitions: Technology and English language learners. Childhood Education, 81(2), p113. http://www.thefreelibrary.com/Promoting+language+acquisitions%3A+technology+and+English+language...-a0126651977
Leki, I. (2000). Writing, literacy and applied linguistics. Annual Review of Applied Linguistics, 20, 99-115. http://dx.doi.org/10.1017/S0267190500200068
Lyons, J. (n.d.). Chomsky. UK: Fontana Modern Masters.
Mansheng, Z & Keeves, J.P. (2003). Research in education: Nature, impact, needs and priorities. In J. P. Keeves & R. Watanabe (Eds.), International handbook of educational research in Asia-Pacific Region, pp. 1193-1206, The Netherlands: Kluwer Academic Publishers.
Millard, E. & Marsh, J. (2002). Words with pictures: The role of visual literacy in writing and its implication for schooling. Reading, 35(2), 54-61. http://dx.doi.org/10.1111/1467-9345.00162
Nallaya, S. (2010). The impact of multimodal texts on the development of English language proficiency. Unpublished Doctoral dissertation. The University of Adelaide, Adelaide, Australia. http://digital.library.adelaide.edu.au/dspace/bitstream/2440/62385/2/02main.pdf
Netz, R. & Noel, W. (2007). The Archimedes code x: How a medieval prayer book is revealing the true genius of antiquity's greatest scientist. London: Weidenfeld & Nicholson.
Neuman, W. L. (2006). Social research methods: Qualitative and quantitative approaches. (6th ed.). Boston: Pearson Education.
New London Group (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 66(1), 60-93. http://her.hepg.org/content/17370n67v22j160u/
Patton, M. Q. (1990). Qualitative evaluation and research methods (2nd ed.). London: Sage Publications.
Popper, K. R. (1972). Objective knowledge. Oxford: Clarendon Press.
Sellin, N. (1989). PLSPATH Version 3.01 Program Manual. Hamburg, West Germany.
Sellin, N. & Keeves, J.P. (1997). Path analysis with latent variables. Education: The Complete Encyclopaedia, 1-11.
Tan, L. (2006). Literacy for the 21st century. Educational Technology Division, Ministry of Education, Singapore. http://iresearch.edumall.sg/iresearch/slot/fm3_posts/ah01/4c8407c08_u3112.pdf
The Knowledge Loom: Elementary Literacy (2004). Meeting the literacy needs of English language learners (ELLs). [viewed 8 Oct 2007, verified 20 Aug 2012] http://knowledgeloom.org/elemlit/ells_meetnds.jsp
Thesen, L. (2001). Modes, literacies and power: A university case study. Language and Education, 15(2-3), 132-145. http://dx.doi.org/10.1080/09500780108666806
Vygotsky, L. (1978). Mind in society. London: Harvard University Press.
Warschauer, M. (2000). Technology and literacy in the 21st century. Paper presented at TESOL 2000, Vancouver, BC.
Willett, J. B. (1994). Measuring change more effectively by modelling individual change over time. In T. Husen & T. N. Postlethwaite (Eds.), The International Encyclopaedia of Education, 2, Elmsford, NY: Pergamon Press.
Wold, H. (1982). Soft modelling: The basic design and some extensions. In K.G. Joreskog & H. Wold (Eds.), Systems under indirect observation, Part II. Amsterdam: North Holland Press.
Xiao, L. (2003). What can we learn from a learning needs analysis of Chinese English majors in a university context. Asian EFL Journal, 8(4), Article 4. http:////www.asian-efl-journal.com/December_06_home.php
| Author: Dr Sasikala Nallaya is a Language and Learning Coordinator in the University of South Australia, Adelaide, Australia. She is responsible for coordinating and providing language and learning support to students studying in the University. Her research interests include needs assessment, multimodal texts in language learning and second language acquisition. Email: Shashi.Nallaya@unisa.edu.au Please cite as: Nallaya, S. (2012). The measurement of change in English language proficiency. Issues In Educational Research, 22(2), 149-168. http://www.iier.org.au/iier22/nallaya.html |
