|
Does supply always come on the heels of demand? Matches and mismatches in e-learning
Bhavani Sridharan
Deakin University, Australia
Hepu Deng
RMIT University, Australia
Kinshuk
Athabasca University, Canada
Developing sustainable e-learning requires a better understanding of the perceptions and preferences of e-learning providers and e-learners on the four crucial dimensions for e-learning success including pedagogies, technologies, learning resources and management of learning resources. There is, however, little research on evaluating whether these critical dimensions are perceived as critical by e-learning providers and e-learners. To address this issue, this study investigates the gap between e-learners' and e-learning providers' perceptions and preferences on these critical dimensions for e-learning effectiveness. Such an investigation paves the way for developing appropriate measures to reduce the gap between the supply and the demand for sustainable e-learning.
To be competitive in such a dynamic environment, higher education institutions have to adequately recognise and consequently respond to the change. This means that the demand of learners (Irvine, Code, & Richards, 2013) needs to be met adequately. Against this backdrop, higher education institutions are rethinking about 'what and how' to scaffold 'teaching and learning' for stepping up the game. The crucial question is, however, whether these changes are aligned with the demand of e-learners and the supply of e-learning providers. In other words, the question is whether higher education institutions have the knowledge about 'what learners want' to reap the full benefits of this change and to cope with the demand of the competition (Jafari, McGee, & Carmean, 2006). In this scenario, the quote by Robert Collier (1926, p. 57), "supply always comes on the heels of demand", is pertinent in many ways owing to the power shift in favour of learners in the competitive higher education market.
Aligning the supply (what e-learning providers facilitate) and the demand (what e-learners want) is crucial for developing sustainable e-learning. There is, however, little research in evaluating whether the critical factors perceived by e-learners and e-learning providers respectively are compatible with each other. To fill this gap, the objective of this paper is to examine whether there is an alignment of the perceptions and preferences between these two stakeholders' on the critical factors influencing the effectiveness of e-learning.
This study builds on the earlier work in evaluating the critical dimensions for the e-learning effectiveness by providing a comparative analysis of the critical factors for sustainable e-learning based on the perceptions of e-learning providers and e-learners (Sridharan, Deng, Kirk, & Corbitt, 2010). Such an analysis helps identify the disparities and similarities between the preferences and perceptions of these stakeholders for sustainable e-learning. The study extends existing research on the critical factors for sustainable e-learning and provides higher education institutions with recommendations on the development of specific strategies and policies for sustainable e-learning.
The paper is organised as follows. Section 2 provides the background to this study. Section 3 presents a review of the related studies. Section 4 explains the research methodology used in this research. Section 5 illustrates the models and the hypothesis for this research. Section 6 presents a comparative analysis of the study. Finally, section 7 discusses the limitations and future research directions.
The purpose of higher education with respect to 'what to teach' is about the transfer of discipline-specific knowledge. The significant shift in the focus towards developing employable graduates is due to the demand of employers and national and international accreditation agencies (Oliver, 2013). Traditionally, there has not been a very explicit focus on taxonomies of learning in tertiary education. This is because the primary focus of higher education is to teach and test the absorption of concepts and theories in a given domain for the purpose of certification. Modern educational reform calls for preparing learners with the development of deep-learning skills to face the real-life challenges and for being 'world-ready', 'future-ready'and career-ready' (Spencer, Riddle, & Knewstubb, 2011).
There has been significant evidence-based research for supporting the criticality of learner-centred teaching and learning (Hall Jr, 2013). To effectively integrate learner-centred teaching and learning, there is a call for the alignment between 'what learners do' and 'what teachers do' in knowledge acquisition (Biggs, 2012). This shows that there are revised expectations from learners, e-learning providers, employers, national quality and standards agencies, and international accreditation agencies. Figure 1 provides a summary of a comparative analysis between traditional and modern higher education.
Figure 1: A comparative analysis between traditional and modern higher education
There has been a paradigm shift in the role of teachers from "sage on stage" to "guide on side" with increased expectations and responsibilities. There are numerous challenges for teachers including innovatively scaffolding pedagogies and technologies for enhancing the learner experience, fostering a sense of community (Garrison, Anderson, & Archer, 2010), creating effective and diverse learning resources, upskilling on the use of the latest tools and technologies (Wong, 2012), participating in profession development sessions (Sun et al., 2008), and developing students' employability in addition to discipline specific knowledge (Oliver, 2013). It is evident that the power of 'teaching presence' is vital for positively impacting students' learning experiences (Marks, Sibley, & Arbaugh, 2005).
The discussion above shows that it is imperative that teachers' and learners' perceptions and preferences be given serious consideration for developing sustainable e-learning. There is, however, a dearth of research in understanding, identifying and matching e-learners' and e-learning providers' perceptions and preferences on the critical factors for e-learning. This study attempts to identify the critical factors as perceived by these two stakeholder groups for sustainable e-learning.
The demand and supply theory highlights the importance of matching the supply and the demand for specific items to achieve efficiency in resource allocation in a competitive world in which no individuals can dominate the market. In this study, the demand is related to the factors desired by e-learners, and the supply is about the factors perceived critical by e-learning providers for sustainable e-learning. There are many reasons why this is critical in higher education, such as resource constraint, lack of incentives for academics, rapid changes to information and communication technologies, and globalisation of the higher education sector (Hall Jr, 2013).
There are four inter-related critical dimensions for sustainable e-learning including pedagogies (Alexander & Boud, 2001), technologies (El-Mowafy, Kuhn, & Snow, 2013) , learning resources (Tzeng, Chiang, & Li, 2007), and management of learning resources (Duval, 2006). The development of sustainable e-learning depends on the effective interaction between these dimensions.
Pedagogies refer to the principles and methods of teaching and learning used in the knowledge transfer process. Several pedagogies are acknowledged as critical in e-learning including collaborative learning (Laurillard, 2009), interactive learning (Craighead, 2008), explorative learning (Yi, 2008), adaptive learning (VanLehn, 2006), and concept mapping (Novak, 2010). Collaborative learning refers to instructional methods where learners have an opportunity to work in groups and develop their team working skills by exchanging their ideas (Goodyear & Zenios, 2007). Interactive learning is where learners construct their own knowledge by interacting with subject matter through hands-on activities (Laurillard et al., 2013). Explorative learning is where learners construct their own knowledge by exploring and discovering inconsistencies in understanding within their learning environments (Dalgarno, 2001). Adaptive learning accommodates the differences in levels, styles and preferences of learners in contrast to the 'one-size-fits-all' concept. Concept mapping is a diagrammatic representation of the relationships between concepts in a specific situation (Novak, 2010).
Technologies refer to the use of educational tools and systems to support diverse pedagogies. Some of these technologies include learning management systems (LMS), intelligent tutoring systems (Craighead, 2008), collaborative technologies (Laurillard, 2009), Web 2.0 technologies (Conole, 2013), adaptive learning systems (VanLehn, 2006), concept map technologies (Liu, Chen, & Chang, 2010), semantic technologies (Charlton, Magoulas, & Laurillard, 2012), search and retrieval technologies (Yi, 2008), clicker technologies (Evans & Matthew, 2013) and mobile learning technologies (Wong, 2012).
The critical role of learning resources in scaffolding multiple pedagogies and technologies for enhancing e-learning is widely recognised in the literature. Several types of learning resources such as Web 2.0 resources, open educational resources, massive open online course resources, and interactive multimedia resources have been proposed for facilitating learner-centred pedagogies (Conole, 2013). Various approaches have been developed for adequately utilising existing learning resources. Richards (2007), for example, suggested an active re-use of learning resources for improving e-learning effectiveness. Craighead (2008) shows that using adaptive learning resources to fit individual learners' levels and styles can enhance e-learning. Liu et al. (2010) state that the adoption of diagram-based resources for supporting concept mapping can enhance the comprehension of individual learners.
The deployment of these pedagogies, technologies and learning resources often leads to the generation of a massive number of valuable e-learning resources. This necessitates the effective management of learning resources for overcoming the problem of information overload by filtering relevant and re-usable learning resources (Demidova et al., 2005). Several factors have been identified as critical for the effective management of learning resources (Nonaka & Toyamma, 2003). These factors include effective resources organisation and presentation (Yi, 2008), effective knowledge retrieval and re-use (Huang & Mille, 2006), filtering and pruning e-learning resources (Sridharan, Deng, & Corbitt, 2008), effective retrieval of multimedia objects (El Saddik, Fischer, & Steinmetz, 2001), and re-use of lessons (Ras & Rech, 2009).
Recent developments in the semantic web can augment the creation, extraction, organisation, retrieval and re-use of learning resources in e-learning (Huang, Webster, Wood, & Ishaya, 2006). Two aspects of the semantic web including metadata and ontologies play a significant role in the effective management of learning resources. Metadata is "any data which conveys knowledge about an item without requiring examination of the item itself" (Haase, 2004, p. 204). Ontologies are "the metadata schema providing a controlled vocabulary of concepts" (Maedche & Staab, 2001, p. 72). They define the relationship between concepts (Berners-Lee, Handler, & Lassila, 2006). Ontologies facilitate communication between people and computers through a shared understanding of resources (Davies, Harmelen, & Fensel, 2002).
The four dimensions described above are mutually interdependent (Sridharan, Deng, & Corbitt, 2010). Pedagogies have no value in e-learning without embedding the associated technologies, learning resources and facilitates for retrieval of learning resources. Analogously, instructional technologies and learning resources have limited usefulness without understanding the pedagogical principles behind these dimensions. Figure 2 represents the intertwined nature of the four dimensions for enhancing the e-learning effectiveness.
There are innumerable factors within each dimension that are critical for sustainable e-learning. Identifying the critical factors from the perspective of both demand and supply is vital for e-learning success. There is, however, little research in understanding the compatibility of these critical factors between e-learners and e-learning providers. This study evaluates the similarities and the variations in the perceptions of e-learners and e-learning providers on the critical dimensions and the critical factors for sustainable e-learning. This evaluation leads to a reduced number of factors perceived as critical by both e-learners and e-learning providers, as represented in Figure 3.
The main research question addressed in this study is as follows: Is there a match between the perceptions of e-learners and e-learning providers on the critical dimensions and the critical factors within each dimension for sustainable e-learning?
Figure 2: The interconnection between the four dimensions of e-learning
Figure 3: A conceptual framework for identifying the critical factors
The qualitative data collection consists of five phases including identifying the critical factors for each dimension, developing preliminary interview questions, pilot-testing and revising interview questions, conducting interviews, and transcribing and analysing interviews. The interview questions include (a) demographic information, (b) perceptions on the influence of pedagogies, technologies, learning resources and management of learning resources, and (c) perceptions on the critical factors for sustainable e-learning. A total of twenty-nine interviews were conducted, out of which twenty-seven were from five universities in a south eastern city in Australia, and two were with representatives from the Open Universities Australia. Prospective candidates for interviews were identified through an Internet search of university websites and contacts provided by colleagues and interviewees.
Figure 4: The research process
The quantitative data collection includes formulating the survey instrument, pilot-testing questions with domain experts and e-learners, conducting an online survey of e-learners, and cleaning and preparing collected data for analysis. There are seven latent constructs with 62 items in the survey instrument including pedagogies, technologies, learning resources, management of learning resources, metadata ontologies, management effectiveness and e-learning effectiveness. The instrument items are derived from an iterative process of literature review and content analysis of the interview. The online data collection was accomplished over a period of nine months. To prevent errors arising from data collection, specific measures were taken to avoid incorrect data entries, invalid entries and missing responses. This lead to two hundred and ten valid responses being collected.
A content analysis of the transcribed interviews was conducted, resulted in the extension of items, formulation of preliminary hypotheses, and the development of the base model for this study. The analysis was used to see whether or not a hypothesis was possible (Bouma & Atkinson, 1995). Such an analysis would help identify probable factors for sustainable e-learning in developing a base model.
Several measures were taken to ensure the reliability and validity of the findings in this study. To ensure the reliability, this study adhered to Kleven's (2008) consistency checks for guaranteeing the uniformity of measurements. To ensure the validity, this study used Johnson's (1997) framework to evaluate its methodological strengths and weaknesses, with appropriate steps taken to overcome the weaknesses. Researchers' bias and three types of validity (descriptive, interpretive and theoretical) were adhered to. In addition, the internal and external validity (Maxwell, 1992) were applied due to the exploration of the cause and effect relationship and the plausible generalisation of the findings.
The quantitative data analysis consisted of preliminary multivariate analysis, instrumental validity, exploratory factor analysis and confirmatory factor analysis (CFA). KMO measure and Bartlett's test of sphericity were used to validate the sampling adequacy and suitability of data for factor analysis. Mahalanobis distance, skewness and kurtosis measures were used to confirm the normality of the dataset. Instrumental validity was tested through three reliability measures including internal consistency, item reliability and construct reliability and two validity measures including convergent and discriminant validity measures. The use of these measures helped to reduce the number of items that did not fit the recommended guidelines for developing a more parsimonious model.
The primary focus of this study was to test the relationship between the latent constructs in the hypothesised model. A two-step approach to structural equation modelling was considered including estimating the measurement model and the structural model. A CFA was conducted in the measurement model for assessing the contribution of each indicator variable and for measuring the model's adequacy. Three stages were involved in assessing the measurement model including (a) the model specification, (b) the iterative model modification, and (c) the estimation of parameters. The iterative model modification process required refinement and retesting of individual measurement models. This resulted in developing a more parsimonious limited set of items to represent a construct, followed by an estimation of the structural model (Anderson & Gerbing, 1988).
The overall model fitness was evaluated using several measures of the goodness-of-fit (GOF) including the chi-square test (?2), the ratio of ?2 to degrees of freedom (DF), the goodness-of-fit index (GFI), the adjusted goodness-of-fit index (AGFI), the root mean square error of approximation (RMSEA), and the Tucker-Lewis index (TLI). The significance of the path coefficient was assessed using the standard error and the t-value (Holmes-Smith, Cunningham, & Coote, 2006).
A comparative analysis of the perceptions of the stakeholders was conducted that consisted of (a) compilation and recording of key findings from both the study, (b) identification of the critical factors within each dimension from both groups of stakeholders, (c) identification of matching factors, (d) identification of mismatching factors, and (e) identification of the reasons for those differences. The details were recorded and organised using a systematic classification process using Microsoft Excel worksheets.
Figure 5 represents the summary of the proposed relationship and hypotheses for the base model and the refined model. The independent latent constructs are represented in blue shaded boxes, and the dependent latent constructs in the green shaded boxes. The base model and the refined model are evaluated using interview results and the survey findings respectively. The base model represents the direct relationships between the six constructs and e-learning effectiveness in six respective hypotheses represented as H1 to H6. The refined model extends the base model to capture both the direct and indirect relationships between the six constructs and the e-learning effectiveness. Consequently, nine hypotheses were proposed for considering the direct effects, and three hypotheses were developed for consideration of the indirect effects.
Refined model and hypotheses
Figure 5: Summary of the relationships in the base model and the refined model
There are specific differences between the base model and the refined model. In the refined model, the first set of hypotheses H1, H2a, H2b and H3 deals with the direct influence of four latent constructs on the e-learning effectiveness. These four hypotheses are the replica of the base model hypotheses of H1, H2, H3 and H5. The second set of hypotheses H4, H5a, H5b and H6 examined the direct influence of four latent constructs on the management of learning resources. The third set of hypotheses H7, H8 and H9 examined the indirect influence of two mediating latent constructs on the management effectiveness and the e-learning effectiveness. Except for H8, the other two hypotheses H7 and H9 are the same as the base model hypotheses represented as H4 and H6.
Most interviewees acknowledged the importance of combining pedagogies for improving e-learning effectiveness. Two widely-used strategies were collaborative learning and explorative learning. There is great variation, however, in the implementation of these strategies. For instance, collaborative learning is adopted from the proactive involvement of academic staff through incorporating fully moderated collaborative forums to unmoderated discussions by students. Explorative learning is used with additional course-related learning resources through external links. However in most instances, the assessment of authenticity, quality and value of exploratory e-learning resources is left to the judgment of individual learners. With respect to interactive learning, adaptive learning and concept maps, interviewees highlighted the importance of embedding them in online courses. Notwithstanding these views, in reality interviews reveal a general under-utilisation of these pedagogies and allied technologies.
The necessity for using appropriate technologies and learning resources in e-learning was extensively recognised by the interviewees. The adoption of appropriate technologies and learning resources for e-learning, however, has not become a reality due to several barriers to their implementation. These barriers include the ineffectiveness of LMS for enhancing a learner-centred learning process, a lack of understanding the underpinning pedagogy behind the use of these technologies, and the efficient use of academics' time and effort and the effect on personal career prospects. The two most popular technologies include LMS and collaborative learning technologies. Two popular learning resources include multi-media resources and external learning resources.
There are mixed views on the management of learning resources and associated dimensions such as metadata ontologies and management effectiveness. The study shows that LMS are not effective particularly in terms of reusability and searchability. It reveals that e-learning resources within LMS are neither reusable nor inter-operable. The key challenges in creating a reusable learning object repository include reusability, shareability, searchability, authenticity, version controls, granularity and copyright issues.
The findings indicated a positive association between pedagogies (H1), technologies (H2) and learning resources (H3), and e-learning effectiveness. This is in contrast to a lack of clear support for the positive influence of management of learning resources (H4), metadata ontologies (H5) and management effectiveness (H6) on the e-learning effectiveness. Interviewees from the library and content management division in one university believed that these factors are critical for the reusability of learning resources. This view, however, was only endorsed by some interviewees. Several interviewees felt that it would be an absolute waste of time and resources to create these repositories which would soon become obsolete.
The interview findings substantiate the literature on the perceived effectiveness of multiple pedagogical approaches for enhancing the e-learning effectiveness. Overall, most interviewees agreed that adopting multiple pedagogical strategies and associated technologies and learning resources can achieve sustainable e-learning success. The interview findings reveal various views on the power of the management of resources to enhance e-learning effectiveness. Further research is required to validate this finding by using surveys to collect data from key stakeholders with a micro-level data analysis.
Figure 6: A hypothesised refined model (above) and a final structural model (below)
The first criterion for assessing the structural model is the GOF measure. The GOF measure and the recommended value for the structural model suggest that the model displays a good fit for the dataset with a χ2 result of 112.04 with 92 DF. This observation is further endorsed by a χ2/df result of 1.22, and the GFI, AGFI and TLI results with values greater than 0.90 for each of these measures. An RMSEA of 0.03 is also within the recommended value of 0.1.
The second criterion is the explanatory power (R2) of each dependent construct. These R2 results indicate that the model explains 50% of the variance in the e-learning effectiveness, 41% of the variance in the management of learning resources, and 18% of the variance in management effectiveness.
The third criterion is the significance of the model coefficients for all structural paths in the structural model. Table 1 presents the hypothesis results from the structural models including the hypothesis statements, their path coefficient values, the significance of each hypothesis, and the results in terms of the acceptance or rejection of each hypothesis. The findings suggest strong support for H3, H5 and H8, moderate support for H2, H6 and H7, weak support for H9 and no support for H1 and H4. This implies that e-learning effectiveness, as perceived by e-learners, can be explained by the management of learning resources, technologies and learning resources, and the metadata ontologies supporting the management of learning resources. The findings also show the rejection of H1 and H4 indicating that pedagogies have a non-significant effect on the e-learning effectiveness and the management of learning resources. However, the indirect influence of pedagogy can be seen from the elevated total effect represented in parentheses in Table 1.
| Hypothesis | Path | Significance | Result |
| H1:Pedagogies positively influence the e-learning effectiveness | -0.08 | n.s | N |
| H2:Technologies and learning resources positively influence the e-learning effectiveness | 0.22 (0.37) | * | Yes |
| H3: Metadata ontologies positively influence the e-learning effectiveness | 0.33 (0.42) | *** | Yes |
| H4: Pedagogies positively influence the management of learning resources | 0.16 | n.s | No |
| H5: Technologies and learning resources positively influence the management of learning resources | 0.41 | *** | Yes |
| H6: Metadata ontologies positively influence the management of learning resources | 0.25 | ** | Yes |
| H7: Management of learning resources positively influences the e-learning effectiveness | 0.29 (0.35) | * | Yes |
| H8: Management of learning resources positively influences the management effectiveness of learning resources | 0.42 | *** | Yes |
| H9: Management effectiveness positively influences the e-learning effectiveness | 0.15 | * | Yes |
| *p<0.05; **p<0.01; *** p<0.001 | |||
Table 2 presents a comparative analysis of the perceptions of e-learners and e-learning providers on the influence of key constructs on e-learning effectiveness. With respect to pedagogies, the qualitative study indicates the criticality of using multiple pedagogies to enhance the e-learning effectiveness. There is strong support for all five strategies (interactive, collaborative, adaptive, concept mapping and explorative learning strategies) to enhance e-learning effectiveness. In contrast, the quantitative result suggests that pedagogies per se are not critical from the perspective of e-learners. This result is inconsistent with the finding in the literature on the positive role of pedagogies on influencing e-learning effectiveness (Walker & Fraser, 2005).
The perception that pedagogy per se is not critical is consistent with the view that "pedagogy and interactions are determined by system rather than learners or instructional designers" (Carmean & Brown, 2005, p. 155). Analogous with this view, Brennan (2001, p. 25) report, "there is a disjunction between the reform pedagogy assumptions that policy-makers hold and what actually happens. It is not surprising because in the online environment it is shockingly difficult to get beyond transmission".
| Hypothesis | E-learning providers support | E-learners support | Match/ mismatch |
| H1:Pedagogies positively influence the e-learning effectiveness | Strong | No Support | Mismatch |
| H2:Technologies and learning resources positively influence the e-learning effectiveness | Strong | Low Support | Match |
| H3: Metadata ontologies positively influence the e-learning effectiveness | Partial | High Support | Match |
| H4: Pedagogies positively influence the management of learning resources | N/A* | No Support | N/A* |
| H5: Technologies and learning resources positively influence the management of learning resources | N/A* | High Support | N/A* |
| H6: Metadata ontologies positively influence the management of learning resources | N/A* | Medium support | N/A* |
| H7: Management of learning resources positively influences the e-learning effectiveness | Partial | Low Support | Match |
| H8: Management of learning resources positively influences the management effectiveness of learning resources | Partial | Strong Support | Match |
| H9: Management effectiveness positively influences the e-learning effectiveness | Partial | Low Support | Match |
| * N/A - Not applicable | |||
There are some probable reasons that could be attributed to the differences in the perceptions of e-learners and e-learning providers. For e-learners, pedagogies are at the back-end of e-learning systems. They are 'behind the scenes' and are not of any concerns to e-learners. In contrast, a choice of pedagogies is a central issue for e-learning providers, as the design of e-learning courses is pedagogy-based (Carmean & Brown, 2005). E-learners have more expectations on the 'what' side of the coin rather than 'how' aspects of the pedagogy underpinning what is provided to them. However, the 'how' aspects of pedagogies are critical for e-learning providers due to their importance for the design and development of e-learning courses. Further research into the views of e-learners and e-learning providers using an identical survey simultaneously would provide more insights into the criticality of pedagogies for sustainable e-learning.
Both e-learning providers and e-learners perceive technologies and learning resources to be critical for e-learning success. This is consistent with the findings that technology positively influences the e-learning effectiveness (Chandra & Lloyd, 2008). Differences, however, exist on the extent of actual use by e-learning providers. For instance, the qualitative study indicates the lack of a wide use of technologies and learning resources for supporting active learning, visual learning, and explorative learning. In reality one of the most prevalent technologies is the technology supporting collaborative learning integrated in the LMS. Within those collaborative technologies, the qualitative study identifies an underutilisation of the technology. In contrast, the quantitative study confirms the criticality of technologies and learning resources related to concept mapping to enhance e-learning effectiveness. However, the earlier qualitative results suggest that a wide use of concept mapping technologies and associated resources in e-learning is lacking.
The qualitative study identifies many challenges in incorporating the critical factors in the technology dimension. These challenges include the lack of understanding of the theory behind the technologies, the lack of knowledge of the full potential of these technologies (McGill & Klobas, 2009), and the time and effort required to create learning resources. This shows that similarities and differences exist in the perceptions of e-learners and e-learning providers on the influence of the technologies and learning resources on the e-learning effectiveness.
With respect to the management of learning resources, the qualitative findings suggest a lack of unanimous views about its positive influence on enhancing e-learning effectiveness. As a result, the learning resources generated are not transferred to reusable learning resources. In addition, the findings clearly indicate a lack of support for the creation of metadata ontologies to enhance the reusability and searchability of learning resources. This contradicts the findings in the literature that have indicated the positive role of the management of learning resources, metadata ontologies (Hatem, Ramadan, & Neagu, 2005) and management effectiveness (Shaw, Dicks, Venkatesh, Lowerison, & Dai, 2004) in enhancing e-learning effectiveness. In comparison, the quantitative results are consistent with the findings in the literature which identify management of learning resources, including metadata ontologies and management effectiveness, as the critical dimensions for the e-learning effectiveness.
Many reasons can be attributed to the philosophical differences between e-learning providers and e-learners on the influence of the management of learning resources and metadata ontologies in enhancing e-learning effectiveness. For example, e-learning providers may not pay much attention to the management of learning resources due to lack of time and the huge amount of effort required to manage of learning resources and create of metadata ontologies. There may be resistance to share resources, copyright issues, quality, granularity, version control and validation of learning resources and metadata ontologies. In contrast, quick accessibility to relevant and authentic resources is critical for e-learners' knowledge acquisition.
Table 3 summarises the similarities and differences in the perceptions between e-learning providers and e-learners on the e-learning effectiveness based on the qualitative and quantitative findings. It is apparent that both e-learning providers and e-learners view the use of technologies and learning resources as critical for sustainable e-learning. However, many technologies and learning resources are not widely used due to practical difficulties. One of the key obstacles is a lack of LMS for incorporating various technologies supporting identified pedagogies.
| Dimension | E-learning providers' perceptions | E-learners' perceptions |
| Pedagogies | Perceived as critical, but not widely practiced in reality. Specific perceived critical factors are interactive, collaborative, adaptive, concept mapping and explorative learning strategies | Not perceived as critical |
| Technologies and learning resources | Perceived as critical, but many obstacles and challenges exist. Specific identified critical factors are LMS, Collaborative and external links | Perceived as critical. Specific critical factors are concept mapping technologies, push technologies and diagram-based learning resources |
| Management of learning resources (including metadata ontologies and management effectiveness) | Perceived as critical only by a few, but many challenges and obstacles exist. Specific identified critical factors are reusability, search facilities, keywords, version controls | Perceived as critical. Specific critical factors are search facilities, presentation, metadata details in particular prerequisite and co-requisite learning resources |
The comparative analysis indicates both similarities and disparities of the perceptions on the dimensions for sustainable e-learning, as represented in Table 3. The table reveals the difference in the perceptions of two e-learning dimensions: pedagogies and management of learning resources and also shows the similarity in the perceptions of the other two e-learning dimensions: technologies and learning resources.
This study contributes to the e-learning domain by identifying the necessity for developing more aligned policy measures to unify the critical e-learning dimensions and to eliminate the barriers for sustainable e-learning. The study may assist e-learning providers to implement policy measures which better align the demand with the supply to enhance e-learning effectiveness. Specific measures include providing capacity building sessions to educate academic staff about the effective use and alignment of all four critical dimensions, embedding powerful plug-in technologies to overcome the inherent limitations of LMS, offering incentives for proactive and innovative initiatives of academic staff, and adopting a balanced approach to innovative teaching and the expectation for quality research. The study has a limitation of aggregating and dichotomising huge constructs. To overcome this limitation, future research is proposed by disaggregating and performing a micro-level analysis at each construct level.
Anderson, J.C., & Gerbing, D.W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 103(3), 411-423. http://psycnet.apa.org/doi/10.1037/0033-2909.103.3.411
ASTD (2001). A vision of e-learning for America's workforce: Report of the commission on technology and adult learning. American Society of Training Directors: Policy and Public Leadership. Alexandria.
Berners-Lee, T., Handler, J., & Lassila, O. (2006). The semantic web. Database and Network Journal, 36(3), 7.
Biggs, J. (2012). What the student does: Teaching for enhanced learning. Higher Education Research & Development, 31(1), 39-55. http://dx.doi.org/10.1080/07294360.2012.642839
Bouma, G.D., & Atkinson, G.B.J. (1995). A handbook of social science research (2 ed.). Oxford, New York: Oxford University Press.
Brennan, R. (2001). One size doesn't fit all: Pedagogy in the online environment (pp. 1-82). Adelaide: NCVER. http://www.ncver.edu.au/wps/poc?urile=wcm:path:/wps/wcm/connect/NCVER_Shared/Publications/965
Carmean, C., & Brown, G. (2005). Measure for measure: Assessing course management systems. Hershey, PA: Information Science Publishing.
Chandra, V., & Lloyd, M. (2008). The methodological nettle: ICT and student achievement. British Journal of Educational Technology, 39(6), 1087-1098. http://dx.doi.org/10.1111/j.1467-8535.2007.00790.x
Charlton, P., Magoulas, G., & Laurillard, D. (2012). Enabling creative learning design through semantic technologies. Technology, Pedagogy and Education, 21(2), 231-253. http://dx.doi.org/10.1080/1475939X.2012.698165
Collier, R. (1926). The secret of the ages. http://newthoughtlibrary.com/collierRobert/secretOfTheAges/
Conole, G. (2013). Designing for learning in an open world. New York: Springer.
Craighead, J. (2008). Distributed, game-based, intelligent tutoring systems - the next step in computer based training? Proceedings of the International Symposium on Collaborative Technologies and Systems, Irvine, CA, USA. http://dx.doi.org/10.1109/CTS.2008.4543938
Dalgarno, B. (2001). Interpretations of constructivism and consequences for computer assisted learning. British Journal of Educational Technology, 22(2), 183-194. http://dx.doi.org/10.1111/1467-8535.00189
Davies, J., Harmelen, F.V., & Fensel, D. (2002). Towards the semantic web: Ontology-driven knowledge management. England: John Wiley & Sons Ltd.
Demidova, E., Ternier, S., Olmedilla, D., Dual, E., Dicerto, M., Stefanov, K., & Sacristan, N. (2005). Integration of heterogeneous information sources into knowledge resource management system for lifelong learning. Proceedings of the TEN Competence Workshop on Service Oriented Approaches and Lifelong Competence Development Infrastructures, Manchester, UK. http://dspace.ou.nl/bitstream/1820/883/1/KRMSforLifelongLearning.pdf
Duval, E. (2006). IEEE standard for learning object metadata (LOM). http://dx.doi.org/10.1109/IEEESTD.2002.94128
El-Mowafy, A., Kuhn, M., & Snow, T. (2013). Blended learning in higher education: Current and future challenges in surveying education. Issues in Educational Research, 23(2), 132-150. http://www.iier.org.au/iier23/el-mowafy.html
El Saddik, A., Fischer, S., & Steinmetz, R. (2001). Reusable multimedia content in web-based learning systems. IEEE Multimedia, 8(3), 30-38. http://dx.doi.org/10.1109/93.939998
Evans, R., & Matthew, A. (2013). A new era; Personal technology challenges educational technology. In Electric dreams: Proceedings ASCILITE Sydney 2013. http://www.ascilite.org/conferences/sydney13/program/papers/Evans.pdf
Garrison, D.R., Anderson, T., & Archer, W. (2010). The first decade of the community of inquiry framework: A retrospective. The Internet and Higher Education, 13(1-2), 5-9. http://dx.doi.org/10.1016/j.iheduc.2009.10.003
Haase, K. (2004). Context for semantic metadata. In Proceedings of the 12th ACM International Conference on Multimedia, New York, USA. http://www.beingmeta.com/pubs/mm2004haase.pdf
Hall Jr, O.P. (2013). Assessing faculty attitudes toward technological change in graduate management education. Journal of Online Learning & Teaching, 9(1), 39-51. http://jolt.merlot.org/vol9no1/hall_0313.htm
Hatem, M.S., Ramadan, H.A., & Neagu, D.C. (2005). E-learning based on context oriented semantic web. Journal of Computer Science, 1(4), 500-504. http://thescipub.com/PDF/jcssp.2005.500.504.pdf
Holmes-Smith, P., Cunningham, E., & Coote, L. (2006). Structural equation modelling: From the fundamentals to advanced topics. Education & Statistics Consultancy, Statsline.
Huang, W., & Mille, A. (2006). ConKMeL: A contextual knowledge management framework to support multimedia e-learning. Multimedia Tools and Applications, 30(2), 205-219. http://dx.doi.org/10.1007/s11042-006-0024-4
Huang, W., Webster, D., Wood, D., & Ishaya, T. (2006). An intelligent semantic e-learning framework using context-aware semantic web technologies. British Journal of Educational Technology, 37(3), 351-373. http://dx.doi.org/10.1111/j.1467-8535.2006.00610.x
Irvine, V., Code, J., & Richards, L. (2013). Realigning higher education for the 21st-century learner through multi-access learning. Journal of Online Learning & Teaching, 9(2), 172-186. http://jolt.merlot.org/vol9no2/irvine_0613.pdf
Jafari, A., McGee, P., & Carmean, C. (2006). Managing courses, defining learning: What faculty, students, and administrators want. EDUCAUSE Review, 41(July/August), 50-71. https://net.educause.edu/ir/library/pdf/erm0643.pdf
Johnson, R.B. (1997). Examining the validity structure of qualitative research. In A. K. Milinki (Ed.), Cases in qualitative research: Research reports for discussion and evaluation (pp. 160-165). Los Angeles, CA: Pyrczak Publishing.
Karunasena, A., Deng, H., & Kim, B. (2013). Structural equation modelling for Web 2.0 based interactive e-learning in Sri Lanka. International Journal of Information Technology and Computer Science, 12(3), 1-8. http://www.ijitcs.com/volume%2012_No_3/Anuradha.pdf
Klein, L. (1983). The economics of supply and demand. Oxford: Basil Blackwell.
Kleven, T.A. (2008). Validity and validation in qualitative and quantitative research. Nordic Educational Research, 28, 219-233. http://www.idunn.no/ts/np/2008/03
Laurillard, D. (2009). The pedagogical challenges to collaborative technologies. International Journal of Computer-Supported Collaborative Learning, 4(1), 5-20. http://dx.doi.org/10.1007/s11412-008-9056-2
Laurillard, D., Charlton, P., Craft, B., Dimakopoulos, D., Ljubojevic, D., Magoulas, G., Masterman, E., Pujadas, R., Whitley, E.A., & Whittlestone, K. (2013). A constructionist learning environment for teachers to model learning designs. Journal of Computer Assisted Learning, 29(1), 15-30. http://dx.doi.org/10.1111/j.1365-2729.2011.00458.x
Liu, P.-L., Chen, C.-J., & Chang, Y.-J. (2010). Effects of a computer-assisted concept mapping learning strategy on EFL college students' English reading comprehension. Computers & Education, 54(2), 436-445. http://dx.doi.org/10.1016/j.compedu.2009.08.027
Maedche, A., & Staab, S. (2001). Learning ontologies for the semantic web. IEEE Intelligent Systems and Their Applications, 16(2), 72-79. http://ceur-ws.org/Vol-40/maedche+staab.pdf
Marks, R.B., Sibley, S.D., & Arbaugh, J.B. (2005). A structural equation model of predictors for effective online learning. Journal of Management Education, 29(4), 531-563. http://dx.doi.org/10.1177/1052562904271199
Maxwell, J.A. (1992). Understanding and validity in qualitative research. Harvard Educational Review, 62(3), 279-299. http://hepg.org/her-home/issues/harvard-educational-review-volume-62,-issue-3/herarticle/_377
McGill, T.J., & Klobas, J.E. (2009). A task-technology fit view of learning management system impact. Computers & Education, 52(2), 496-508. http://dx.doi.org/10.1016/j.compedu.2008.10.002
Nonaka, I., & Toyamma, R. ( 2003). The knowledge-creating theory revisited: Knowledge creation as synthesizing process. Knowledge Management Research and Practice, 1(1), 2-10. http://www.palgrave-journals.com/kmrp/journal/v1/n1/pdf/8500001a.pdf
Novak, J.D. (2010). Learning, creating, and using knowledge: Concept maps as facilitative tools in schools and corporations. NY: Routledge.
Oliver, B. (2013). Graduate attributes as a focus for institution-wide curriculum renewal: Innovations and challenges. Higher Education Research & Development, 32(3), 450-463. http://dx.doi.org/10.1080/07294360.2012.682052
Ragowsky, A., Somers, T.M., & Adams, D.A. (2005). Assessing the value provided by ERP applications through organizational activities. Communications of AIS, 16(18), 1-50. http://aisel.aisnet.org/cgi/viewcontent.cgi?article=3036&context=cais
Ras, E., & Rech, J. (2009). Using wikis to support the net generation in improving knowledge acquisition in capstone projects. Journal of Systems and Software, 82(4), 553-562. http://dx.doi.org/10.1016/j.jss.2008.12.039
Richards, D. (2007). Collaborative knowledge engineering: Socialising expert systems. Proceedings of the 11th International Conference on Computer Supported Cooperative Work in Design, Nice, France. http://dx.doi.org/10.1109/CSCWD.2007.4281510
Rokeach, M. (1969). The role of values in public opinion research. Public Opinion Quarterly, 32(4), 547-559. http://dx.doi.org/10.1086/267645
Shaw, S., Dicks, D., Venkatesh, V., Lowerison, G., & Dai, Z. (2004). An empirical evaluation of topic map search capabilities in an educational context. Proceedings of the 5th International Conference on Information Technology Based Higher Education and Training, Istanbul, Turkey. http://dx.doi.org/10.1109/ITHET.2004.1358242
Spencer, D., Riddle, M., & Knewstubb, B. (2011). Curriculum mapping to embed graduate capabilities. Higher Education Research & Development, 31(2), 217-231. http://dx.doi.org/10.1080/07294360.2011.554387
Sridharan, B., Deng, H., & Corbitt, B. (2008). Evaluating intertwined critical success factors for sustainable e-learning. In Proceedings of the 19th Australasian Conference on Information Systems, Christchurch, New Zealand. http://aisel.aisnet.org/cgi/viewcontent.cgi?article=1099&context=acis2008
Sridharan, B., Deng, H., & Corbitt, B. (2010). Critical success factors in e-learning ecosystems: A qualitative study. Journal of Systems and Information Technology, 12(4), 263-288. http://dx.doi.org/10.1108/13287261011095798
Sridharan, B., Deng, H., Kirk, J., & Corbitt, B. (2010). Structural equation modelling for evaluating the user perceptions of e-learning effectiveness. In Proceedings of the 18th European Conference on Information Systems, Pretoria, South Africa. http://aisel.aisnet.org/ecis2010/59
Tzeng, G.-H., Chiang, C.-H., & Li, C.-W. (2007). Evaluating intertwined effects in e-learning programs: A novel hybrid MCDM model based on factor analysis and DEMETEL. Expert Systems with Applications, 32(4), 1028-1044. http://dx.doi.org/10.1016/j.eswa.2006.02.004
VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227-265. http://iospress.metapress.com/content/AL6R85MM7C6QF7DR
Walker, S., & Fraser, B. (2005). Development and validation of an instrument for assessing distance learning environments in higher education: The distance education learning environments survey (DELES). International Journal of Learning Envrionments Research, 8(3), 289-308. http://dx.doi.org/10.1007/s10984-005-1568-3
Wong, L.-H. (2012). A learner-centric view of mobile seamless learning. British Journal of Educational Technology, 43(1), E19-E23. http://dx.doi.org/10.1111/j.1467-8535.2011.01245.x
Yi, M. (2008). Information organization and retrieval using a topic maps-based ontology: Results of a task-based evaluation. Journal of the American Society for Information Science and Technology, 59(12), 1898-1911. http://dx.doi.org/10.1002/asi.20899
| Authors: Dr Bhavani Sridharan is a Lecturer in Learning Innovations with the Faculty of Business and Law, Deakin University. Her areas of research interests include online learning, assurance of learning, rubrics, graduate attributes, learning technologies, authentic assessment, learning analytics, database management and quality of teaching and learning. Email: bhavani.sridharan@deakin.edu.au Dr Hepu Deng is a Professor in Information Systems in the School of Business Information Technology and Logistics, RMIT University. His research interests are in the areas of decision analysis, intelligent systems, digital business, knowledge management, electronic government, e-learning, and their applications. Email: hepu.deng@rmit.edu.au Professor Kinshuk is Full Professor and Associate Dean of Faculty of Science and Technology at Athabasca University, Canada. He is also the NSERC/iCORE/Xerox/Markin Research Chair for Adaptivity and Personalization in Informatics. His research interests include adaptivity and personalization, learning analytics, mobile and ubiquitous technologies, and cognitive profiling. Email: kinshuk@athabascau.ca Please cite as: Sridharan, B., Deng, H. & Kinshuk (2014). Does supply always come on the heels of demand? Matches and mismatches in e-learning. Issues in Educational Research, 24(3), 260-280. http://www.iier.org.au/iier24/sridharan.html |