Best Practices Review

The positive trend toward online learning has certainly captured attention from university and corporate learning channels the past two decades.   Early renditions of online coursework often consisted of porting of traditional face to face content with little concern for effective learning.  This commonly known phenomenon became responsible for the negative label that distance learning earned and what current proponents of online learning find challenge within training and academic circles today. Considering this issue, online learning best practices are a particular area of study. Recently I found three studies relating to adult online learning to look for conclusions on what is considered effective for a distance learning environment and, if there were any connections – similarities found in the these distinctive studies.

Study 1:  The Effect of Student Background in E-Learning- Longitudinal Study (2008)

Summary:  This study’s purpose was to find if the background of a student makes a difference in their academic performance.  A comparison was made between undergraduate and adult learners in both online and face-to-face environments. Nearly 500 students were the subjects in a university setting.  The analysis found that:

·      There was no difference in learning achievement based on delivery or student type

·      Adult students started both types of classes with a better understanding of information systems and digital organization.

·      Online traditional students (undergraduates) indicated a greater learning achievement, based in grade, than their face-to-face counterparts.

·      Students more competent with technology did better in the online environment

·      Suggestive motivations may exist for adult learners to have a flexible time-frame to complete assignments

Future considerations:  Is there a difference between personality of students and their online performance in online and face-to-face classes?

Study 2:  Evaluation of the Effectiveness of a Web-Based Learning Design for Adult Computer Science Courses

Summary:  The study focus was on the design, development and evaluation of twenty-two online science courses for adult learners. The goal was to find what was needed to improve design for the learning environment.  A total of 786 student participants enrolled in classes limited to 20 students in each learning management system class. The courses included synchronous and asynchronous activities. Three axes were selected to report findings, 1) Information and support provided to learners in the beginning of and during their studies; 2) Learner performance and; 3) Learner satisfaction. Further division of these three areas was provided in what was termed sub axes – divisions under each of the three axes. The axis one results were derived from (B) beginning, (M) middle and (E) end-term questionnaires. The results indicated that:

•(B) learners chose an online course because no face-to-face course was required

•(B) motivation for enrollment was to improve their background or gain a diploma

•(B) sufficient materials were found to support science content within the course

•(M) 78% of students found that course meets expectations

•(M) Difficulties in course were primarily technical based – not able to log in or forgetting passwords. Secondary student concern was the growing level high level of scientific content.

•(M) The majority of learners would have liked to have more support from their instructor studying the material.

•(E) Seventy-five percent of learners reported at least moderate difficulties that were overcome with some effort

•(E) Six percent of learners experienced trouble that made them want to abandon the course.

•(E) Fifty-five percent felt very satisfied with their online experience support.

Axis two, Learner performance results were as follows:

•(B) Seventy-six percent of the learners expected to face moderate difficulty with the science content

•(B) Ninety-four percent indicated extended experience on the use of personal computers

•(B) More that 60% of learners planned to spend 1-3 hours per week on work

•(M) There were 45% of learners who were experiencing difficulty with the content with 55% attributing it to their background

•(M) Fifty-eight percent found interaction with peers helpful/41% found it neutral

• (M) Forty-five percent were spending 1-3 hours on work, 38% five or more hours

•(E) Twenty-five percent were not satisfied with their performance, 58% were satisfied.

•(E) Twenty percent felt that they did not reach their goal in the course (23.7 % failed)

Axis three, Learner satisfaction results were as follows:

•(B) Seventy-seven had great expectations for the knowledge and experience to be gained

•(B) Ninety-one percent online wanted to communicate with classmates once a week.

•(B) Fifty percent wanted only one to two evaluations per course while 32% wanted continuous evaluation

•(M) Fifteen percent felt discouraged and alone, 33% hardly used the forum at all, and 23% used the forum for help frequently

•(M) The majority of students said that continuous evaluation would be more motivating and also help with studying rate

•(M)  More than 80% were satisfied at mid-point

•(E) More than 80% of students found the learning content and experience beneficial

•(E) Only 47% believed the course to be beneficial in creating professional relationships

•(E) Only 50% of learners found that the learning platform was sufficient to use as a learning environment

There were many other results in this study and wading through the percentages was a feat; however, just from those report here on the primary axes, it is clear that there were strengths and weaknesses in the learning model. The researchers reported that the failure and dropout rate was similar to that of traditional courses so it was not a concern. One point of interest was that instructors considered the online courses as an opportunity to teach through a structured, linear sequence of learning problem solving activities and believed it had the potential to be more effective than other models offered.

Study 3:  The Influence of Adult Learner’s Self Directed Learning Readiness and Network Literacy on Online Learning Effectiveness: A Study of Civil Servants in Taiwan

Summary:  The study examined the self-directed learning readiness (SDLR) in a web-based training program designed for adults. Participants were 283 civil servants enrolled in an asynchronous learning program. The question of the study was to discover how SDLR, computer literacy and online learning effectiveness impact the overall learning of content.

Results:  The study found that the most important factor in online learning success was the self-directed learning factor (SDLR) measured by the Guglielmino SDLRS covering motivational personality traits.  Also, two areas of computer literacy were dominant in the most successful learners – Internet literacy and information evaluation.  The only low scoring areas on usage by the majority of the adult learners was related to discussion boards I often express my thoughts on the discussion board and I often reply to others’ postings on the discussion board. These were not required in the course but made available which few chose to use. An interesting study was cited that described how students ignore posts unless the instructor actively responds in discussions (Conrad, 2002).

Conrad, D. (2002). Engagement, Excitement, anxiety and fear:  Learners’ experiences of starting an online course. The American Journal of Distance Education, 16(4), 205-226.

A Comparison of ideas:

Although the three studies are about adult online learning, there was not a great deal that they shared as conclusive best practices; however, there were several points made that could be listed as best practice principles.  The first study basically proved that online learning is just as effective as face-to-face learning and that good teaching and bad teaching can occur in either environment. The outlying point was that adult students seemed to be better motivated to the online environment due to its flexibility.

Study two conformed that content must have multiple support mechanisms and instructor interaction to facilitate the learning process.  As the content got more difficult, learners needed to support one another more as well. This did not happen automatically within the courses. The workload increased beyond student expectations by mid-course. Time in study for at least a quarter of the group went from up to three hours to five or more hours. Evaluations were also viewed as a form of communication with the instructor and the continuous type became the mode of choice as students progressed.

Study three found that a personal drive to learn is the most important factor. This would correlate to the skill levels of the adult learners in all studies, as they as a majority were experienced in field and more motivated to take on the learning online.  Study three discounted the use of discussion boards but referenced the potential need for such activities. This would also be indicated in the second study.  In all three studies, it would indicate that comfort using technology is a key factor to online course success. Personally, I would like to learn more about the Guglielmino SDLRS profile exam.

Referenced Studies

Antonis, K., Daradoumis, T., Papadakis, S., & Simos, C. (2011). Evaluation of the Effectiveness of a Web-Based Learning Design for Adult Computer Science Courses. IEEE Transactions On Education, 54(3), 374-380. doi:10.1109/TE.2010.2060263

Boghikian-Whitby, S., & Mortagy, Y. (2008). The Effect of Student Background in E-Learning -- Longitudinal Study. Issues In Informing Science & Information Technology, 5107-126.

Lai, H. (2011) The Influence of Adult Learner’s Self-Directed Learning Readiness and Network Literacy on Online Learning Effectiveness: A Study of Civil Servants in Taiwan. Educational Technology and Society, 14(2), 98-106.

Module 5: Webliography

Simulations and Games in e-Learning

Ruth Clark and Richard Mayer make an excellent team combining backgrounds of learning research with multimedia research and have developed a bit of research on their areas of expertise. It is from their book, E Learning and the Science of Instruction’s fifteenth chapter that provides the background for this discussion on the use of simulations and games within an online learning environment.  Clark and Mayer (2008) preface their discussion by stating that there is more that we do not know about simulations and games than we do know; however, there are certain conclusions that can be drawn based on related application of skills and methods. They continue by defining games as (1) a competitive activity with a challenge to achieve a goal;(2) a set of rules and constraints; (3) a specific context. Simulations are defined as real world working model environments.  We begin with some questions, will games and simulations replace step-by-step learning and books? What is the evidence that such a thing might happen?

Summary

 The topic begins with an argument that younger learners or, digital natives, will benefit from online coursework designed to be game-like.  Issenburg, McGaghie, Petrusa, Gordon, and Scalese (2005) report that research studies on nearly 400 medical real-world simulation exercises were published in a four year period. Add to that the 2006 Learning Guild conference included over 12% of the presentations were game based.  Compare that to the 2011 conference where gaming presentations earned their own non-exhaustive category. Proponents site the popularity of games not only as an indication of pastime activity but also how the digital natives’ brains are programmed to work with information in a way Prensky (2001) described as twitch speed processing. Prinsky continues with two other points. He adds that digital natives thrive in high learner control, and in highly visual environments.

A second question is offered asking whether or not games and simulations teach.  This is where the strength of Clarke and Mayer shines. They report on a literature review of Gosen and Washbush (2004) that of 155 studies reviewed, not one met the criteria for sound research to make any sound conclusions.  They cite three more literature reviews with similar results.  What they found in support of game style learning is:

• The activity’s goals, feedback, and interfaces in simulation play must align with instructional outcomes.

•Instructional objectives must be integrated into game storyline so that learning is essential to progress

•Guidance, structure and focused goals are critical- no discovery learning

•Promote reflection on correct answers

•Manage the complexity – simple to complex with easy interface control and fading (total example to on your own) techniques provide stepped challenge.

There is still quite a bit not known about games and learning – cost/benefit, who prefers games, effective interfaces and how much interactivity are a few unknowns.

To answer the questions, it is determined that if a child is raised on games, it does not equate to more effective learning.  Can students learn with games? The answer is that if games can be designed with known effective methods it is certainly possible.

Reflection

            

The thirty-two-page reading was interesting as it provided more questions than answers. The reader familiar with other chapters in the text understands the media principles and cognitive load principles that would also apply to games and simulations as with any media. The first question, about replacing books does not seem to be answered directly but on a more assumption that it would be difficult to establish all objectives in game activities within a course and thus, the need for reading would still be necessary.  In the chapter it is mentioned that about 62% of all gamers are male and the average age is thirty-three with 25% over fifty! I am not sure that taking a format like gaming, which is an entertainment activity, can be used to teach objective skills and still be motivating. There may be a few possibilities but not in a widespread manner.

           

Naturally it is exciting to think of gaming your way through college or a course but it seems too unrealistic.  Perhaps an activity or two would be great. The process of learning has merit that may not be able to be duplicated in a game.  While I may be able to fly to the library in second life, I still need to be able to know how to access it through LU’s website! It sounds like a lot of extra work to me! 

Clark, R. & Mayer, R. (2008). E-Learning and the Science of Instruction (2^nd ed.). John Wiley & Sons: Pfeiffer.

Gosen, J., Washbush, J. (2004). A review of scholarship on assessing experiential learning material. Simulation and Gaming, 35(2), 270-293.

Issenberg, S., McGaghie, C., Petrusa, E., Gordon, D. & Scalese, R. (2005). Features and uses of high fidelity medical simulations that lead to effective learning: A BEME systematic review. Medical Teacher, 27(1), 10-29.

Module 4: Assessment

The evolution of mobile and online education has revolutionized and introduced challenges for assessment.  What are those revolutions? What are the challenges? What is current online and mobile assessment methods being used and what is their effectiveness?

     Mobile technology use in education is certainly a topic of interest as I learned this week. It started with an ISTE blog on the topic, how can we be innovative if everything has to be research-based? I got caught up in a debate with several people regarding using new technologies that the old guard as they called it, was hampering the use of mobile tech.  The majority were blaming the need for research stating that it was useless to spend time trying to find out, people knew it already worked, etc.- all of this while initially beginning the search on mobile technology and assessment!

     Few articles were found that specifically dealt with mobile technologies. Even articles as recent as 2005/6 focused on technologies like Palm pilots rather than the current idea of using iPads, cellular phones, slates or interactive pens; however, there were articles that provided a great overview of recent technologies in learning environments. The mobile revolution in learning centers around the ability to learn anywhere. That includes distance learning and virtual environments, interactive technologies that may be used online such as interactive pens, video conferencing, etc. These technologies have reached a point that now people can benefit from learning without having to travel to a school site for every class session. A major challenge is that students actually spend time with the content and are assessed effectively. Brown (2006) describes several types of turnkey learning assessment systems where students use mobile units to work and respond to questions focusing primarily on the benefits of immediate feedback and efficiency during testing. The systems described are primarily for face-to-face environments. 

     Anghel and Salomie (2003) describe a complex assessment system called JADE to be used in an eLearning environment focusing on the virtual learning environment (VLE) assessment and site communication issues, security issues, evaluation types, student answer analysis and grading as challenges to virtual learning assessment. They also describe the future use of mobile phones as part of the learning process and expansion of wireless agent systems. The focused assessment model describes questions of limited type. That would likely refer to multiple-choice questions.  McGuire (2005) describes the need for considering various types of assessment in online courses using both synchronous and asynchronous methods. She outlines the use of portfolios showcasing work and the use of annotation tools to create a work trail. Similarly, Fisher and Baird (2006) describe a dynamic and varied activity set to assess students rather than answering test questions in courses and working with mobile technologies to research, message and work together while learning. 

     Considering the articles found there is a great deal of discussion regarding the need for assessment in online courses that takes a different approach than a formal test format. The varied structure of synchronous and asynchronous activity not only sets the framework for dynamic learning activities but also provides evidence as to who is completing assignments.  The JADE program seemed to have merit but it seems to be more a process of learning in an online environment rather than how assessment should be done. The latter articles confirm one another on the use of various types of assessment building to a portfolio or showcase of work completed by students as a more accurate sampling of objective mastery. The articles mentioning alternative assessment mention the need for more such assessment. That would certainly infer that currently much of what is currently available online is more formal or traditional testing in virtual learning. The perception is that such testing is less effective even though it is more prevalent. It seems that it all depends on the purpose of the testing – to report a comprehensive evaluation or to be used as a review or learning activity. 

Brown, J. (2006, August). WORKING SMARTER NOT HARDER. T.H.E. Journal, 33(13), 32-34,36. doi: 1123828911

Fisher, M., & Baird, D. E. (2006). Making mLearning Work: Utilizing Mobile Technology for Active Exploration, Collaboration, Assessment, and Reflection in Higher Education. Journal Of Educational Technology Systems, 35(1), 3-30. 

McGuire, L. (2005). Assessment using new technology. Innovations in Education and Teaching International, 42(3), 265-276.  doi: 1157106301

Module 2: Theory of Design

Module 1 Intro Blog

Greetings to everyone in EDUC 633. This is my first attempt at authentic blogging! Our school district blogs on an intranet so my knowledge pertains only to that type of posting. My introduction video is located on YouTube at:

http://www.youtube.com/watch?v=5N2nSOsH1qA

Enjoy!