Level+1+evaluation-online+course

=What do learners think of your e-learning courses?= That’s a good question because one of the most vexing challenges in e-learning is getting Level 1 feedback (reaction) from learners. The challenge is two-pronged: encouraging learners to respond and determining what to ask.

Encouraging Learners to Respond
Because the response rate to Level 1 surveys in classroom courses approaches 100 percent, the low response to Level 1 surveys in e-learning courses (sometimes less than 1 percent) is especially troublesome.

Part of the challenge is environmental. Because instructors control the environment and can place subtle pressure on students to complete the survey, high response rates are relatively easy to obtain.

E-learning instructors lack such direct control over learners-especially learners taking asynchronous courses-and need to work harder to encourage students to complete Level 1 reaction forms.

Some techniques for doing so:
When learners complete a course, display a pop-up that says “So we can provide you and future learners with the best possible learning experience, please take a few moments to complete the post-class survey.”

Immediately below, display buttons that say Take the Survey, Remind Me in a Day, and Exit. Those that choose “remind me in a day” should be sent a reminder e-mail, with a direct link to the survey (or the survey embedded in the e-mail message).

Rather than distribute the survey before learners exit a course, send a separate e-mail to each learner immediately after they complete the course. Ask them to respond, and offer an opportunity to receive a premium (such as a gift certificate for a latte or a souvenir teeshirt) as an incentive to do so within 3 days.

Choosing Questions to Ask
The other issue in Level 1 evaluation for e-learning is asking relevant questions. Following is an annotated sample of a survey that asks questions appropriate to the elearning environment.

1. In a word, how would you describe this course: _ (Logic behind the question: Solicits qualitative feedback as well as assess whether students accidentally transposed the numerical scales (if a student responses “Excellent” to this question, then circles “1” to the next question, chances are the student misread the number system.) 2. Using a number, how would you describe this course: 1 2 3 4 5 Abysmal Average Outstanding (Logic behind the question: Provides quantitatively-oriented organizations with the numbers they seek.) 3. How much did you know about SUBJECT NAME before taking this course: 1 2 3 4 5 Nothing at all Some A lot 4. After taking this course: 1 2 3 4 5 Nothing at all Some A lot (Logic behind the question: Although this question does not assess actual learning, it does assess perceived learning. That is, do learners perceive that they learned something in the elearning program.) 5. How likely are you to use the information taught in this course on the job? 1 2 3 4 5 Not at all Not sure Very likely (Logic behind the question: e-learning promises to provide just-in-time skill development for the job, one assessment of its effectiveness is whether learners perceive that they can apply these skills on the job.) 6. Describe the experience of taking this course online: I have taken online courses before (circle one) Yes No

Rate the quality of the Content 1 2 3 4 5 Thrown together OK Superb

Video (Note: only ask if relevant) 1 2 3 4 5 Barely audible OK Crystal clear

Audio (Note: only ask if relevant) 1 2 3 4 5 Barely audible OK Crystal clear

Rate the usefulness of the exercises 1 2 3 4 5 A waste OK Insightful

Linked websites 1 2 3 4 5 Distracting OK Enlightening

Glossary 1 2 3 4 5 Distracting OK Enlightening

Rate the availability of the course 1 2 3 4 5 Perpetually down OK Always available

Instructor (for the purpose of contacting by -mail) 1 2 3 4 5 Missing in action There Dependable

Overall, taking this course was Convenient 1 2 3 4 5 No way I guess so Totally

Compared to a classroom course 1 2 3 4 5 Much worse The same Much better

I’d take another course online 1 2 3 4 5 I'd rather have surgery without an anesthetic If I must In a heartbeat

(Logic behind the question: Ask about various aspects of the course that are important to you, just as you might ask questions about the appearance of the handouts, convenience of the location, and quality of the facilities in a Level 1 evaluation of a classroom course.)

7. The best part of this course was: _____________________________________________________________________________________________ 8. This one thing that could improve this course most is:________________________________________________________________________ ____ (Logic behind the question: The last two questions provide qualitative feedback on the course and help prioritize work in a revision. Items mentioned on several responses warrant attention. For example, if 10 learners respond that the exercises are excellent, you should probably incorporate exercises like these in similar courses in the future. Similarly, if 18 learners comment that the pace of the course is slow, the pace should be quickened in the next revision.)