Literature Review - Using Videos for Effective Online Course Design

Background

• The Department of Organizational Performance and Workplace Learning (OPWL) is considering how to achieve consistency in the design of their online courses. To this end, the Research and Innovation Team collects and analyzes useful literature pertaining to the designs of online courses including usability, structure, organization, and navigation. 

Purpose

The purpose of this literature review is:

• To explore and understand the current effective course design practices and strategies.

• To identify and summarize effective course design strategies and help the stakeholders make an informed decisions about making a master template. 

Research Questions

• R1:  What are the learning outcomes in association with instructor-made videos in an online course?

• R2: How do instructors use videos in their online courses?

• R3: What is the optimal length of videos in an online course? 

Stakeholders

• The primary stakeholders of this literature review include eCampus Center staff.

• The secondary stakeholders are anyone interested in the benefits of the content at Boise State University

Executive Summary: Online Video Use

Research and Innovation Team, eCampus Center

Recommendations

Based on our review of the literature surrounding the use of videos in online courses, we recommend the following:

1. Overall, videos appear to be effective for student learning in an online course, specifically for learner engagement and knowledge retention. 

2. To enhance the likelihood of student success, instructors or instructional designers should utilize videos with relevant exercises and further integration of course materials.

3. Instructors should use videos for online courses that are relevant. Instructor-made videos are more effective than videos made by third-parties.  

4. Short videos are the most effective. Some research suggests that 10 or less minutes is the most optimal length, while others said that 6 minute-long-videos are the most appropriate. 

Research Questions

The Research and Innovation Team conducted a literature review with the following three research questions. Findings are summarized in the order of research questions.

• R1: What are the learning outcomes in association with instructor-made videos in an online course?

• R2: How do instructors use videos in their online courses?

• R3: What is the optimal length of videos in an online course?

R1: What are the learning outcomes in association with instructor-made videos in an online course?

• Jang and Kim (2014) stated that online clinical videos for learning clinical skills are beneficial. They provided mobile access as well, which has shown to be effective. The conventional belief states that the field of medicine may need hands-on learning, while the present study shows the opposite results: there have been positive learning outcomes reported by medical students.

• Giannakos, Chorianopoulos, and Chrisochoides (2015) found that there is a relationship between video navigation (repeated views) and the level of cognition/thinking required for a specific video segment. Results indicated that learning performance progress was slightly improved and stabilized after the third week of the video-assisted course. They also found that attitudes regarding easiness, usability, usefulness, and acceptance of this type of course remained at the same levels throughout the course.

• A literature review done by Sherer and Shea (2011) reaffirmed that using videos in the design and delivery of a college course, whether taught in a traditional, online, or a hybrid format, provides endless opportunities to enhance lectures, assignments, class discussions, or even exams, while simultaneously developing students’ skills.

• Delen and Willson (2014) stated that when an online video-based learning environment is designed and developed, embedding supplemental functions with potential users' needs in mind, could enhance learning by making the environment more interactive. Further, results suggest that learning environments with macro-level interactive functions (e.g., note-taking, supplemental resources, and practice questions) provide students with affordances to become actively engaged in their learning and to invest or spend more time in the learning process, resulting in enhanced or superior learning outcomes.

• Van der Zee, Admiraal, Paas, Saab, and Giesbers (2017) made a different claim. Contrary to a range of previous studies, the authors found strong evidence that subtitles neither have a beneficial nor a detrimental effect on learning from educational videos.  

• In a traditional study conducted by Choi and Johnson (2005), it is evident that learners’ motivation in terms of attention using the video-based instruction is higher compared to the traditional text-based instruction. In addition, the learners reported that the video-based instruction was more memorable than the traditional text-based instruction.

• In terms of using instructor-made videos, Rose (2009) found that students’ perception of using instructor’s video is very positive. All of the students surveyed expressed satisfaction with this method of instructor-learner interaction. Open-ended answers revealed that students felt more connected to the instructor when able to view instructor-made videos.

• Videos should be related to the assessments (either content or reviews) so that learners focus on the videos and try their best to learn. It's advisable to utilize this approach in the required videos, but not in optional videos (Hibbert, 2014). 

R2: How do instructors use videos in their online courses? 

• Tan and Pearce (2011) stated that students' use of OER videos is an effective and valid way of helping learning. The videos, however, can be very effective with the inclusion of further explanation and proper integration of materials. The video usage is also beneficial to initiate class discussions. Recommended "viewing" is another finding, indicating that a teacher recommending which videos to watch is a trustworthy way to spend time and efforts. 

• Lee et al. (2014) investigated how the display of time anchored comments (dynamic or static) and type of comments (content-related or social-oriented) influenced users’ perceived engagement, perceived social interactivity, and learning outcomes. Results show that dynamically displaying time-anchored comments can indeed enhance learners’ perceived social interactivity. Moreover, the content of comments would further affect learners’ intention of commenting. 

• Cross et al. (2014) reports a usability study on VidWiki, a crowdsourced video-editing platform for quality management, in which they found that with only a small investment of time on the part of viewers, it may be possible to make meaningful improvements in online educational videos. In specific, through the platform, users can improve the legibility of handwriting, correct errors, or translate text in videos by overlaying typeset content such as text, shapes, equations, or images. If one teaches disciplines dealing with constant changes, this might be a good way to maintain educational content up-to-date. 

R3: What is an optimal length for online videos?

• In terms of the optimal length of online videos, Schaffhauser (2015) reported that the optimal length of time for videos is 10 minutes or less (71%). However, 28 percent of educators specifically suggested that the preferred length is between 10 and 30 minutes, reflecting "the popularity of the 18-minute TED video format." Those results come out of the latest edition of "The State of Video in Education," produced by Kaltura, a company that sells video tools and services. The 2015 survey questioned 1,200 respondents in April in both higher education and K-12. 

• Guo, Kim, and Rubin (2014), conducted a large-scale analysis on video engagement, using data from 6.9 million video watching sessions across four courses on the edX MOOC platform. They measured engagement by how long students are watching each video, and whether they attempt to answer post-video assessment problems. As a result, they concluded that 6 minutes or less is the optimal length of online videos.

Annotated Bibliography

Jang, H. W., & Kim, K. J. (2014). Use of online clinical videos for clinical skills training for medical students: benefits and challenges. BMC medical education, 14(1), 56. (R1)

• Background - To examine student use and perceptions of online clinical videos for learning clinical skills and in preparing for OSCE (Objective Structured Clinical Examination).

• Findings - The present study confirms the overall positive impact of OSCE videos on student learning of clinical skills. Having faculty integrate these learning resources into their teaching, integrating interactive tools into this e-learning environment to foster interactions, and using mobile devices for convenient access are recommended to help students make more effective use of these resources

• Implications - The conventional belief states that the field of medicine may need hands-on learning, while the present study shows the opposite results: there have been positive learning outcomes reported by medical students. Limitations include a lack of interaction as well as lack of integration into curriculum. The authors also pointed out that provision of mobile access would be effective. 

Giannakos, M. N., Chorianopoulos, K., & Chrisochoides, N. (2015). Making sense of video analytics: Lessons learned from clickstream interactions, attitudes, and learning outcome in a video-assisted course. The International Review of Research in Open and Distributed Learning, 16(1). (R1)

• Background- This article reports research findings from their self-developed video analytics used in an open video-based class. 

• Findings - Through the video analysis, the authors found that there is a relationship between video navigation (repeated views) and the level of cognition/thinking required for a specific video segment. Results indicated that learning performance progress was slightly improved and stabilized after the third week of the video-assisted course. The authors also found that attitudes regarding easiness, usability, usefulness, and acceptance of this type of course remained at the same levels throughout the course.

• Implications - This research is unique, as the authors developed an open-source video analytics tool and provided findings based on the use of the tool. Considering the correspondence between the level of cognition/thinking and the size of respective peak (based on "wisdom of the crowd"), video-assisted platforms/lectures should accompany extra affordances like annotations, slower-pace, or even extra visual information to the students in these particular video segments. 

Sherer, P., & Shea, T. (2011). Using online video to support student learning and engagement. College Teaching, 59(2), 56-59. (R1)

• Background  - This article is to report current literature with their own classroom experiences, describe different types of assignments for which online video can be effectively used.

• Findings - Incorporating online videos in the design and delivery of a college course--whether teaching in a traditional classroom, an online format, or a hybrid--provides endless opportunities to enhance lectures, assignments, class discussions, or even exams while simultaneously developing students’ skills.

Delen, E., Liew, J., & Willson, V. (2014). Effects of interactivity and instructional scaffolding on learning: Self-regulation in online video-based environments. Computers & Education, 78, 312-320. (R1)

• Background - This article is to report research findings in response to the two research questions: 1. Do students recall more information and spend more time learning in the enhanced video-based learning environment compared to a common video-based learning environment? 2. Are learners' self-regulation strategies associated with self-regulated learning behaviors in an enhanced video-based learning environment?

• Findings -  The effects of interactivity on students' learning outcomes and behaviors were examined. A cross-sectional experimental research design was utilized. Interactivity affected students' performance in online video-based learning environments. Positive relationship between students' self-regulation and interactivity was found.

• Implications - Results highlight the importance of macro-level interactivity and self-directed (student-centered) learning features (e. g., note-taking, supplemental resources, and practice questions) in online learning environments. The general findings suggest that when an online video-based learning environment is designed and developed, embedding supplemental functions with potential users' needs in mind could enhance learning by making the environment more interactive. Instructional designers need to build learning environments with interactivity in mind. Study results suggest that learning environments with macro-level interactive functions provide students with affordances to become actively engaged in their learning and to invest or spend more time in the learning process, resulting in enhanced or superior learning outcomes.

Van der Zee, T., Admiraal, W., Paas, F., Saab, N., & Giesbers, B. (2017). Effects of subtitles, complexity, and language proficiency on learning from online education videos. Journal of Media Psychology. (R1)

• Background - This study investigates whether subtitles are beneficial or detrimental for learning from a video, and whether this depends on characteristics of the learner and the video. The researchers hypothesized that the effect of 2nd language subtitles on learning outcomes depends on the language proficiency of the student, as well as the visual-textual information complexity of the video.

• Findings - Contrary to a range of previous studies, the authors found strong evidence that subtitles neither have a beneficial nor a detrimental effect on learning from educational videos. In addition, the presence or absence of subtitles appears to have no effect on self-reported mental effort ratings. The effect of video complexity shows how video design can have a noticeable effect on test performance, either positively or negatively.  

• Implications - The visual–textual information of a video and especially the language ability of the student are both strong predictors of learning from content videos. By contrast, English subtitles neither increased or decreased the students' ability to learn from the videos. However, this does not lead to the conclusion that English subtitles should not be made available, as they are vital for students with hearing disabilities. Furthermore, students might prefer watching videos with subtitles for other reasons, even though this might not directly affect their learning. These results have several consequences for educational practice. First, it is important to make sure that educational videos are designed in such a way that they do not hamper the learning process. Specifically, the visual-textual information complexity of educational videos should not be too high, such as by having too much irrelevant information, or a suboptimal physical organization of information. Secondly, educators of online courses should be aware of the possible detrimental effects of lower levels of English proficiency and aim to help these students as much as possible; merely providing English subtitles is not enough to guarantee accessibility.

Choi, H. J., & Johnson, S. D. (2005). The effect of context-based video instruction on learning and motivation in online courses. The American Journal of Distance Education, 19(4), 215-227 (R1)

• Background - The purpose of this study was to investigate the potential of a constructivist approach to context-based video instruction for enhancing learning

• Findings - There was a significant difference in learners’ motivation in terms of attention between the video-based instruction and traditional text-based instruction. In addition, the learners reported that the video-based instruction was more memorable than the traditional text-based instruction. This study implies that context-based videos in online courses have the potential to enhance learners’ retention and motivation

• Implications - Video-based instruction using a constructivist approach appears to be effective. The videos were context-based instruction, more of a case with authentic problems involved. Instructional designers might want to try similar videos, rather than merely explaining 

Rose, K. K. (2009). Student perceptions of the use of instructor-made videos in online and face-to-face classes. Journal of online learning and teaching, 5(3), 487. (R1)

• Background - This article describes results from a study conducted to examine students’ perceptions about the use of instructor-made videos that provided explanations of course assignments or syllabus requirements, discussed weekly topics, reviewed for exams, and answered student questions in video format in both a 100% online course and in several face-to-face courses. 

• Findings - Student perceptions of this instructional strategy were overwhelmingly positive, with all of the students surveyed expressing satisfaction with this method of instructor-learner interaction. Open-ended answers revealed that students felt more connected to the instructor when able to view instructor-made videos.

• Implications - Instructor-made videos can bring a positive result. Instructional designers and course developers should think about implementing videos made by instructors. 

Kolowich, S. (2010). The Human Element

• Background - The purpose of this study was to report effectiveness of using instructor-made videos online facilitated in the Blackboard environment.

• Findings - Professor Hersh at UC Santa Barbara started using Blackboard with incorporating instructor-made videos to enhance the presence of human element. Student engagement in general has improved. 

• Implications - It is worthwhile to think about implementing videos made by instructors so that students are more engaged in the class activities by improving social interactions and social presence. 

Kaltura (2015). The state of video in education 2015. (R2)

• Background -To get better insight into how video is perceived and used across educational institutions today, as well as the latest thoughts on digital/video literacy, best practices, and future use cases, Kaltura undertook their second annual online survey during April 2015.

• Findings - This report provides findings in the categories of (1) digital literacy, (2) video in the classroom (use), (3) using videos outside of classroom, (4) the source of videos used in class, (5) how to increase the use of videos, (6) video ROI, and (7) some interesting quotes from respondents. In essence, students and instructors have very good or good digital literacy, active use of videos in classroom is still in its infancy, videos are widely used for many different purposes (for marketing, education, communication, and even admission processes), free videos are the most frequently used source of videos, easy-to-use is the most effective motivator to increase video usage, and 79% of institutions employ ROI measurement to analyze their use of videos. 

• Implications - These findings may be informative for instructional designers and even instructors interested in knowing more about others' teaching practices. Might be a good reference to start a dialogue with faculty at the beginning of course development/design.

Tan, E., & Pearce, N. (2011). Open education videos in the classroom: exploring the opportunities and barriers to the use of YouTube in teaching introductory sociology. Research in Learning Technology. (R2, R3)

• Background - This study addresses that if a student has free and easy access to the same materials that are being used to teach them in class, how does this affect their perceptions when they are presented with this material in the classroom environment? What are their perceptions regarding the perceived value for money, efficacy, and authority of the material? This research specifically investigated the use of open education videos in the classroom environment and their incorporation into an associated space in the virtual learning environment.

• Findings - Results suggest that students watching the OER videos is an effective and valid way. The videos, however, can be very effective with inclusion of further explanation and proper integration of materials. The video usage is also beneficial to initiate class discussions. Recommended "viewing" is another finding, indicating that teacher recommending which videos to watch is a trustworthy way to spend time and efforts. 

• Implications - Based on the results, several implications were made. First, the role of teacher should focus on retention of imparter of knowledge. The video usage can also enhance social interaction by students sharing the videos through SNS. Lastly, this finding might be dependent on disciplines, but in sociology, watching videos was helpful to understand different perspectives with proper rationales. Instructional designers and course developers can facilitate the use of videos for such reasons as fostering a diversity of understandings/perspectives on a certain topic, initiation of discussion forum, and creating the bond between students. 

Lee, Y. C., Lin, W. C., Cherng, F. Y., Wang, H. C., Sung, C. Y., & King, J. T. (2015, April). Using time-anchored peer comments to enhance social interaction in online educational videos. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 689-698). ACM. (R2, R1)

• Background - This article investigates how the display of time anchored comments (dynamic or static) and type of comments (content-related or social-oriented) influenced users’ perceived engagement, perceived social interactivity, and learning outcomes.

• Findings - Results show that dynamically displaying time-anchored comments can indeed enhance learners’ perceived social interactivity. Moreover, the content of comments would further affect learners’ intention of commenting. 

• Implications - Based on the findings, the authors made several suggestions for design. (1) Dynamic display tends to work better than statistic display, (2) Social-oriented comments are desirable, and (3) Video-centered, time-anchored comment exchange supports collaborative learning.

Cross, A., Bayyapunedi, M., Ravindran, D., Cutrell, E., & Thies, W. (2014, February). VidWiki: enabling the crowd to improve the legibility of online educational videos. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing (pp. 1167-1175). ACM. (R2, R3)

• Background - This article reports a usability study on VidWiki, a crowdsourced video-editing platform for quality management.  

• Findings - Results suggest that with only a small investment of time on the part of viewers, it may be possible to make meaningful improvements in online educational videos. In specific, through the platform, users can improve the legibility of handwriting, correct errors, or translate text in videos by overlaying typeset content such as text, shapes, equations, or images.

• Implications - In the case of video-heavy online courses requiring frequent updates and maintenance of terms/languages/graphics, etc., using the WikiVid approach can be beneficial for the users and instructors. Similar to Wiki document, it is a collaborative effort, in which both instructors and students contribute to the maintenance of content and quality assurance. 

Hibbert, M. C. (2014). What makes an online instructional video compelling?. Educause Review Online. (R3)

• Background - To learn more about compelling video, a team at the Columbia University School of Continuing Education examined analytics from the video hosting platform and recruited 10 students to participate in in-depth interviews.

• Findings - This article introduces two key takeaways: (1) Video has supported education for many years, and in online courses instructional videos are often a key component. (2) Findings show that videos with high numbers of views usually have a direct connection to course assignments (or course assessments).

• Implications - Videos should be related to the assessments (either content or reviews) so that learners focus on the videos and try their best to learn. It's advisable to utilize this approach in the required videos, but not in optional videos. 

Schaffhauser, D. (2015) Latest Word on Optimal Length for Education Videos, The Journal. (, R4)

• Background - To report an optimal length of video used in education. 

• Findings - The report shares the findings of effectiveness of using videos online. According to the report, the optimal length of time for videos is 10 minutes or less (71%). However, 28 percent of educators specifically suggested that the preferred length is between 10 and 30 minutes, reflecting "the popularity of the 18-minute TED video format." Those results come out of the latest edition of "The State of Video in Education," produced by Kaltura, a company that sells video tools and services. The 2015 survey questioned 1,200 respondents in April in both higher education and K-12. 

• Implications - Considering student attention span, making videos about 10 minutes or less seems reasonable. Instructional designers and course developers should keep this length in mind to achieve optimal learning outcomes as desired. 

Guo, P. J., Kim, J., & Rubin, R. (2014, March). How video production affects student engagement: An empirical study of mooc videos. In Proceedings of the first ACM conference on Learning@ scale conference (pp. 41-50). ACM. (R4)

• Background - To examine appropriate length of online videos for instructional purposes. 

• Findings - The authors found that shorter videos are much more engaging, that informal talking-head videos are more engaging, that Khan-style tablet drawings are more engaging, that even high-quality pre-recorded classroom lectures might not make for engaging online videos, and that students engage differently with lecture and tutorial videos.

• Implications - The authors make a few recommendations: (1) Instructors should segment videos into short chunks, ideally less than 6 minutes, (2) Record the instructor’s head and then insert into the presentation video at opportune times, (3) Try filming in an informal setting where the instructor can make good eye contact, since it costs less and might be more effective than a professional studio, (4) Record Khan-style tutorials when possible. If slides or code must be displayed, add emphasis by sketching over the slides and code using a digital tablet.