March 29, 2017

Quality Assurance in Blending Learning


ONE SENTENCE SUMMARY

Standards-based instruments with course component checklists are useful for reviewing instructional course design but can not adequately assess instructor experience and student-teacher interaction.


REACTION    

The BlendKit Reader Chapter 5: Quality Assurance is a fitting conclusion to an already excellent series of chapters. The chapter presents a balanced view on the merits and limitations of course standards review instruments and encourages a holistic approach as part of the evaluation process.  

Caveat emptor - course design rubrics and checklists

Course design rubrics such as UCF’s Blended Course Peer Review form (PDF) and the OSCQR Rubric and Process (page links to a list of excellent resources including rubrics and examples) are important tools for assessing course design. UCF’s rubric is designed for peer review and lists criteria for implementation of a blended course. Peer review can result in engaging and "meaningful dialog about the course and teaching/learning process". 

Checklists and rubrics/review forms are especially helpful to faculty who are new to blended or online learning or would like to do more than a simple document transfer from traditional to online. Based on cumulative experiences these tools address most aspects of course design and serve as a reminder to what might be considered for inclusion. They are typically broken down into easily assessed components. 

However, as the BlendKit Reader points out there are limitations.  

A caveat in using the same standards rubric repeatedly over time:

We all like the ease of copying and pasting to repurpose documents such as revised syllabi but the BlendKitReader points out the danger of repurposing standards-based rubrics without critically evaluating each criteria: 

The challenge of implementing a one-size fits all rubric would be daunting: 

Complementing or blending checklists

The chapter discusses the merits but limitations of measuring instruments, which can lead to a "focus on the designed (online) environment of the course to the exclusion of the experience of instructors and students in the teaching/learning process (whether online or face-to-face). ” So what to do if these rubrics are not sufficient? One start is to examine the Monterey Institute course evaluation (PDF) provides a balance between "most checklist-based reviews and the intensity of the criticism model". This screenshot displays part of evaluation tool. The document was last updated in 2010 so downloading the file for personal archiving is recommended. As pointed out in the Faculty Development section below there are many opportunities to obtain feedback from peers and students such as the Muddiest Point. 

Effective teaching is difficult to measure with rubrics and is ideally assessed holistically to consider the student-teacher and student-student teaching and learning:

Faculty Development

In addition to taking the BlendKit course there are many teaching resources that can assist faculty with course design and teaching strategies. Most of these resources are written by faculty for faculty:


but this chapter quote stands out: “One might argue that faculty in meaningful dialogue with other faculty about the teaching/learning process is the most effective form of faculty development with everything else being merely layers of facilitation.” That is why peer-to-peer review, training, attending conferences, and informal discussions are so invaluable to developing and improving blended/online courses. As with blended courses, faculty initially meeting with other faculty in person is optimal with the option to follow up face-to-face or online. The chapter suggests developing a journal with personal teaching goals, benchmarks, and collect formative feedback such as peer review and self-evaluation, online suggestion box, one-minute threads, polling, and focus groups. Note taking tools such as Evernote or OneNote fit well into developing an ongoing journal and collection of ideas. 

MOST IMPORTANT TAKE-AWAYS
  • High quality faculty development is crucial to effective blended programs
  • Develop a journal with personal teaching goals, benchmarks, and formative feedback from a variety of tools and methods
  • Assess different course design rubrics to determine which criteria meets your institutional and teaching needs but with a critical eye on what appears to be commonly accepted.
  • Balance rubrics with other evaluation methods that assess teaching effectiveness
MY FAVORITE QUOTES
  • “These limitations have to do with the prescriptiveness, credibility, scope, and atomism of such standards groupings.” 
  • “Nearly all sets of blended/online course standards bear the imprint of an overt instructional design emphasis (e.g., instructional objectives, constructivist influence, technology-dominated, etc.). ”
  • “One might argue that faculty in meaningful dialogue with other faculty about the teaching/learning process is the most effective form of faculty development with everything else being merely layers of facilitation.”
HELPFUL LINKS





at 9 comments:
Labels:

March 23, 2017

Blended Content and Assignments


ONE SENTENCE SUMMARY

Designing an integrated (F2F and online) blended course with a focus on a modular structure of diverse content and learning activities requires careful planning, experimentation, and ongoing feedback. 



REACTION    

Opportunities (and Challenges)

The list of Technology tools for teaching and learning is seemingly endless, continuously refreshed with updated and newer tools, gradually leaving others less tended to behind. There is the consideration of ease-of-use cost, obsolescence (e.g. Google Reader, Dropio, Zaption), and most significantly its potential for learning and assessment. Faculty typically adapt their school's learning management system and after an initial comfort level explore technologies that complement what the LMS lacks or insufficiently provides. 

Help for Faculty

There are number of teaching focused Twitter feeds and chats, blogs, conferences, and courses (in addition to BlendKit of course) that can assist faculty with course design, teaching strategies, and choosing technology materials and tools that have been tested and evaluated by others.  A partial list:
Technology Tools and Outside Course Resources

There are blogs, Tweets, free courses, and these teaching with technology lists:
but there is no substitution for face-to-face interactions (meetings, conferences) with faculty, instructional technologists and designers, and students. 

Course Design (including integration into LMS)

The chapter points out that uniformity guides students through the content and helps reinforce learning. To add, a diversity of content provides students with different experiences and perspectives. Instructors can align their structured content and activities into learning management system modules, which are included in most new systems such as Canvas, which will be used in the examples in this post. HTML pages, links, documents, lecture recordings, and videos can be added as course materials to these modules. For activities instructors can include quizzes, discussions, conferences, and collaborations.  


The Canvas.net course United States History is module based as shown in the diagram below.  Regardless of LMS modules and content/activity tools, course organization, learning resources, student engagement, and assessment depend on teaching excellence.



Content creation Tools

The chapter referred to content creation/screencasting tools such as Jing, Camtasia, and Audacity (audio). Snag-it, Screenflow, and Microsoft Office Mix are newer tools that allow instructors to add embedded video, audio, call-outs, and annotations to PowerPoints. Although screen recording software has evolved with more features and improvement, the time to create and produce lecture videos can be considerable. Some instructors prefer the simplicity of PowerPoint’s build-in recording tools to add audio annotations for each slide. However, this process requires students to download the final PowerPoint lecture to their desktop and having the software available to open and play. 

Ideally, enhancements to online presentations tools such as Google Slides will evolve to include live recording and embedding multimedia onto slides such as audio, video, and annotations so that live lectures can be viewed and played online.  

MOST IMPORTANT TAKE-AWAYS
  • Designing courses by modules facilitates integration of F2F and online environments
  • Uniformity guides students through the content and helps reinforce learning.
  • technologies are adopted more readily when cast in the “context of existing teaching and learning activities
  • freely available online learning resources provides an opportunity for educators to either link to or create derivative works based upon many educational resources
  • Implementing different and new technologies for the first time in a course can be challenging. Start simple to help ensure success for students and faculty and then explore other tools. 
MY FAVORITE QUOTES

“I'd devote more attention to integrating what was going on in the classroom with the online work. This was true even though the project's faculty development sessions repeatedly emphasized the importance of connecting in-class material with out-of-class assignments.”

"When applied to learning, certain activities can be utilized to greater effect when appropriate matching occurs between: the technology used, the learning desired, the context of use, the learner experience, the instructor experience, and the nature of content."

HELPFUL LINKS
at 73 comments:
Labels:

March 17, 2017

Blended Assessments of Learning



ONE SENTENCE SUMMARY

Different modes of assessments based on course objectives provide faculty with comprehensive learning portfolios of their students and promote academic integrity through prevention. 

REACTION    

My experience teaching (face-to-face in the classroom or computer lab) took place before learning managements systems were available. Assessments included exercise and project assignments, in-class exams, and the use of Minitab. As an instructional technologist I have had the opportunity to work individually with faculty to assist in the effective use Canvas (learning management system) and other cloud based resources to assess student learning. Some of my thoughts are below:

Informal Assessments

Providing multiple opportunities to participate in informal assessments lessens the stress for students but these assessments should be purposeful and include faculty feedback. Some tools to consider:
  1. Most learning management system include practice quizzes, which provides preliminary feedback to faculty and help students prepare for actual graded exams. 
  1. I look forward to sharing the One sentence summary with faculty who have not used this assessment.  A step-by-step procedure :
Description. This simple technique challenges students to answer the questions "Who does what to whom, when, where, how, and why?" (represented by the letters WDWWWWHW) about a given topic, and then to synthesize those answers into a simple informative, grammatical, and long summary sentence.
    1. Select an important topic or work that your students have recently studied in your course and that you expect them to learn to summarize.
    2. Working as quickly as you can, answer the questions "Who did/Does What to Whom, When, Where, How and Why?" in relation to that topic. Note how long this first step takes you.
    3. Next, turn your answers into a grammatical sentence that follows WDWWWWHS pattern. Not how long this second step takes.
    4. Allow your students up to twice as much time as it took you to carry out the task and give them clear direction on the One-Sentence Summary technique before you announce the topic to be summarized
  1. The Muddiest Point is a simple but effective way to determine what point was least clear to students. However, if implemented regularly (e.g. once weekly or after each class) summarizing and acting on student feedback is essential for making the assessment meaningful. 
  1. Student generated questions: Online discussions provide a forum for students to share questions, provide feedback, and collect their questions into shared documents such as a Google doc, spreadsheet, or presentation. Using the groups tool available in many learning management systems students could write questions for their own group and then quiz other groups who have not seen the questions. In turn faculty can assess these questions and determine which are suitable for practice quizzes or graded assessments. 
Formal Assessments

Developing different types of online assessments: summative, formative, multiple choice, short essay, practice, graded exams, assignments, discussions, and group presentations to measure student learning offers several advantages:
  • Help maintain academic integrity by not focuses on one type of assessment such as multiple-choice quizzes.
  • Provides an in-depth assessment of students and determines their strengths and weakness based on the type of assessment.
  • Allows faculty to collect feedback on what works best for their students in achieving course objectives.
Exams and quizzes

Multiple choice exams with an emphasis on application and higher-level thinking are difficult to write effectively and can take significant time. Taking advantage of resources such as 10 examples of question improvement and Examples of Multiple Choice Items at the Levels of Bloom’s Taxonomy help faculty with this difficult, time-consuming process.

Publisher test banks provide faculty a reasonable efficient way to create quizzes that can be used for practice and low-stakes assessment (e.g. ensure that students are completing assignments such as readings). In addition faculty can curate the best questions and write those of their own to build their own improved question banks. 

Essays/Academic Prompts

In a small college where class sizes are typically less than 30 students, the focus is less on multiple-choice exams and more on essays, projects, and presentations. 

Learning management systems such as Canvas provide faculty different tools to assess student contributions in writing: such as essay or paper assignment submissions, links to an online blog, or interactions with other students (wiki, discussions, document collaboration in real-time). One of Canvas strengths is the ease in assessing documents with SpeedGrader, which provides inline annotation, text and multimedia commenting), and grading. Students can comment on their assessed papers providing a trail of student-faculty interaction.   

Projects/Authentic Tasks

Audio and video recordings provide opportunities for different modalities of student expression, which can be submitted as assignments and included in discussions. Most learning management systems support these features. In Canvas students can record media with the browser, upload video or audio recordings as files, or use their smartphones to create multimedia recording and submit them. In addition their is the fluency of commenting and feedback that is an essential part of evaluating student recordings. 

ePortfolios such as those for teacher education are a promising but often underutilized assessment that can be summative, formative, and authentic. To see testimonies read The Benefits of E-portfolios for Students and Faculty in Their Own Words. Although there are significant benefits to students in completing ePortfolios, it is essential that student work be consistently assessed by faculty, mentors, peers, and advisors with feedback. Such consistent feedback can take considerable time and must be planned in advance by reviewers. 

Promoting Academic Integrity

Respondus LockDownBrowser, which integrates with most learning management systems, is advertised as a tool to prevent student cheating. It was added to Canvas at Elmira College in response to faculty concerns about academic integrity in high-stakes exams taken in large classrooms. However, online proctoring tools such as LDB can also be framed in a more directed, student-centered approach: a tool for reducing outside distractions and focusing student interaction with the assessment. Proctored exams using tools such as LDB work best in a F2F environment. 

Promoting Academic Integrity in Online Education is a special PDF report from Faculty Focus that provides a variety of helpful articles such as the catchy “91 Ways to Maintain Academic Integrity in Online Courses"

MOST IMPORTANT TAKE-AWAYS
  • Based on previous faculty experiences students taken a blend of online and face-to-face assessments have better outcomes.
  • Writing multiple-choice questions that effectively measure student knowledge (especially) higher-level thinking is difficult and requires time, experience, and re-writing (based on student responses). 
  • Examples of assessment strategies provides faculty a framework to choose which assessments meet the needs of their students
  • “Authentic assessment—assessing student abilities to apply knowledge, skills, and attitudes to real world problems—is not only possible in an online environment; it is getting more popular.”
  • Define course objectives so they are clear to students, determine how they will meet these objectives (e.g. essays, presentations, audio recordings) and choose the appropriate assessments (e.g. online exam, blog, media assignment) to evaluate their learning.
MY FAVORITE QUOTE

“The most crucial step needed in each unit of instruction is the preparation for students’ transfer of learning to new contexts. If learning is not transferred from the place of learning to practical application, there can be no positive return on investment of the time needed to create, implement, and evaluate instruction”

HELPFUL LINKS
at 23 comments:
Labels:

March 05, 2017

Understanding Blended Learning


One sentence summary

An effectively designed blended course combines the strengths of face-to-face and online learning and offers new learning opportunities for students and development opportunities for faculty. 

(Kudos to the UCF team for a great chapter! I used the eBook version for note taking and highlighting on my iPad.)

Reaction

Many years ago I taught face-to-face when online was only by dialup (LOL) but have not taught blended or fully online courses. As an instructional technologist I have worked extensively with faculty who are new to Canvas and teaching blended or online for the first time. As a starting point for my reaction I thought it helpful to think of blended/online courses that reflect lack of experience and expertise in teaching online.

Blended courses that tack on F2F courses typically offer in-class sessions first and last weeks of the course. The last week often is used for in-class presentations, proctored exams, or other methods of final assessment. The remaining traditional part of the course is converted to learning management objects: PowerPoint slides, readings (PDF’s or links), online assignments (drop boxes), and discussion forums to replace in class conversations.  Such courses often lack objectives, integration with the F2F and online components, student outcomes, and an environment that promotes active online engagement and assessment. The focus is often on mastery with the LMS and not how  

Re-designing a (blended) course from scratch, not knowing where to start, and how to define course components is challenging. In Educause’s Report on Blended Learning lists competencies (with example behaviors) and guiding principles that help guide faculty through the development of a blended course. For example, the active learning competency involves planning and implementing tasks that engage the student in an active role (e.g. the instructor encourages students to interact with each other by assigning team tasks and projects where appropriate) and the emulate the student experience principle: use the blended learning model to teach instructors how to teach blended learning courses. 

The Quality Matters Rubric is a useful tool for assessing course design through 8general standards organized into 43 review standards. But this article: What Faculty Need to Know About ‘Learner Experience Design’ puts into perspective such checklists (with points) as the QM rubric by focusing on three big ideas: relationships matter, tech should increase - not replace - social interaction, and walk in students’ shoes. 

Web sites such as UCF Examples of Approaches and U Waterloo examples of blended courses provide concrete examples that instructors can refer to as they design their course.

Most important takeaways

“There is clear consensus that the best strategies for design begins [sic] by clearly defining course objectives before coming up with course activities, assignments and assessments”

"A superficial understanding of blended learning is that it simply adds non-F2F elements into the traditional course structure. But this most often results in a dysfunctional phenomenon known as the“course-and-a-half.”

Meeting with students (in-person, by phone, by conference) before the course starts can help instructors assess student competencies and needs in advance, especially important if F2F are few. 

My favorite quote

In 2002 Troha asked “why do so many blended initiatives turn into frustrating boondoggles, consuming far more time... than anyone anticipated?” 

Helpful links

Not so helpful

at 6 comments:
Labels:
Subscribe to: Posts (Atom)

Distraction Free Reading

You are not alone if you have struggled to keep your focus reading through articles on the web that are bombarded with still and animated...