Does Quality Matter? Yes, Quality Matters™!

When I travel to new locations, I love to check out the local food scene. I guess you could call me a foodie. Sometimes I really get into it: I research restaurants before embarking on a trip and add them to a special collection in Yelp. After returning to my hotel at the end of the day, I sit and write a restaurant review. As I write, I love to think about all aspects of the dining experience. How was the service? Was I promptly greeted and seated? How was the food? Was it fresh and tasty? How long was the wait for my food to be prepared and served? Was my server attentive? You get the idea. While writing these reviews, I apply my own personal opinions about the overall quality of the restaurant dining experience.

Basing a review on opinions is fine for online restaurant reviews, but what about something more important than a meal? If we’re discussing something more important, say, the quality of an online course, we should probably base it on something besides personal opinion. When you design, develop, and teach an online course, how do you know it’s “high quality”? If someone asked you what a quality online course is, how would you define quality?

In mid-2020, my colleagues at the Center for Learning Design (CLD) and I began to ask ourselves some of the same questions. Primarily we wanted to know how to define “quality” for evaluation of online courses. And, after some research, we identified quality benchmarks to guide us.

How Do Quality Benchmarks Impact You?

Most quality standards are already built into Engineering for Professionals (EPs) course design and development process. Translation: this means little or no additional work on your part is required to meet minimum quality standards. When you work with CLD staff to design and develop your course, it will already meet the minimum quality benchmarks.

EP is in the early stages of adopting standards, but in the future, you may have an opportunity to participate in reviews of peers’ online courses. In addition, your reviewed courses will have quality designations (e.g., “quality,” “high quality,” and “exemplary”) which you can proudly display in your course and other areas online. If you want to exceed the minimum, we will support you in your efforts to achieve even higher than minimum course design quality.

What are the Benchmarks?

I’ll now unveil the quality benchmarks. In late 2020, we subscribed to Quality Matters™ (QM)–a research-based, nationally recognized, peer-review process that includes a broad set of online course quality standards. These standards focus primarily on course design, and less on teaching. Quality Matters includes eight General Standards.

2 – Quality Matters(tm) Eight General Standards Each general standard is further divided into related specific standards. The QM rubric provides annotations for each specific standard so course reviewers can easily determine whether the course design meets each standard. In addition to the QM specific standards, we have also identified four EP-specific quality standards. Each of these EP-specific standards fits into one of the QM general standards, and, like the QM specific standards, is supported by research.

Here’s a look at the EP-specific standards.

3 – EP-Specific Course Design Quality Standards

How and Why Did We Select QM?

Quality Matters is not new to EP or the Johns Hopkins University. In Fall 2010, two EP online course development projects were grant-funded. One stipulation was that each course be formally QM reviewed (by external subject matter expert peers). As a result of this positive experience, we fully adopted QM and embedded its standards in our course design and development process. In Spring 2014, we terminated our QM subscription, but preserved most of its standards in our online course design and development process.

In mid-2020, we began researching how to define online course quality. We analyzed six other higher education institutions’ course quality definitions and determined that most reflect the spirit of, if not the actual, Quality Matters rubric. We appreciated some institutions using the QM rubric and adding specific standards to it. We re-adopted Quality Matters in October 2020, with the intent to formally implement it in Spring 2021.

In late 2020, we formed eight working groups comprising faculty, CLD staff, and Center for Digital and Media Initiatives (CDMI) staff. Working groups were tasked to closely evaluate an assigned QM general standard to identify EP-specific quality standards not already included in or covered by QM standards. At the conclusion of working groups’ efforts, we identified 13 EP-specific standards. After further review and iteration among CLD staff, 4 EP-specific standards were identified.

Conclusion

We’ve known for years that our online engineering courses were top-notch. At the time of print, the Johns Hopkins Whiting School of Engineering ranked #11 in the U.S. News & World Report’s “Best Online Masters in Engineering Programs.” With the adoption of quality benchmarks, we now have research-based standards for how we arrived at this lofty rank, and impactful strategies we can integrate to propel us to even higher spheres of quality.