Seeking Validation in Learning

The traditional implementation of a learning management system (LMS) is a solved problem domain. There is a certain amount of rigor involved, typically driven by compliance requirements, as well as some configuration steps, probably an integration or two, and various user testing. Once that’s done, it’s marked “complete” and the system can be sent on its way.

Validation turns the dial up to 11.

Validation is a process that usually comes into play with heavily regulated industries. Healthcare-related companies and power companies are prime examples. The driver for an increased amount of rigor is the safety of customers (for things like medical devices) and employees (for places like nuclear power plants). The failure of software in these fields could easily result in patient death or employee injury as well as opening up companies to costly legal action.

Lawsuits and investigations often follow these types of incidents. One of the first things that happens during an investigation is that the training history for everyone involved is pulled to ensure that everyone was properly trained. By extension, these audits also require the documentation that proves the learning system was properly implemented and supported.

Testing is not the focus of the software validation process; the emphasis is on documentation and sign-off during all phases of the implementation. Because documentation is central to proving that the system is correct, the relevant regulatory body (the Food and Drug Administration [FDA] for the health-related industries) develops and publishes standards that need to be followed to the letter. This helps to ensure that any potential for error is reduced during the project implementation and that risks are properly accounted for and minimized.

The key to the process lies in the word “validation.” The FDA, a major driver in validation process definitions, says that this is “confirmation by examination and provision of objective evidence that software specifications conform to user needs and intended uses, and that the particular requirements implemented through software can be consistently fulfilled.” [FDA, 2002] Essentially, the validation process is testing (manually or automatically) to ensure that the requirements for the project have been met.

The key steps to software validation during this process are the installation qualification (IQ), operational qualification (OQ), and performance qualification (PQ). [FDA, 2002] These steps have specific definitions, but when boiled down, they are:

• IQ – A list of steps and commands taken when installing the software components
• OQ – Testing that the system performs under normal parameters
• PQ – Testing that the system performs to specification when the parameters are pushed hard

During the implementation of a validated learning system, the IQ and PQ steps are the most common. A combination of OQ and PQ is what most would equate with system testing in a typical non-validated implementation. These steps are stringently documented and require multiple sign-offs; this rigor is one of the differentiators of the validation process.

Of course, automation can help gain some efficiency. This is usually realized by automating formal testing during the OQ and PQ phases. Some companies (including GP Strategies) offer a full-blown Validation Toolkit product to test your project requirements to ensure that you are meeting them and that they are checked and tracked methodically.

The tradeoff to this process is, of course, time. The validation and sign-off steps needed add to the project schedule and are usually nonnegotiable in a truly validated project. You gain assurance at the cost of speedy delivery, but when safety is in play, it really pays to take your time.

To learn more about LMS validation or about the GP Strategies Validation Toolkit, contact us.

—

“General Principles of SoftwareValidation; Final Guidance for Industry and FDA Staff.” FDA. (2002, January 11). Retrieved May 22, 2019, from https://www.fda.gov/media/73141/download.

About the Authors

Chris Olive

“Specialization is for insects.” – Robert Heinlein, Time Enough for Love My parents were both technology focused teachers when I was a kid, and my father was a high school math teacher so in the ‘80s he was the “computer teacher” too. This meant I was lucky enough to have an actual computer in our house, an Apple //e. I clearly remember my father bringing home the sample programming exercises (in Basic!) for me to practice. From there, I moved on to entering the example programs from the back of computer magazines myself and tinkering with them. It was this that made me realize that the flexibility of the computer made it the ultimate tool to solve problems. I also read a lot of science fiction when I was a kid. They were mostly the “classics” of the genre from the 1950s or so. I clearly remember Heinlein’s exhortation and that the main characters were inevitably able to do just about anything they needed to do to survive with their wide-ranging knowledge of multiple subjects. They knew metallurgy, biology, survival skills, hunting, cooking, sewing, and more in a large array of skills. I resolved to do the same. While I have not come close to mastery in all these skills, the notion of the “Renaissance Man” drove my early desire to learn as much as possible about as many things as possible. This led me to the Jesuits and their idea of “care for the whole person” and eventually led me to have my undergraduate education at Loyola College in Baltimore (now called Loyola University Maryland). Combining both technology and the humanities helped put perspective on how computers and other technology really were just tools and that we were solving people problems even though we were using technology to do so. That remains true even now with the clients I help. They are just trying to overcome an obstacle and the method to fix it is immaterial. I happen to use technology as a tool for much of the time but there are human fixes that can be applied to human problems too. At the bottom of every request there is a person trying to do their job. Sometimes they don’t really know what the source of their frustration might be which is why I always start by asking “What is the problem you’re trying to solve?” I figured if it was good enough of a starting place for Richard Feynman it would have to be good enough for me. Once these problems are clearly outlined, they are solvable. I enjoy helping people and solving problems. These things all work together.