John Mandy participated in thousands of audits during his 26-year tenure as a Quality Leader at Pfizer. Early on, Pfizer had thousands of GMP suppliers and CMOs located all over the world. In any given year, Mandy’s audit teams were executing over a thousand internal and external audits and assessments. But even he was surprised when he was helping a site prepare for a significant re-inspection and had a local QA lead proudly show him a validation qualification for a new paper shredder. That was just one of many times that he observed teams not taking a risk-based approach to systems validation.
Members of the Quality Leaders Forum (QLF) for Life Sciences met in Durham, North Carolina last week to discuss risk-based systems validation. John Mandy and Tim Reinhardt of RMC, both experts in compliance and quality management systems, moderated roundtable discussions, starting by deconstructing the concept of system validation.
Mandy and Reinhardt stressed the need for Quality Leaders to ask themselves: Is validation useful? Is it helping your business or is it just a scare tactic? If the latter, how do you know where to draw the line? They noted that many of their consulting clients believe that there is a higher risk when validating software in general, including electronic quality management systems, but feel this assumption is unwarranted.
As evidence, the QLF reviewed high level data analysis of all published FDA findings since 2008, which indicate that the incidence of FDA findings related to systems validation is exceedingly low. In fact out of ~92,000 findings, just 427 findings related to software validation (that’s just 0.5%!). And most of those were companies that just didn’t execute a validation at all, didn’t follow their own procedures or have defined procedures to begin with.
And yet, regulated companies continue to dedicate significant resources to extensive validation processes of every system they use- up to and including spreadsheets. Michelle Souder, Senior Manager of Quality Systems at Marken (a UPS company), believes that, “Language is very important- particularly in IT. And the terms validation and qualification are often used interchangeably when they need to be differentiated. Given the high speed at which things change in the tech world, it is challenging to find subject matter experts in these areas that can establish best practices AND speak the language of IT.”
The recent rise of Cloud based systems has compounded the problem, as many of the people who are responsible for executing system validations lack expertise with these types of systems. And the SOPs that govern the process often rely on the old/traditional approach to validation that isn’t always relevant to newer, validated SaaS platforms, like ZenQMS for example.
So how does a company quantify the risk? And what is the risk really based on? Below are the key takeaways from the QLF meeting in North Carolina that try to get at this question:
Follow Your SOPs
The biggest issue, specific to validation, is that companies don’t follow their own SOPs for validation. Some of the largest categories of 483 findings in the FDA database revolve around not following procedures or having properly defined procedures in the first place.Many in the audience pointed to situations where someone inherits or is stuck with poorly written SOPs for validation. The good news is that this can be addressed! Two key points:
1. Make sure your SOPs, like those for qualifying new systems, are written and approved by people who will own the process. And don't be shy about asking advice from experts -- this is a great example of where a few hours of technical review by a consultant can pay dividends.
2. SOPs should allow proper instruction for excluding items that are not relevant to a qualification. For instance, it shouldn’t be a deviation if you don’t produce a full IQ/OQ/PQ on an already validated SaaS system. Per GAMP 5 Category 4, all you need is a documented risk assessment, documented requirements, and appropriate User Acceptance Testing.
Take a risk-based approach that goes beyond Quality and Compliance risks
Regulators want you take a risk-based approach. However, John’s story of the manufacturing plant with a full and detailed IQ/OQ/PQ for an office paper shredder makes it evident that despite best intentions, we should always stop and think about risk in a practical way.
QA needs to think more holistically about risk for validation to account for Business stage and Capital risks, as well as quality, compliance and manufacturing risks. Logically this means risk assessments should allow all key stakeholders to weigh in. As the scope and possible impact of the underlying item expands, so should the stakeholder cohort. A good example that was raised was the early stage biopharma company with very limited capital— the focus here should be on attainment of the milestone, rather than validating all systems to the same standards as commercial stage companies. The risk is in protocol definition and execution, which largely relies on CROs. The point here is that you can always update the qualifications of your internal systems as your company matures, based on the evolving set of risks your company faces.
Establish and Defend your Quality Identity
Your quality system is a living, breathing part of your company culture—don’t let it get pushed around to suit a 3rd party’s expectations without good reason and deliberation. This is most important for GxP service providers who get audited by clients and regulators of varying experience/expectations. Making changes haphazardly will only lead to a disjointed set of processes that won’t hold together and will increase the likelihood of mistakes. Run findings through your Change Control process and contemplate impact and change holistically. And remember, it’s ok to say ‘no’ to non-regulatory inspectors with thoughtful justification and to ask for findings to be grounded directly to the relevant regulatory citations.
Make these Validation Documents Easy to Understand and Explain
This seems obvious, but it never really hits home until you take a job at a new company and have to defend existing documentation in the first audit you host-- especially if you engage a 3rd party service to do this for you. If you can’t explain and defend a how a critical system was validated/qualified, you leave your company open to findings (from a good auditor) or having a non-expert auditor giving you opinions.
Join us for future events - apply to become a member of the Life Sciences Quality Leaders Forum today.