Research Vice President
The past few years have been turbulent and disruptive. The life science industry has pivoted on its feet; it has pulled out all the stops to keep clinical trials going, while prioritizing patient safety, and has indeed been successful in bringing vaccines to the market in record time. Accomplishing this remarkable feat meant reinventing business models, including the adoption of decentralized clinical trials (DCTs), and adopting new processes and technologies, often even while a trial was in progress. This has only added to complexity and has scaled risk. In addition, as per the Tufts Center for the Study of Drug Development (CSDD), the average number of data points per trial has increased from less than a million in 2010 to about 3.5 million data points in 2020 (this number could be significantly higher in oncology clinical trials). Tufts CSDD has also noted that there has been a 113% increase in the number of substantial amendments and a 105% increase in the subject drop-out rate in the past decade in the case of pivotal Phase III trials. Thus, implementing strategies and technology to address this risk in a lean, dependable, and a consistent manner is critically important. As per a survey published by the Association of Clinical Research Organizations (ACRO) in Therapeutic Innovation and Regulatory Science (TIRS), the peer-reviewed journal of the Drug Information Association (DIA) in March 2022, 77% of the 5,987 trials that were conducted in 2020 implemented at least one risk-based monitoring (RBM)/risk-based quality management (RBQM) component, an increase from 47% for studies ongoing at the end of 2019. Among the individual RBM/RBQM components, the highest adoption was seen in the case of cross-functional risk assessments. The 2021 survey showed the percentage increasing from 77% to 88% across 4,889 studies, with a fairly even distribution of around 30% primarily Phases I, II, and III. Interestingly, almost 60% of the studies where RBQM was implemented were small studies, with a sample size of less than 300. Since the sample represented studies outsourced to contract research organizations (CROs), this could be indicative of pharma still testing the waters on smaller studies. Companies also want to run parallel studies to validate the returns from implementing an RBQM strategy, and it is more pragmatic to do so on a smaller study.
In 2013, the U.S. Food and Drug Administration (FDA) released guidance on a risk-based approach to monitoring, and TransCelerate BioPharma (a nonprofit organization driving collaboration across the biopharmaceutical research and development community to develop solutions to drive the efficient, effective, and high-quality delivery of new medicines) published a position paper on RBM. In addition, in 2013, the European Medicines Agency (EMA) released a reflection paper on RBQM. As per the FDA, RBM directs sponsor oversight activities on preventing or mitigating important and likely risks to data quality and to processes critical to human subject protection and trial integrity by appropriate use of centralized monitoring and reliance on technological advances.
While RBM requires sponsors to use technology and real-time data to proactively monitor risk, RBQM applies the principles of quality by design (QbD), as well as RBM to all elements of a study, from planning right through to execution, providing holistic clinical trial oversight and addressing systemic risk. RBQM is an ICH- and FDA-advocated approach to managing risk for the entire clinical trial life cycle. RBQM has driven a shift from a reactive to a predictive and a proactive approach toward managing risk, from a piecemeal and a tactical approach to a holistic and a systemic approach toward managing risk, with the core goal of integrating a QbD strategy into the very DNA of the organization, into its processes, and into the design of the study protocol, so that driving quality does not come as an afterthought but is embedded into every process, driving a first-time right approach.
The life science industry has effectively adopted Lean Six Sigma (L6S) methodologies from the manufacturing industry, assessing impact, probability, and detectability to arrive at a risk probability number (RPN) for scoring risk. TransCelerate developed the original risk assessment and categorization tool (RACT) to define different categories of risk and key risk indicators (KRIs) within each category. The expertise lies in not only identifying a generic set of KRIs but also recognizing which ones may be specific to a certain therapeutic area and to a study. One also needs to develop composite KRIs, as risk is an interplay of multiple components. While KRIs are usually used at a site level, parameters apply at a study level or higher. Defining accurate thresholds for KRIs and quality tolerance limits (QTLs) for parameters is not an easy task. While KRIs may change during the course of the study, QTLs should typically not change. The impact of modifying any parameter or QTL can have a multiplicative effect on quality at a systemic level. The upcoming ICH E6 R3 guidance instead of focusing on QTLs emphasizes the importance of setting acceptable ranges and identifying whether the issue is a systemic one or not, and that the rationale for the choice of the monitoring strategy (centralized/remote/onsite) should be documented in the monitoring plan.
Implementing an RBM-RBQM strategy does not come without its challenges. The very first challenge lies in driving adoption. Site monitoring accounts for about 30% of the trial budget. RBM drives a shift toward centralized monitoring, where focused site visits are triggered when certain risk thresholds are crossed, not only reducing risk but also driving cost efficiencies. This has created concerns among site monitors regarding their employment. In addition, site monitoring has been a key source of revenue for contract research organizations, creating a pushback toward adopting this model.
When RBM was initially rolled out, a fair percentage of the industry rubbished it, adamantly declaring that they had always managed risk and viewed RBM with some cynicism as a new-fangled methodology. While the industry certainly has been focused on addressing risk even in the past, a structured framework, complemented by powerful technology to derive rich insights from big data that are available today, has certainly transformed the way the industry is now addressing risk. The importance of this model is being increasingly recognized and endorsed by regulators and by the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use (ICH) as well. Along with a reduction in site visits, one also sees a reduction in the source data verification (SDV) and, instead, a sharper focus on source data review (SDR). While some organizations apply a reduced and a targeted SDV model, a few have even completely eliminated it as they do not see a significant improvement in quality by performing SDV. It is important to remember that RBM is not just reduced SDV, through a risk-based reduction in SDV may be included in the RBM implementation strategy. In addition, one needs to keep in mind that regulations regarding remote site monitoring and remote SDV vary significantly across geographies, though many of the countries relaxed their regulations during the pandemic. As many monitoring visits may be conducted remotely or centrally, and as data from multiple sources gets integrated, it is essential to ensure that the security and confidentiality of records is maintained.
A key conundrum that has not yet been entirely resolved is where should risk management and centralized monitoring (CM) sit within an organization. Some organizations include it in data management (DM), some in clinical operations, and others build it out as a completely new capability within the organization. Risk monitors/central monitors need to be trained to look at risk holistically, and this means having a deep understanding of the protocol and why certain thresholds and mitigation actions have been outlined to be able to truly take ownership of managing study risk. Concerns also prevail about issues that may slip through the cracks unnoticed if the KRIs and thresholds have not been well defined, impacting regulatory approvals. Fears exist about reducing the level of engagement with the sites as in-person visits are reduced. Sites also perceive this model as increasing their workload, thus increasing their resourcing requirements and their costs. For those organizations that do not have a lot of historical data, defining thresholds or QTLs can be challenging as they do not have data that they can depend on. Technology platforms come at a significant cost, and as is the case with investment in any technology, it does take time to realize returns. One cannot expect to see an instant return on investment (ROI). Organizations also tend to rush into adopting new models without putting the right structure in place, defining standard operating procedures (SOPs), and implementing training, which can actually increase risk, rather than decreasing it. Choosing the right study, including the right phase and therapeutic area, to implement RBM is essential to demonstrate success. Very often, companies that are new to the process pick an ongoing study and try and quickly implement this methodology, but that is not going to yield the desired results. Thus, change management, training, and robust technology are critical elements when trying to implement these strategies. This is not applicable only with respect to sponsors and CROs but to sites as well. Sites can become both concerned and confused if they find that their site monitor has suddenly stopped visiting them regularly and may worry why their performance is being scrutinized with increased rigor. Hence communication is key to drive the success of RBM implementation. In addition, business leaders are always under pressure to demonstrate ROI, but they need to define key performance indicators (KPIs) up front, and the technology should enable those data points to be captured automatically. That often is not the case and not many have predefined KPIs. TransCelerate BioPharma has released a metrics report covering the critical KPIs that should be measured to determine the success of implementation of an RBM strategy. These include the average number of major audit/critical findings, the percentage of unreported confirmed serious adverse events (SAEs) compared with the total number of SAEs, the number of significant protocol deviations, the average interval between onsite monitoring visits, and the median number of days for closing an issue, from patient visit to electronic case report from (eCRF) data entry, and for closing a query, as well as cost savings. Participating companies reported 62% gains with respect to reduction in audit findings and on an average over 50% gains across the rest of the parameters by implementing RBM. The ICH E6 R2 guideline advises sponsors to take a systematic, prioritized, risk-based approach to monitoring clinical trials. It requires organizations to implement the steps of risk identification (identifying critical data and processes), risk evaluation, risk control, risk communication, risk review, and risk reporting. There are some critical takeaways here. All risks cannot be eliminated, and critical risks with significant impact need to be prioritized, and efforts need to be invested in addressing those risks. This requires the right stakeholders to be at the table to develop an integrated quality risk management plan (IQRMP), which, as the name implies, requires a cross-functional team of experts. Even more important is identifying which of these risks are systemic and will have a much wider impact on the study or even on the organization. In addition, ICH E6 R2 requires sponsors to describe the quality management approach implemented in the trial and summarize important deviations from the predefined QTLs and remedial actions taken in the clinical study report, covering the entire spectrum of managing risk, from planning to reporting.
There is a need to manage risk across the quality management system (QMS), moving down the pyramid from an organizational level, looking across critical processes and critical data points, managing vendor oversight as well, and drilling down further to a portfolio, asset, study, and at a site level. In fact, the latest update to the FDA’s Bioresearch Monitoring Program (BIMO) guide gives more importance to QMS in line with the fact that good clinical practice (GCP) inspections are now placing increased emphasis on safety oversight, outsourced services, and the selection and monitoring of clinical investigators. A section of the guideline focusing on safety oversight describes how FDA investigators will assess whether the sponsor has established and followed a risk management plan.
Despite the accelerated adoption of RBQM, fueled by the by the far-reaching ramifications of the COVID-19 pandemic, there are still only a few niche players that have true RBQM expertise and technology solutions. The choice of a partner really depends on where the pharma/biotech company is in its RBQM journey and the pace at which it wants to grow this model: whether it is driving enterprise-wide implementation or just wants to run a few studies within a portfolio or whether it is still piloting its first study. The most complex of the three is enterprise-wide implementation and calls for partnering with a company with expertise, not only in supporting the implementation of RBQM but also in organizational change management and in creating business value cases to demonstrate return on investment to generate the necessary executive buy-in to drive this model. A partner that can help outline standard operating processes, define best practices, define key performance indicators and key risk indicators, drive organizational training and site training, and build awareness, engagement, and adoption of this new model would be a valued partner. The depth of expertise in these kinds of engagements needs to be carefully assessed. The choice of a partner also depends on whether the pharma/biotech company has its own platform or is looking to choose a vendor to provide one, and whether it is looking for data and technology integration support. The choice also depends on whether the sponsor is looking to implement central statistical monitoring (CSM) and is seeking a partner with relevant expertise and with that methodology embedded into the technology. Unlike KRIs, which are indicative of the potential for risk associated with specific predefined data points and thresholds, CSM is based on all clinical data (all variables are deemed equally indicative of quality and are given equal importance when assessing risk), is unsupervised, and is based on a large number of diverse statistical tests. The risk associated with the KRI approach increases when there is a lack of domain expertise in identifying KRIs and when there is inadequate historical data available to determine thresholds accurately. If KRIs and thresholds are not accurately determined, this could result in a risk being missed. On the other hand, CSM may lack specificity and highlight insignificant data issues. Experience in guiding technology implementation strategy, selecting vendors, and providing vendor oversight serves as a differentiator.
As organizations attempt to drive the adoption of a RBQM strategy and build robust frameworks to manage risk, they are seeking partners that can help lead the way. In IDC’s view of the RBQM technology and solutions ecosystem, key attributes that life science companies are looking for in their preferred RBQM solution providers include:
This section briefly explains IDC’s key observations resulting in a vendor’s position in the IDC MarketScape. While every vendor is evaluated against each of the criteria outlined in the Appendix, the description here provides a summary of each vendor’s strengths and challenges.
After a close evaluation of ThoughtSphere’s offerings and capabilities, IDC has positioned the company in the Major Players category in the 2022 IDC MarketScape for worldwide life science R&D risk-based monitoring solutions.
Founded in 2015, ThoughtSphere has around 30 employees, with 75% focused on technology and the rest on domain. A majority (60%) of employees come from the industry and have about six years of life science industry experience. It is headquartered in San Jose, California, with offices in APAC and the United States. It is privately held, with revenue below $100 million. It received $5 million funding in 2021. Its risk-based quality management offering represents 60% of its revenue, and it sees 50% growth in RBQM revenue and in RBQM customers over the next 3 years. It invested 70% of its revenue in R&D in 2021 and expects the investment to grow by 50–70% per year over the next 3 years. It serves 10 RBQM customers: 40% pharma and 60% contract research organizations. Of its RBQM customers, 60% are based in the United States, 25% are in Europe, and the remaining 15% are in APAC. Its RBQM customers have revenue ranging from $8 billion to below $50 million. It offers RBQM technology solutions to 75% of its customers and both technology solutions and consulting services to 25% of its customers. It has supported over 150 clinical trials and has helped implement RBQM solutions in 75% of them. Further:
ThoughtSphere is a clinical data and analytics company that provides an end-to-end cloud-based platform built using data lake architecture. The platform’s stacked solution is built around ClinHUB, its patented data ingestion/standardization engine that harmonizes both clinical and operational data to support RBQM, central monitoring, medical, safety, and data management tasks across study phases and data sources. The platform leverages AI/ML to auto-map raw source data to a chosen target model, reducing study implementation time from weeks to days. The ThoughtSphere platform is hosted in Google Cloud (though it can be hosted in other cloud platforms) and is currently being used by 10 customers for over 120 studies. The platform is developed using modern web technologies, which provides intuitive user interfaces to automate risk review and auditable and traceable cross-functional data review workflows that are fully auditable and traceable. Its ClinACT solution supports risk planning, execution, and issue management. The platform allows risk assessment tools, such as TransCelerate’s risk assessment and categorization tool, Metrics Champion Consortium’s risk assessment tool, and customer-specific risk assessments to be uploaded in its platform. Risk assessments can also be directly created, versioned, and approved within the platform. Its Smart Risks feature leverages ML to pre-identify risks. By digitizing the risk assessment, risk mitigation strategies take the form of automated alerts that are assigned to the appropriate risk review workflow to streamline the risk management process. Its embedded business intelligence (BI) tool generates out-of-the-box (OOTB) and custom analytics dashboards. It offers 68 KRIs (40 are generic) across 7 risk categories and offers KRIs specific to 7 therapeutic areas. It also offers 6 study-level quality tolerance limits. It employs predictive analytics to project the probability and timing of a QTL breach. Its DMSphere solution allows users to automate cross-domain and cross-source data checks and reconciliations. The platform’s customizable patient profiles support holistic subject-level medical and clinical reviews. By facilitating cross-functional review activities and automating many manual tasks, ThoughtSphere enables small/midsize biotechs to bring DM and biostatistics work back in-house. ThoughtSphere has implemented RBQM across large organizations and has the expertise to drive alignment between people, processes, and platform. Its largest implementation engagement was with a top 5 contract research organization looking to integrate cross-functional reviews and optimize RBQM methodologies. Core organizational systems had to be connected to facilitate single sign-on. The Clinical Trial Management System (CTMS) and several clinical data capture systems had to be integrated with the ThoughtSphere platform. ThoughtSphere developed efficient processes and workflows to implement RBQM. The CRO had a library of over 50 KRIs that had to be programmed and validated at the study level. ThoughtSphere configured the specifications for these 50 KRIs in its BI tool at the enterprise level driving efficiencies for the organization. Other custom features implemented included a library of QTLs, a set of default risks and mitigations to support RBQM consistency across studies, a clean patient tracker, and a sponsor oversight solution. ThoughtSphere was deeply involved in the setup of the initial studies to ensure a smooth transition to the platform. The consulting engagement required driving alignment across many stakeholders and executive leaders. “ThoughtSphere underpins our clinical delivery model. They developed OOTB QTLs, and 13 KRIs. They are very collaborative in building out custom requirements. They are always looking to improve their product. They are always a joy to work with — on time, on budget. It’s been an absolute pleasure working with them. This year we will do 25–30 studies with them,” said the associate director, product management of a top global CRO.
ThoughtSphere is seen as a pure product development company and should hire strong clinical or biostats experts to support the product and build its consulting expertise in RBQM. It should use ML to create more predictive analytics; as currently, it is using ML primarily for data transformation. It is seen as a small, regional player and needs to build its global footprint. It should develop an enablement program to drive user adoption of its platform and further refine and enhance the user interface to be more intuitive.
Consider ThoughtSphere when seeking an agile, fit-for-purpose, cost-effective, and cloud-native stacked RBQM and DM platform that can do large-scale data ingestions very efficiently, digitize and automate data review processes, and provide expertise in technology implementation including the build out of quality metrics (such as KRIs/QTLs).
IDC frequently has unique visibility into vendor selection processes within life science companies through clients and contacts in the industry. For a vendor to be considered for inclusion in this study, the vendor’s services must have been significantly evaluated for the potential to engage clients within the target IDC MarketScape space. Further, research and due diligence were then conducted to narrow the list of vendors to only those that IDC views as legitimate contenders for future deals within the life science space, based on an assessment of the vendors capability in providing technology solutions and consulting services to support the implementation of a risk-based quality management strategy.
For the purposes of this analysis, IDC divided potential key measures for success into two primary categories: capabilities and strategies.
Positioning on the y-axis reflects the vendor’s current capabilities and menu of services and how well aligned the vendor is to customer needs. The capabilities category focuses on the capabilities of the company and product today, here and now. Under this category, IDC analysts will look at how well a vendor is building/delivering capabilities that enable it to execute its chosen strategy in the market.
Positioning on the x-axis, or strategies axis, indicates how well the vendor’s future strategy aligns with what customers will require in three to five years. The strategies category focuses on high-level decisions and underlying assumptions about offerings, customer segments, and business and go-to-market plans for the next three to five years.
The size of the individual vendor markers in the IDC MarketScape represents the total revenue of each individual vendor being assessed.
IDC MarketScape criteria selection, weightings, and vendor scores represent well-researched IDC judgment about the market and specific vendors. IDC analysts tailor the range of standard characteristics by which vendors are measured through structured discussions, surveys, and interviews with market leaders, participants, and end users. Market weightings are based on user interviews, buyer surveys, and the input of IDC experts in each market. IDC analysts base individual vendor scores, and ultimately vendor positions on the IDC MarketScape, on detailed surveys and interviews with the vendors, publicly available information, and end-user experiences in an effort to provide an accurate and consistent assessment of each vendor’s characteristics, behavior, and capability.
For the purposes of this study, risk-based monitoring solutions are defined broadly as:
Despite a slow uptick, the adoption of RBQM models has increased, driven by the turbulence that the world is experiencing and the critical need to manage risk effectively. While the FDA has focused more on RBM and the EMA more on RBQM, organizations will implement RBQM strategies to address far-reaching systemic risks and will leverage RBM to assess site-level operational risks. Technology and strategy will go hand in hand to scale this capability. This will need to be complemented by a cultural change and openness to adoption of these solutions, along with careful scrutiny on the return on investment.
Efforts at the forefront include: