Risk-based monitoring: Defining thresholds for risk
Now that we have put together a data set containing the important metrics to monitor safety and site performance, we need to define the thresholds that constitute elevated risk. Unfortunately, there is no one-size-fits all solution to this problem. The study population (e.g., pediatric, elderly or at particular risk of safety issues), the experience of clinical sites involved with the trial, the sponsor’s familiarity with these sites, the complexity of the protocol, or the sponsor’s experience with the drug under investigation (is this trial first in human?) all play a role in determining the level of risk to be tolerated.
Risk is usually elevated when there is an excess of undesirable outcomes. For example, a site that has excessive queries beyond the average performance of participating centers, takes too long to respond to queries or enter case report forms (CRFs), or has excessive protocol deviations is an obvious concern. However, there are situations where “underperformance” can be as problematic – as is the case for under-reporting adverse events, or difficulties in enrolling patients.
However we choose to define risk, it should be possible to specify:
Direction – does excess or scarcity elevate risk? Both?
Percentage – how far away from the overall average response as a percentage would be considered excess risk? Does the mere presence of some factors (e.g., unsigned informed consent) elevate risk?
Magnitude – risk may be elevated based on percentages, but are the number of errors (magnitude) sufficient to justify action?
Contribution – the level to which each individual risk indicator contributes (or not) to an overall measure of risk. Are some indicators more important than others?
Fortunately, these risks can be defined in a straightforward manner. The Define Study Weight Data Set (Figure 1) report from JMP Clinical allows the user to define and tailor risk weight data sets based on the risk management plan of the current clinical trial using the traffic-light system suggested by TransCelerate BioPharma. These risk weight data sets can be applied to future studies and can incorporate user-defined variables for analysis. Further, several risk weight data sets can be defined to evaluate the sensitivity of the Risk-Based Monitoring analysis to varying assumptions and risk thresholds. When integrating with a SAS server, a single individual can be responsible for defining risk thresholds, making them available to the entire clinical team through a drop-down menu in the analysis dialog. Also, several overall indicators are defined, allowing for the straightforward combination of related factors.
Figure 1. Risk Weight Data Set
With the data available and risk thresholds defined, we can now perform our Risk-Based Monitoring analysis (Figure 2). Color easily identifies the variables for which risk thresholds were defined, as well as the particular site-indicator combinations with elevated risk.