Risk-based monitoring: Let's start with the data

[#1]

To begin our discussions on risk-based monitoring (RBM), we first need to start with the data. The data include various metrics to assess site performance, and may include several key measures of safety such as deaths and adverse events (to assess safety concerns or under-reporting). But where to start? These data can come from a variety of sources including, but not limited to, the study database; randomization systems; SAS programs used to assess whether participants, in fact, met study eligibility criteria; or other documents and spreadsheets maintained by monitors that visit the clinical sites. A further wrinkle is that these data may not all be within easy reach of the sponsor, so careful planning with various vendors or contract research organizations (CROs) is needed to make timely use of any collected information.

An important part of the RBM-potential of JMP Clinical is the use of your CDISC-formatted study database. Why CDISC? First, relying on these data standards allow us to deliver you a product with powerful out-of-the-box features. Second, it allows you to perform a more thorough and informative analysis for RBM than relying on a spreadsheet of site metrics alone. Here are a few simple examples:

It’s straightforward to calculate the number of randomized subjects per study site to define risk indicators as “Response per Randomized Subject”, such as ”# Queries per Randomized Subject”. When we make use of the study database, it is easy to calculate the duration of each subject’s participation in the clinical trial. It’s possible to then calculate Risk Indicators normalizing by total Patient-Weeks at each site, such as ”# Queries per Patient-Week”. This can identify important differences between two sites with the same number of subjects and queries, but with varying “exposures” (say, for example, between four subjects with a year average follow-up compared to four subjects with three months average follow-up. The latter site’s performance is worse in this case).

CDISC captures in numerous places whether a subject died: in demography (DM), adverse events (AE) and possibly disposition (DS). As part of the monitoring process, edit checks can assess the similarity of these various fields when calculating the number of deaths, an important safety measure to compare across sites. Other edit checks are possible to help assess quality as part of the monitoring process.

When problem sites, countries or even monitors are identified, we can delve deeper into the participants at those sites to assess safety more thoroughly with narratives and profiles. Alternatively, we can view notes or compare performance before and after site intervention using the snapshot comparison tools.

However, not all data to assess site performance are contained within the study database. This can include the total number of queries, overdue queries, site responsiveness to query resolution or CRF entry, the number of missing CRF pages or deviations identified onsite or computed using other SAS programs. JMP Clinical lets you add these data to your RBM analysis using the Update Study Risk Data Set Analytical Process (AP). By default, the AP allows you to enter the variables presented in Figure 1, which includes the aforementioned variables as well as the date at which a site became active for patient enrollment, the primary monitor for that site, and geographical information that can be used to geocode the latitude and longitude of the clinical trial site. If other important variables are needed, it is possible to add them to the current and future studies.

Figure 1. Site Metrics

So how are these data entered?  Data entry is straightforward for JMP tables, or data can be copied from other spreadsheets or imported using various JMP functions. A single individual can be responsible for this task or, taking advantage of JMP Clinical’s ability to integrate with a SAS server, the workload can be shared with several members of the clinical team.

Alternatively, a user can Export Tables to create a file for each monitor containing only the sites for which he or she is responsible. The monitors can fill out their tables and return them on a weekly basis. The Import Tables function updates the study risk data set by pointing to a folder containing one or more completed tables. All tables within the selected folder are added to the study risk data set automatically. In this manner, sponsors relying heavily on CROs can easily get the information needed to assess site performance.

Figure 2. Clinical Site Locations

So why the geographical information? First, it allows you to see the location of your clinical trial sites (Figure 2). More importantly, it gives you the ability to view risk geographically at the site- and country-level. Viewing risk in this manner can provide additional insight into why certain problems are occurring (sites may belong to a specific monitor, differing regulatory or training standards can contribute to problems, or some adverse events may have certain geographical considerations). While the example above geocodes all sites using city and state or province information, clinical sites within the US can be geocoded using five-digit ZIP codes only.

Next time we’ll discuss the second component needed for analysis: the thresholds that define risk.