This article has been published in the Journal for Clinical Studies

The original article can be found here https://www.jforcs.com/adjudication-in-clinical-trials-a-primer/

 

Introduction

Although the main purpose of most clinical trials is to compare the efficacy and/or safety endpoints that occur in two groups, the endpoint management and adjudication process could often be considered the Cinderella of trial management. In today’s event-driven trials, the choice of endpoints and how the endpoint adjudication process is managed can have a major impact on the success of the trial. Endpoint adjudication is the process by which an independent, blinded expert committee reviews clinical events that occur during the trial. These are assessed — adjudicated— against a set of pre-defined criteria to classify the events. Endpoint adjudication is usually managed in an electronic system which provides efficiency, accountability and documentation of the entire process. When a good quality, systematic and standardized clinical even committee (CEC) process is incorporated into a clinical trial, it can enhance the validity of trial results. In this article we will give a brief overview of the endpoint adjudication process, and the key elements that should be considered when the trial is in the planning stages.

 

What is endpoint adjudication and when is it necessary?

Endpoint adjudication is used when the outcomes of interest are made up of clinical events. When a clinical event occurs in a trial, the on-site investigator will have an opinion as to what type of event the patient has had. However, if you were to ask a different doctor for an opinion about what the event was, a different answer might be obtained. Therefore,if one reliessolely on investigator-assignation of outcome events in a clinical trial, there is a risk of introducing a significant degree of variability into the results. There are multiple reasons for the variability inreporting between physicians: differences in medical training; geographical differences in themanagement of disease and availability of diagnostic tests; differences in health care systems;and of course,bias,as often the investigator is also the treating physician.

In multi-centre and multi-country studies, this effect may be amplified. A centralized and standardized adjudication process reducessuch variability in adjudication outcomes by limiting the number of individuals who are tasked with classifying the potential endpoint events and by ensuring that those individuals have special expertise in the relevant clinical area(s) of interest. This enhances consistency and improves theaccuracy of results. Endpoint adjudication can be used to assess both efficacy outcomes such as myocardial infarction and safety outcomes such as the occurrence of angioedema. Centralized endpoint adjudication should improve the reliability of outcomes data, so it is particularly useful when events of interest are more subjective,e.g. hospitalization for heart failure or when sub-categorization of endpoints is needed, for example a sub-type of myocardial infarction or stroke.

There are increasing regulatory requirements for centralized adjudication. For example,for several years such centralized adjudication has been mandated in trials examining the CV safety of new treatments for type 2 diabetes mellitus.In addition to adjudication of events, the endpoint committee can also be used to help identify events. They may recognize potential new events in source documentation and flag these. In addition,CECs can be used to review SAEs for unreported potential endpoint events and thus ensure that this process is done independently of the sponsor.

What is a Clinical Event Committee(CEC)?

A CEC is a group of independent experts tasked with reviewing and adjudicating all the events of interest in the clinical trial. The CEC perform blinded assessment of clinical event data and decide if the event under scrutiny meets pre-specified clinical event definitions.The event definitions are established in advance of trial commencement and are detailed in a CEC charter document that is available for review by regulatory agencies. The CEC results are used purely for trial analyses and are not fed back to the site investigators.

Who is in a CEC?

A CEC is made up of a variable number of subject matter experts and a CEC chairperson. Members should have clinical expertise in the endpoints of interest and ideally have CEC experience. If a broad range of endpoints must be assessed, then subject matter experts from varying specialties may be required. It is most important that the selected chairperson has CEC experience and leadership ability. The chairman’s knowledge of adjudication becomes particularly important during committee meetings where adjudication disagreements are discussed. Trial details must be specified up front to ensure that members are able to commit for the full duration of the study and have sufficient capacity to meet the workload and timeline requirements. Event volume can often be large, so the number of committee members as well as member availability must be taken into consideration, with some redundancy built in to account for illness and vacation time.The CEC members must be free of conflicts of interest and,in addition, not act as trial investigators.Selection of CEC members can be performed by the sponsor or, as is more often the case, by a CRO (clinical research organization) or a specialist adjudication organization. The sponsor usually must approve the committee after selection.

How do CECs make decisions?

There are several different CEC workflows that can be utilized, however, the one most resistant to bias is parallel adjudication. Each outcome event packet is randomly assigned to a pair of adjudicators. Each adjudicator reviews and adjudicates the package independently. If the answers given by each adjudicator are in agreement, then the event is considered adjudicated or “classified.”If there is discordance between the reviewer outcomes, then the event is sent to a committee meeting for a group decision to be made. Minor disagreements such as event datescan often be dealt with before they reach full committee meetings (e.g. by both reviewers re-examining the event data and ensuring that the discordance is not the result of a simple transcription error).

 

Why you need to plan for centralized adjudication in advance?

The decision to use centralized adjudication must be made at the trial planning phase. A CEC expert should be engaged during trial design. Critical decisions regarding types of endpoints used in the primary and secondary outcomes should ideally be made in advance and after discussion with an expert as errors in endpoint selection are difficult to rectify later. In addition to the definition of events, one of the most crucial aspects of endpoint adjudication is defining the workflow, from gathering the required site documents, to translation and presentation of the event-data to the CEC for adjudication. These workflows should be designed in parallel with the CEC charter and aligned with the overall trial design and collection of trial documentation. The adjudication operation can involve multiple stakeholders(e.g. CRO, CEC, study-sites) and it is important that the various steps and proceduresinvolved in the process are defined early on to ensure workflows are correct.

In addition, real-time, prospective adjudication is now considered the gold standard and allows livemetrics to be utilized in assessing trial progress. Gone are the days of waiting until the trial finishes and adjudicating all the events at the end. When adjudication is performed in real-time it is easier for sites to act on CEC requests for more information. In addition, other trial committees such as the executive committee or the data safety monitoring board rely on timely adjudication to perform their functions and run the trial efficiently. There is no one-size-fits-all approach to the CEC and the CEC workflow as it must be tailored to the trial design.

Key elements to be considered when planning for adjudication

Who will manage the process?

It is important that the CEC remains independent from the sponsor. Ideally, sponsors should engage the services of a specialist adjudication provider or a CRO with adjudication capabilities in the early stages of trial design.

Software systems

Gone are the days when it was practical to use paper-based methods to manage endpoint adjudication.  Just as central adjudication reduces the possibility of bias, a software environment that manages the process from collection to final adjudication reduces procedural discrepancies that can introduce important data variances leading to inaccurate outcomes. There are two elements to consider in the software used to support the endpoint adjudication process.  First, the ability to collect data from investigator sites in a way that reduces errors and increases accuracy.  The second is the ability to enable process controls and oversight that provide for early warnings about process, CRO and adjudication performance.  Whatever the system, it needs to allow CROs and sponsors alike to trace every aspect of process evolution, from collection, de-identification, dossier aggregation and adjudicator voting to CEC management.  When a single integrated system follows the data from cradle to final outcome, it enables the types of metrics sponsors and regulators have been expecting for years: the ability to evaluate site performance, the capacity to run QC on the adjudication process, and the means to compare individual adjudicators to their peer group within and across protocols.  Finally, with advanced system capabilities enabling fully auditable communications between stakeholders (e.g., adjudicators and trial managers), it is now possible to compile the most complete record of every endpoint process, eliminating guesswork and reducing reconciliation processes significantly.  This in turn can have enormous benefits not only to patients but sponsors alike.

CEC charter

The CEC charter describes all the key activities of the endpoint process. It should include standard operating procedures for the entire process including endpoint definitions, data capture overview, committee workflow and decision-making rules, dossier compilation responsibilities, key stakeholder roles and responsibilities, and in the case of the CEC, a description of CEC member requirements. It is a critical document that needs input from all stakeholders involved in the process.

 

Identifying the Key stakeholders

Endpoint workflow can be complex and involve multiple stakeholders: site investigators; site research personnel; CEC coordinators; translators; adjudicators; site monitors; project managers. Each project may have variations on these key roles. Key personnel should be identified early, and roles specified in the charter

 

Event definitions

Careful consideration needs to be given to event definitions. Ideally these need to be as clinically meaningful and also as objective as possible. With cardiovascular endpoints there are standardized definition documents that can be used as outlines for event definitions. If there are no standardized event definitions,the definitions need carefully drafted to ensure they are not too ambiguous. If an event definition is too ambiguous and open to interpretation it can introduce further variability into the results. An example of an ambiguous definition might be an event definition including the phrase “kidney failure.” This is too open to interpretation, and ideally key levels of creatinine or eGFR(estimated glomerular filtration rate) should be used within the definition. Similarly, the components of any definition utilized for a trial should be obtainable for review by the CEC. For example, since cardiac biomarkers (particularly Troponin) are central to the FDA-recommended definition of an acute myocardial infarction event, then for trials where acute myocardial infarction (AMI) is an endpoint of interest, there should be a reasonable expectation that, for most suspected AMI events,relevant biomarker measurements will be available to the adjudicators.

 

Summary

Prospective endpoint adjudication is now considered the gold standard in assessing trial progress.  When done correctly, tried and true workflows and event definitions ensure consistency and reproducibility and thus the reliability of final results in clinical trials. Process matters in endpoint adjudication.  From consistent data collection and de-identification, to performance metrics and oversight, long term adjudication functions benefit greatly from systems that can QC and follow the data from event trigger to final outcome. Such automated databases can produce quality reporting and visible metrics that provide the key constituencies including sponsors and CROs with the data necessary to do course corrections and keep clinical trials on track.