Institute for International Research

Case Study 10

Statistical methods for the determination cut-points often make the assumption that the data are “Normally” distributed. Normality implies that the data follow the Gaussian or bell-like shape with some central tendency and spread that is uniform on both sides of the central point. The data from immunogenicity assays are often not Gaussian shaped and tend to follow some other statistical distribution leading us to use additional statistical methods for the determination of the cut point. This session will discuss how to determine if data are “Normal” and methods to determine cut-points if data are non-Normal. Included in this discussion will be simulations and examples from Human Genome Sciences.

Martin Kane, MS, CRE, Senior Manager, Process Statistics, HUMAN GENOME SCIENCES

2:15

Case Study 11

The standard paradigm (Mire-Sluis et-al 2004) for detecting antitherapeutic antibodies (ATA) in human subjects is a screening assay followed by a confirmatory assay. A sample that tests positive in the screening assay is subsequently tested in the confirmatory assay. In order to decrease the overall false positive rate, a sample is declared positive if it tests positive in both assays. The standard paradigm uses a risk based approach to set the cut-point for the screening assay; the cut-point is set such that 5% of samples from any large collection of untreated patients are expected to be declared positive. The standard paradigm does not propose a method for declaring a sample positive in the confirmatory assay. In this paper, we propose that the same risk based method be used to set the cut-point for the confirmatory assay and introduce statistically based criteria for assay optimization.We argue that when the assays are well optimized the assays are conditionally independent and that the overall untreated positive rate (UTPR) is the product of the UTPRs, specifically if the UTPR is set to 5% in both assays the overall UTPR is 0.25%. This result holds even if the same untreated patient samples are used to determine the assay cut-points.We illustrate these methods for the detection of ATA against a monoclonal antibody therapeutic.

Dan Coleman, PhD, Nonclinical Statistics, GENENTECH

Case Study 12

The calculation of screening cut-points for immunogenicity assays has been hotly debated in recent years and there is no consensus on the most appropriate method. Proposed methods from an upcoming immunogenicity white paper by an AAPS focus group include using the mean and standard deviation, median and median absolute deviation, 95th percentile, or some advanced statistical methods called robust estimators. Using data from assays developed by Amylin Pharmaceuticals, we have done a comprehensive study to evaluate the performance of the methods proposed in the white paper as well as methods proposed at other meetings. Using a statistical tool for resampling called the bootstrap, we repeatedly selected drug-naïve subjects from each of ten panels (eight immunogenicity assays) and compared the performance of each of these methods. The results of our study will be presented. In brief, when there are none or few outliers in the panel of drug-naïve subjects, all methods result in similar cut-points. However, when there are a greater number of outliers, the 95th percentile results in the highest cut-point while the method using the median and median absolute deviation has the lowest cut-point. The robust estimators using the mean and standard deviation are between these extremes. Methods based on using a statistical test for normality are less consistent, and can be either lower or higher than the other methods.We also show the impact of outliers and why outliers from the panel (both analytical and biological) should be excluded for a risk-based assessment.We conclude with a comparison of the advantages and disadvantages of each method as well as our recommendations.

Todd Coffey, PhD, Senior Nonclinical Statistician, AMYLIN PHARMACEUTICALS

Panel Discussion

• Development and validation of anti-drug antibody assays
  
• Drug tolerant assay options and considerations
  
• Statistical evaluation of data
  
• Screening compounds by comparing their relative Immunogenicity in mice
  
• Case examples of whether or not preclinical immunogenicity data has paralleled or added to interpretation of clinical data.
  
• What are the true realistic cut-points for various species?
  
• Determine confirmatory and titration assay cut-points
  
• Interference by soluble drug target-more common that one might think
  
• Use of a Design of Experiment approach to develop immunogenicity assays
  
• Determination of positive samples after competition assay – statistical evaluation or unequivocal depletion?

Workshop Leaders and Panel Members:
Dan Coleman, PhD, Nonclinical Statistics, GENENTECH

Martin Kane, MS, CRE, Senior Manager, Process Statistics, HUMAN GENOME SCIENCES

Todd Coffey, PhD, Senior Nonclinical Statistician, AMYLIN PHARMACEUTICALS