Essential controls and reproducibility in flow cytometry

Overview

The construction of complex multicolor panels requires a detailed understanding of the instrumentation that is bigureeing used, including its limitations, and close attention being paid to fluorophore choice, Ag density, and Ab titration. Herein, we highlight the essential controls that should form part of any flow cytometry experiment, and issues relating to reproducibility that should be addressed when establishing good working practices within a research environment.

Introduction

The rapid expansion of flow cytometry instrumentation and automation has driven an explosion in the use of this technology across a range of disciplines and indications. Although the development of “turnkey” instrumentation platforms that do not require the level of technical understanding to operate was required for the earlier platforms has greatly enabled access to this technology, it remains essential that the perceived ease of use does not compromise the appreciation for including the necessary essential controls to ensure robust data and for routinely applying and following necessary procedures that ensure reproducibility. The latter relies both on the practices of the laboratory performing the work and on the correct and comprehensive reporting of the data. However, controls are just one of several important experimental considerations in flow cytometry. Other examples include careful sample preparation, optimization of staining protocols, appropriate handling and storage of reagents and fluorophore products, as well as data acquisition such as doublet discrimination and robust and reproducible approaches for gating on relevant populations. With regards to the latter, several software-based approaches that remove the subjectivity of gating have been and are being developed.

Numerous resources and educational webinars covering all these subjects are available and so the aim of this article is only to highlight the key issues and considerations that should be made and direct the reader to additional, more comprehensive information that can be found elsewhere.

Essential controls

Relevant controls are essential to “Good Flow Cytometry Practice,” without which it would be impossible to robustly generate and analyze data or interpret experimental findings. In a broad context, every experiment should contain three fundamental categories of controls, namely setup (instrument) controls, specificity (gating) controls, and biological controls [[1], [2]].Specifically:

Setup/instrument controls/quality assurance.

Setup/Instrument Controls/Quality Assurance enable the instrument to be correctly set up on installation and at subsequent times thereafter. An essential component of any flow cytometry facility is the regular maintenance and quality control (QC) of the instrumentation. The fundamental purpose of such processes is not only to make sure that the instrument is working ok, but also to ensure longitudinal stability. Broadly speaking, these processes can be separated into three key steps, as defined by Perfetto et al. [[3]]:

- Optimization examines a wide array of critical instrument parameters: the efficiency and performance of optical filters, dichroic mirror reflection and transmission, the timing of lasers (laser delay), laser power, the length of time set to collect instrument signals (e.g., window extension), amplifier linearity, electronic noise, and synchronization of area and peak height signals (e.g., area scaling factor, ASF) [[3]].

- Calibration involves measuring/monitoring the dynamic range and photon efficiency of each photomultiplier tube (PMT) (e.g., detector). This can be achieved using stable hard-dyed fluorescent beads (either manufacturer supplied, or third party such as Rainbow beads). The use of four types of beads has been recommended: Quantum^™ Simply Cellular^® beads (QSCBs), Cyto-Cal beads (or Duke beads), single-peak (1×) Rainbow beads, and unstained compensation beads (COMP beads) [[3]].

- Standardization tracks reproducibility and reliability on a daily basis in order to identify both existing and potential issues. Records of the daily monitoring should be kept electronically in order to better track trends and variances from established acceptable performance ranges and optical backgrounds.

These quality assurance processes ensure the correct and (crucially) reproducible performance of the instrument and provide confidence that any differences/changes in the sample that are detected are due to true biological events not to instrumentation issues. More comprehensive details on a framework that can be used to optimize, calibrate, and standardize flow cytometry for daily use and the periodic requirement for these, as well as advice on troubleshooting any issues have been considered in detail elsewhere [[3], [4]].

Specificity/gating controls.

Specificity/Gating Controls allow specific staining to be distinguished from non-specific staining. Such controls are also used to establish the gates and regions that are used to identify and interrogate cell populations of interest. These controls are crucial for determining positive from negative populations, as well as the intensity of expression of given markers should this be part of the experimental design. Specifically,

- Single-stained controls are used to set the correct compensation value for the fluorochrome panel that is being used in the case of conventional flow cytometry. For spectral flow cytometry, the single-stained sample can be used to determine the contribution of each fluorophore in a mixed sample, using spectral deconvolution (unmixing) algorithms [[5]]. It is essential that the fluorescent “brightness” of the controls is as bright or brighter than the experimental sample, the background fluorescence for the different populations is equivalent, and the fluorochromes used for the control and sample values and the instrument settings used to analyze these populations are the same. It is also essential that the autofluorescence of the populations is the same and that sufficient events are collected. It is not possible to interpret data from multiparameter flow cytometry without including these essential controls [[6]].

- Unstained/Autofluorescence controls enable control for background autofluorescence and can either be unstained cells, or in some instances cells that are known not to express any of the Ags against which Abs are present in the panels being used. It should be appreciated that autofluorescence can change with “biology” and so the appropriate controls should be included for all untreated and treated populations.

- Viability stains are essential when analyzing viable samples, as non-viable (dead) cells are more likely to non-specifically bind Abs and be more highly autofluorescent. The inclusion of viability stains, of which there are very many, therefore enables the more definitive identification of true positive and true negative populations.

- Gating controls are key to correctly identifying the populations of interest. Although these have, for many years, been based on the use of isotype-matched control immunoglobulins, this approach has largely been superseded by the “Fluorescence Minus One (FMO)” approach [[7]]. The FMO control identifies spectral overlap of fluorochromes into the channel of interest and, as the terminology suggests, involves staining the cells of interest with all fluorochrome-conjugated Abs except one. More recently, the need for such controls, apart from when establishing a new panel in order to identify potential problem areas, has been questioned given advances in instrumentation and software [[8]-[10]]. The nature and relative merits of different gating controls, strategies, and software solutions have been discussed in great detail elsewhere [[1], [2], [6]-[12]].

- Non-specific binding negatively impacts the sensitivity of flow cytometry measurements and must therefore be minimized. In many instances, an isoclonic control, which involves staining cells with the fluorochrome-conjugated Ab in the presence of an excess of the identical unlabelled Ab, can reveal whether the conjugate is mediating non-specific binding to the sample. Although some continue to use an isotype control Ab to determine/exclude non-specific binding, the isotype control is not useful in this context [[2]]. Non-specific binding can be reduced using a number of different blocking agents (“Fc blockers”) or by using recombinant Abs that have been engineered not to express the Fc portion, for experiments that are analyzing cells that express Fc receptors.

Biological comparison/experimental controls.

Biological Comparison/Experimental Controls are important as they provide biologically relevant comparisons relating to the difference between unstimulated/treated and stimulated/treated and/or insight into differences between “normal’ and “diseased.” These controls are therefore essential for determining if a treatment or condition induces a true detectable change in the observed biology of the cell population(s) of interest. Depending on the experimental question and design, these samples can also be used for setting relevant gates and regions for conditions. For example, unstimulated samples or healthy donor samples. In some cases, these can function as gating controls for identifying the biological consequences of different settings, treatments, etc. This category of control might also include a reference control that is either a defined cell type and/or stored samples that can be used to determine the longitudinal performance of the assay and/or instrument. If this control “fails,” then appropriate remedial action should be taken before undertaking additional experiments. These controls might also include fully characterized and validated cell lines that are known to be negative or positive for the relevant Ag(s), cells in which expression of the relevant Ag(s) has been knocked out or transfected to express/overexpress the relevant Ag, cells treated with commercial stabilization products, and unstimulated and stimulated populations that could also be generated in bulk and preserved using a range of cell stabilization solutions.

Although tempting to think otherwise, it is essential to appreciate that a well-performed and controlled flow cytometry experiment requires all the above considerations and controls to ensure the correct and robust generation and interpretation of the data that are generated.

Experimental reproducibility

The reproducibility of published research (or rather the lack of it) has become an important and critical issue to the scientific, publishing, and research and development communities, as well as the funders of the work that is being performed. It is essential that research is undertaken in a way that delivers robust and reproducible findings. A landmark study conducted by Glenn Begley and his team in 2012 concluded that most cancer research studies were not reproducible, by alarmingly reporting that 47 of 53 published findings could not be replicated [[13]]. This observation has been followed by a number of additional reports, the summary of which is, in essence, that 77-90% of biomedical experiments are not reproducible [[14]-[17]]. The financial and other implications for biomedical research are clearly staggering considering the financial resources that are allocated to scientific research, and the impact that this lack of reproducibility has on the ability to translate findings into meaningful treatments and impact. Although many factors are likely to be involved (see below), research reagents, particularly Abs in the case of flow cytometry used in a wide variety of applications, have been at the center of the discussion.

Although clearly a complex issue, multiple factors contributing to a lack of reproducibility in research have been proposed [[18]]. These include selective reporting, pressure to publish, low statistical power or poor analysis, insufficient number of replications, insufficient oversight of the work being performed, lack of information on methodology, poor experimental design, lack of availability of original data, fraud, insufficient peer review, problems with efforts to reproduce data and technical expertise required to achieve this, variability of standard reagents, plus simple “bad luck.” Suggestions to improve reproducibility have included a better understanding of statistics, better mentoring/supervision and teaching, more robust experimental design and validation, provision of incentives to improve practice and formal reproduction of data, more external validation, more robust standards for evaluation and reporting by journals, and more oversight and routine checking of laboratory notebooks [[18]]. Inter-laboratory variations are also an issue, with Maecker et al. having reported that individual laboratories typically experience a ~20% CV in the data analysis, whereas data from a central laboratory only shows a variance of ~4% [[19]]. Again, these differences are likely to be driven, at least in part, by a lack, or suboptimal QC systems, an issue that is especially relevant for research-focused laboratories and environments.

With specific reference to flow cytometry, the potential for improving interlaboratory reproducibility of flow cytometry measurements and studies in general by consensual use of methodological approaches has been discussed in detail elsewhere [[20]]. Key questions that should be asked include (not in any order):

- Are my flow cytometry experiments reproducible – could/has another laboratory reproduce(d) my data and findings?

- Have I had problems reproducing experimental data reported in the literature, or data that I have seen presented at meetings?

- How can I ensure that the data I am reporting are “correct” and reproducible?

- Does my choice of Abs and controls, their titrations, and how I combine them as well as my instrumentation and instrument settings align with the needs of the experiment and best practice(s)?

- When was my cytometer last calibrated?

- Have PMT voltage and other relevant instrumentation Quality Control and optimization beads been used?

- Is my compensation strategy robust and appropriate/correct for the experiments being performed?

- Am I using the correct controls for gating on my population of interest?

- Am I using the correct statistical approaches?

- Am I maintaining adequate experimental records – could somebody repeat my experiment exactly based on my experimental records?

- What Quality Control procedures am I using for my instrument, reagents, and experimental assay?

- Do I have (and importantly, am I following) the Standard Operating Procedures (SOPs) for everything that have been established and approved in my laboratory…?

Complementing the extensive “Guidelines for the use of flow cytometry and cell sorting in immunological studies” published by the European Journal of Immunology in 2017 [[21]] and 2019 [[22]], the third edition of which is being presented herein, is A Special Issue of Cytometry Part A entitled “Enhancement of Reproducibility and Rigor” [[23]] and its associated Editorial entitled “Drawing the Bow for Reproducibility” [[24]] have considered the current landscape regarding the enhancement of reproducibility and rigor in scientific research, specifically in studies where cytometry is utilized.

Standard operating procedures/quality management systems

Although the above issues go some way to ensuring robust data, all work needs to be undertaken in an environment that facilities and ensures reproducibility and thereby enables the translation of results into meaningful and impactful applications, for which appropriate QC and quality management systems (QMS) must be in place.

Systems, environments, and working practices to mitigate against issues of reproducibility and improve the rigor, reproducibility, and value/impact of basic and translational biomedical research are “routine” in clinical and commercial settings; however, such Quality Management Systems are less common in preclinical laboratory settings, despite these being particularly challenged with issues of experimental reproducibility. The introduction of Quality Control and Laboratory Quality Management Systems aligned to ISO 9001, Good Laboratory Practice (GLP), Good Clinical Laboratory Practice (GCLP), and Good Manufacturing Practice (GMP) into an idiosyncratic academic laboratory setting is challenging due to the procedures and documentation that are required, a high turnover of staff and prejudice, to name but a few [[25]]. Arguably, a more achievable approach for this setting would be to align work to Guidelines such as the WHO “Quality Practices in Basic Biomedical Research” [[26]] and the “Quality in Research Guideline for Working in Non-Regulated Research” published by the British Research Quality Association RQA [[27]].

The Enhancing Quality in Preclinical Data (EQIPD; originally called European Quality in Preclinical Data) consortium has developed a novel preclinical research quality system that can be applied in both public and private sectors and is free for anyone to use [[28]]. At the core of the EQIPD Quality System is a set of 18 key requirements that can be addressed flexibly, according to user-specific needs, and following a user-defined trajectory. The EQIPD Quality System proposes guidance on quality-related measures, criteria for adequate processes (i.e., performance standards) and exemplifies how such measures can be developed and implemented [[28]]. A practical approach for introducing a QMS into an academic setting has also been described by Hewera and colleagues [[29]], as has a pragmatic and risk-based quality system and associated assessment process to ensure reproducibility and data quality of experimental results while making the best use of the resources [[30]].

Importance of correct reporting of data

Also integral to the delivery of reproducible science is the correct reporting of data, the importance of which cannot be underestimated as it is via this route that others typically base their own experiments and attempt to reproduce the data that has already been reported upon. The reproducibility of flow cytometry data is therefore critically dependent on correct reporting of these data and the provision of the key essential information on experimental design and other parameters that have been used to generate these data. It is for this reason that the flow cytometry community has defined the minimal amount of data that should be provided to understand and be able to reproduce the data via the MIFlow-Cyt [[31]] and MiSET RFC Standards [[32]], more information on which is provided elsewhere in these Guidelines.