A multi-center example—EQAPOL¶
The External Quality Assurance Program Oversight Laboratory (EQAPOL) is a National Institute of Health (NIH), National Institute of Allergy and Infectious Diseases (NIAID)-Division of AIDS (DAIDS) funded resource located at Duke Human Vaccine Institue’s (DHVI) Immunology Quality Assessment Center (IQAC) to support the development, implementation and oversight of external 8 quality assurance programs that monitor laboratories involved in HIV/AIDS research and vaccine trials around the world.
Methods¶
Results¶
Basic subsets – CD3¶
Comparing automated and manual results for CD3 basic subsets. Center IDs are shown on the x-axis, and relative frequencies expressed as percentages on the y-axis. Brefeldin-A control, CEF and CMV stimulated conditions are in the rows from top to bottom, while samples from subject 1, 2 and 3 are in the columns from left to right. Orange circles represent results from manual analysis, and blue circles represent results from automated analysis.
Basic subsets – CD4¶
Comparing automated and manual results for CD4 basic subsets. Center IDs are shown on the x-axis, and relative frequencies expressed as percentages on the y-axis. Brefeldin-A control, CEF and CMV stimulated conditions are in the rows from top to bottom, while samples from subject 1, 2 and 3 are in the columns from left to right. Orange circles represent results from manual analysis, and blue circles represent results from automated analysis.
Basic subsets – CD4¶
Comparing automated and manual results for CD8 basic subsets. Center IDs are shown on the x-axis, and relative frequencies expressed as percentages on the y-axis. Brefeldin-A control, CEF and CMV stimulated conditions are in the rows from top to bottom, while samples from subject 1, 2 and 3 are in the columns from left to right. Orange circles represent results from manual analysis, and blue circles represent results from automated analysis.
Cytokine subsets – CD4¶
Comparing automated and manual results for CD4 cytokine positive subsets. Center IDs are shown on the x-axis, and relative frequencies expressed as percentages on the y-axis. Brefeldin-A control, CEF and CMV stimulated conditions are in the rows from top to bottom, while samples from subject 1, 2 and 3 are in the columns from left to right. Orange circles represent results from manual analysis, and blue circles represent results from automated analysis.
Cytokine subsets – CD8¶
Comparing automated and manual results for CD8 cytokine positive subsets. Center IDs are shown on the x-axis, and relative frequencies expressed as percentages on the y-axis. Brefeldin-A control, CEF and CMV stimulated conditions are in the rows from top to bottom, while samples from subject 1, 2 and 3 are in the columns from left to right. Orange circles represent results from manual analysis, and blue circles represent results from automated analysis.
Summary¶
The basic subsets were found using the same sequence of gates as manual analyses. The difference lies in the fact that events can be pulled into or excluded from a gate based on the the locations of centroids within the gate. We observed in this analysis that the CD3 gates recovered a slightly smaller percentage of events in the automated analysis as compared to the manual one. This is perhaps due to a tendency of automated approaches to identify populations that share more characteristics across multiple dimensions, which in turn provides a purer population. Given the pattern of identifying fewer events as CD3 and that it is likely that more of these events are in fact lymphocytes under the automated method we expect higher percentages in CD4 and CD8 with automated that we would with manual analyses. Now for CD4 and CD8, given that automated approaches start with a larger denominator, even if both automated and manual approaches identify the same number of cytokine events we expect a smaller percentage for automated than manual analyses. The differences in cytokine percentages are not striking and the background (Brefeldin-1) appears to be smaller in automated approaches when compared to manual ones.
The differences we observed between manual and automated method for the most part can be explained by the imposed selectivity of gating based on clusters rather than individual events. Again gating based on clustered events will either pull in or exclude events based on similarities in several dimensions. Given these results we can safely say that the parameter values for positivity thresholding of beta = 0.8 and theta = 2.0 are robust to different instruments, settings and centers at least in the case of ICS analyses.