When is an Adverse Impact Analysis really significant?

When is an Adverse Impact Analysis really significant?
by Desiree Throckmorton, Consultant, Biddle Consulting Group

Before the advent of personal computers it was necessary for EEO enforcement agencies to enlist very simple mathematical techniques for identifying potential discrimination in selection processes. As a result, when the Federal Uniform Guidelines for Employee Selection Procedures (UGESP) were released in 1978 they discussed a mathematical process known as the 80% test (aka Impact Ratio Analysis or IRA), whereby the selection rates of the disadvantaged groups were compared to the selection rate of the advantaged group. If the disadvantaged group’s rate was less than 80% of the rate of the advantaged group it was said there was an “adverse impact” against the protected group. Many employers still rely on this test, because it is simple to compute and understand. However, to determine true problem areas we suggest utilizing more sophisticated techniques.
A more advanced, yet limited approach to comparing selection rates is the chi-square (x2) statistic. The strength of the x2 lies primarily in two areas: 1) it’s simplicity to calculate (from a computational standpoint), and 2) its foundations in a normal probability distribution (i.e., the ability to identify the likelihood that the differences in selection rates are occurring by chance). Unfortunately, as with many other statistical formulas, the x2 suffers from a need for sufficient sample size. As a result, the x2 is not recommended when sample sizes are less than 30. The Federal Contract Compliance Manual (FCCM) as written by the U.S Department of Labor, Employment Standards Administration, Office of Federal Compliance Programs (OFCCP) states that if samples sizes are less 30, and the number of expected minority/female selection is less than 5, then the Fisher’s Exact test should be employed. Fisher’s Exact test is considered a more accurate statistic. Unfortunately, the Fisher exact calculation is extremely complex and requires processor power that was not readily available in 1978, when the UGESP were published. As a result, the analytical philosophy was to recommend the Fisher exact test ONLY when the sample sizes were less than 30. However, with personal computers now able to handle billions of computations in a matter of seconds they can quickly and efficiently calculate the Fisher exact statistic regardless of how large the sample sizes are.
It has been our experience that during OFCCP Compliance Reviews that many Compliance Officers are still focusing on x2 results. We recommend that you arm your company with more legitimate analyses using the Fisher’s Exact test.
To summarize, while the chi-square results will closely approximate the Fisher’s Exact test when sample sizes are greater than 30, there is no longer any reason to use the chi-square statistic. The Fisher’s Exact statistic will result in the exact probability associated with selection rate differences every time, regardless of sample size. Additionally, it is much more accurate with sample sizes less than 30. In EEO compliance work, Biddle Consulting Group finds the use of the Fisher’s Exact Test to be a critical tool for determining if comparison results are statistically significant. It is very common with moderate sample sizes for the Fisher’s Exact probability result to be slightly lower than the chi-square statistic, which can mean the difference between showing a significant or non-significant result in your affirmative action planning efforts. If OFCCP is making an argument that your have adverse impact based on the chi-square statistic and your Fisher’s Exact test says otherwise, wouldn’t you want to know?