Conducting Adverse Impact Analyses – How Should I Analyze My Transactions?

Conducting Adverse Impact Tests – How should I analyze my transactions?
by Danielle Yokoi, Analyst II, Biddle Consulting Group

Adverse Impact is a hot topic amongst federal contractors. Adverse Impact is a sensitive topic because the Office of Federal Compliance Programs (OFFCP) is generating the majority of their make-whole relief dollars from adverse impact in Applicants v. Hires cases. That being said, the regulations (41 CFR § 60-2.17) require federal contractor’s to analyze their “personnel activity (including but not limited to applicant flow, hires, terminations, and promotions) to determine whether there are selection disparities.” However, the regulations do not provide guidance on how to analyze this information. This is left to the discretion of the contractor.

There are many ways to analyze your transaction data. The key to a meaningful analysis is to compare appropriate data or to use a common phrase “compare apples to apples”. Your first priority should be to ensure that you are comparing complete and accurate data. For those of you who have dealt with similar analyses in your work you know that collect good data is a surprisingly difficult task. Let’s use External Applicants v. External Hires (Selection Rate Comparison) as an example. Since applicant data is the most difficult item to track, and as a result is heavily scrutinized during compliance reviews, it should be the first thing you focus on when preparing for an audit. If the new definition of an internet applicant applies to your hiring process, then you should start by applying the four criteria to ensure that you have the most accurate file possible. The new definition on an internet applicant can be found here: http://www.dol.gov/esa/regs/fedreg/final/2005020176.htm. Once you are confident that you have an accurate pool of external applicants who were considered for an external hire, then you are ready to start conducting your analyses. If you need a tool to analyze your transactions, we offer a free Adverse Impact Toolkit on our website (http://www.biddle.com/adverseimpacttoolkit/).

There are several ways to group data in order to analyze personnel activity. A few approaches would be: by job group, job title, requisition, or job title + department. Often the use of a job group to analyze transactions is too broad because it aggregates multiple positions that may have dramatically different skills, qualifications, and responsibilities. You should look closely at the most appropriate grouping for your organization and your hiring processes.

After determining what grouping method is appropriate, the next decision to make is which test should to employ. I will discuss the three most common, 80% (4/5th Rule), Chi2, and Fisher’s Exact.

The 80% rule is the old-school Impact Ratio Analysis method that is still around but it is fading fast in it’s usage since OFCCP started focusing on Systemic Discrimination. It compares groups to determine if the selection rate of females/minorities is within 80% of the selection rate of male/non-minorities. This rule was appropriate years ago when analyses were conducted by hand. Today, we have computers that allow us to use more powerful statistical tools with a click of a button.

Two common statistical tools previously mentioned are the Chi2 and Fisher’s Exact. The tools are similar in that they both generate a standard deviation (threshold ≥ 1.96). However, the Chi2 will generate an estimated standard deviation while the Fisher’s Exact will generate an exact standard deviation. Additionally, the Chi2 test decreases in accuracy as sample sizes become smaller and are therefore not appropriate in some cases. However, since OFCCP is accustomed to the simpler test method, they often use Chi2 during audits. We encourage you to rely on the Fisher’s Exact to ensure accurate results and to counter an potential arguments by OFCCP that you have statistical significance when in fact the Exact test may not. Exact tests can handle large and small sample sizes and as I previously stated, it provides exact results. BCG has successfully argued the use of the Fisher’s Exact test and its results during many audits in recent years.

A final note. It is recommended that any potentially sensitive analyses be conducted under Attorney Client Privilege to keep the results from being used against you by OFFCP or a plaintiff attorney.

Below is a summary of the steps detailed above.
Clean data thoroughly using the updated definition of an applicant
Choose an appropriate grouping method
Run analysis to identify legitimate problem areas
Choose a meaningful statistical test
Interpret results and re-analyze if necessary

Happy Analyzing!