Amazon scraps secret AI recruiting tool, showes bias against women

According to them, the system was meant to review job applications and give candidates a score ranging from one to five stars.

Amazon.com Inc's machine-learning specialists uncovered a big problem: their new recruiting engine did not like women. A new report from Reuters shows the technology giant had to get rid of an internal project used to vet job applications after the recruiting tool displayed an inherent bias against women.

"Everyone wanted this holy grail", one of the people told Reuters, adding: "They literally wanted it to be an engine where I'm going to give you 100 resumes, it will spit out the top five, and we'll hire those". It downgraded CVs if found words such as "women's" and penalised graduates of all-female colleges.

Engineers found the AI was unfavourable towards female candidates because it had combed through male-dominated resumes to accrue its data.

Amazon never exclusively relied on these online recruiting tools though, and disbanded the unit that created it by the start of a year ago, sources said.

According to a 2017 survey by talent software firm CareerBuilder, 55 per cent of human resources managers in the United States expect AI to be a regular part of their work within the next five years. To do this, the team created 500 computer models and taught each of them to recognize 50,000 terms from previous applicants' resumes.

Just like Tay, it seems Amazon's AI project was a victim of its upbringing.

Stevie Buckley, the co-founder of United Kingdom job website Honest Work, which is used by companies such as Snapchat to recruit for technology roles, said that "the basic premise of expecting a machine to identify strong job applicants based on historic hiring practices at your company is a surefire method to rapidly scale inherent bias and discriminatory recruitment practices".

According to Reuters, the company's human resources department used the system to generate recommendations but never relied entirely on those recommendations when filtering candidates.

A large number - including Hilton and Goldman Sachs - are turning to machine learning options to automate their recruitment process, either in-house or through recruitment as a service solutions.

Still, John Jersin, vice president of LinkedIn Talent Solutions, said the service is not a replacement for traditional recruiters.