5
Amazon Made A Sexist HR Bot
As one of the world's biggest companies, Amazon deals with a lot of job applications. We're not sure what percentage of those applications hail from the water sports community, but the cumulative amount was enough that in 2014, Amazon tasked one of their research groups with building an AI capable of filtering out "bad" applicants.
The good news? They managed to build a working demo. The bad news? It automatically disregarded any resume submitted by a woman. (We suppose shutting out half of the population is a way of cutting down on applications.) In order to help HRBot figure what a "good" application looked like, the development team fed it over a decades' worth of resumes, with the hope that it could learn how to identify a list of forerunner applicants that could then be passed on to the company's flesh-and-blood recruiters. The only problem was that the sheer sausage-factory-ness of that resume pile accidentally taught HRBot to weigh male candidates more favorably than female ones.
HRBot was more thorough than merely looking at the name at the top of each resume, though. It also analyzed the language of the resume in order to find any women who had sneaked past the initial purge. If a candidate included a reference to a women's college or a women's sport team, they were disqualified. If a candidate used language that didn't "read" male -- men are more likely to use verbs like "executed" and "captured" in resumes, for instance -- they were disqualified.
When technicians peeled back the system's code, they found it was weighing a candidate's gender characteristics more heavily than their technical knowledge or coding proficiency. Meaning that a frontrunner candidate could get away with knowing less than nothing about technology. Amazon tried to correct the bugs in the system, but no matter what they tried, its inner sexist would always win. In one of its final iterations, the team had written in so many safeguards against sexism that HRBot lost its mind and start recommending any applicant that was put in front of it. With no guarantee of success, or even that the system wouldn't self-engineer a way around any anti-sexism safeguards, the project was scrapped and placed in the same forgotten storage closet as the Fire Phone and Jeff Bezos' sexts.
4
Tumblr Uses AI To Find Porn, Disaster Predictably Ensues
In December 2018, Tumblr decided to take a big dump over its business model by announcing it was banning all forms of adult content. The ban, which prohibited users from posting genitalia, pornography, erotic artwork, and whatever "female-presenting nipples" are, was also a purge of existing content on the site, regardless of its age, all for the apparent aim of creating a "better, more positive" experience. (We're still not quite sure how a site featuring nothing except for "male-presenting nipples" wouldn't be the scariest, serial-killerest thing imaginable, but we'd' probably get used to it eventually.)