AI in HR: Beyond the noise, there are benefits

During the summer, I spend a lot of time on one of my favourite projects: the Top HR Products of the Year Award sponsored by Human Resource Executive and the HR Technology Conference. Every year, we receive hundreds of submissions, and conduct dozens of meetings and demonstrations to arrive at 10 or so of the very best, most innovative products of the year.

So far this year, we have seen many cutting-edge examples of how, where and why new forms of artificial intelligence technologies, especially generative AI, are being introduced into HR technologies. In fact, we have seen so many examples of AI in the Top HR Products submissions, that in our last demo of the day recently—of a really interesting solution that was not driven by AI—I remarked, “It was kind of refreshing to have a demonstration where no one mentioned ‘GPT’ once!”

But while we may be getting a case of “AI fatigue” lately, it is important for HR leaders not to allow concerns about the hype to get in the way of understanding how these new AI technologies can positively impact HR and help drive better people and business outcomes. Certainly, the HR technology providers can share lots of examples of how their particular flavour of AI solutions is helping customers and employees, but sometimes it is grounding to look for alternative sources of information about heavily hyped technology like AI.

Recently, while researching industry trends and news for our ongoing The Workplace Minute mini-podcast, I came across two interesting pieces of independent research on how AI technology is impacting work and workplaces. Notably, these studies were not conducted or sponsored by HR technology solution providers themselves—which, if nothing else, makes them at least neutral to the hype machine surrounding AI in HR right now.

The effect of AI on improving diversity in hiring

In a paper titled Does Artificial Intelligence Help or Hurt Gender Diversity? Evidence from Two Field Experiments on Recruitment in Tech, researchers at Monash University and the University of Gothenburg explored how AI recruitment tools can impact gender diversity. In particular, they looked at the male-dominated technology sector, both overall and separately for labour supply and demand.

In part one of the experiment, the researchers created an online job posting for a web designer, eventually identifying over 700 interested job seekers to complete an application. They randomly varied whether the candidates were informed that their application was to be evaluated by AI software or by a human hiring team, not changing any other aspect of the application process.

The researchers found that, on the application side, the use of AI in recruitment increased the proportion of women completing the application by about 30 percentage points relative to men. This led to a narrowing of the gender gap in application completion rates by 36% when compared to recruiting without assistance from AI, resulting in both women being more likely to complete the application and men being less likely.

In part two of the experiment, the researchers studied the impact of AI technology usage during assessment of the applications by using over 500 “real” people within the technology industry to act as the hiring team. The study randomised whether these professional evaluators had access to the applicants’ assessment scores provided by the AI technology, as well as whether the human evaluators could infer the applicants’ gender from their names, or other personally identifiable information.

The researchers examined evaluations for candidates who applied under the hiring team and those who applied using the AI software assessment method. The results were striking. The researchers found that women were scored significantly lower than men when names revealing gender were shown, but equal to men when names, and thus gender, were hidden from the human assessment process.

Ultimately, the researchers found that adding AI to the recruiting process increased the representation of women and changed the gender distribution of potential hires—in some cases, more than doubling the fraction of top female applicants. Essentially, AI generated more qualified women to apply and reduced human evaluators’ bias against women in the review process, leading to more opportunities for women in technology roles overall.

The productivity effect of AI technologies like ChatGPT

Most of us have tried out ChatGPT by now but perhaps have seen mixed results, leaving us wondering about the importance and relevance of ChatGPT in the workplace. The major question now for business leaders is whether tools like ChatGPT will improve productivity and business outcomes and empower workers with newfound skills.

A study from MIT’s Department of Economics designed to try to answer this question found that participants using OpenAI’s ChatGPT solution saw improved productivity, and there was an increased likelihood they would use ChatGPT in future tasks. In the controlled study, the researchers concluded that “the technology will be more strongly complementary to human workers,” meaning it assesses tools like ChatGPT as a way to empower workers. But how these tools are actually implemented in the real world remains uncertain.

The experiment included over 450 college-educated professionals, half of whom were randomly assigned to have access to ChatGPT after completing their first assignment. The assignments were writing-based tasks including press releases, short reports and “delicate emails,” mimicking those that grant writers, marketers, consultants, data analysts and HR professionals would do in their day-to-day work.

READ: HR leaders take wait-and-see approach to generative AI

The study found the group that was given access to ChatGPT was able to decrease the time taken to accomplish a task by 11 minutes, while the tool also increased the quality of their work outputs. Notably, the performance of the treatment group (those using ChatGPT) increased between their first assignment (without ChatGPT) and subsequent assignments (with ChatGPT), which the study concluded could close the inequality gap between skilled and unskilled labour, over the long-term.

This improvement in speed and quality has been anecdotally noted for many of us using ChatGPT. But the study provides hard evidence that workers armed with ChatGPT can be more productive and perform tasks better.

With AI technology hype seemingly at its apex, it is important to seek out legitimate, independent sources that analyse the real impact of these emerging AI tools in the workplace. For now, the AI technology effects appear to be promising, and HR leaders should take the results from these early studies as proof points that AI technology can deliver on what are certainly elevated expectations.

About the author: Steve Boese is HRE’s Inside Tech columnist and chair of HRE’s HR Technology Conference. This article first appeared on HRExecutive.com.