Hot(line) Topics: Can AI Write Performance Reviews?

By Ben Opp, SPHR, HR Hotline Advisor and Blaise Adams, aPHR, OD and Data Analytics Specialist
Published May 7, 2024

Q: We suspect that one of our managers used ChatGPT to write performance reviews for his direct reports, and we confirmed our suspicion using an online ChatGPT detection tool. Is it OK to use ChatGPT to write reviews? Should we confront this manager?

BEN: To help address these excellent questions, I’ve invited my colleague Blaise Adams, generative AI expert extraordinaire, to join me for this article. Blaise, let’s start with the question of identifying ChatGPT usage. Are online detectors, like the one this member used, any good?

BLAISE: Differentiating between human and AI work products is a fantastic goal. Unfortunately, the technology isn’t quite up to the task yet. Tools like ZeroGPT, GPTZero, and Copyleaks can easily give false positives when attempting to detect AI content. For example, if you ask GPTZero to examine the U.S. Constitution, it confidently proclaims (with up to 96% certainty) that the document was written by AI. Who knew the Founding Fathers had access to a large language model?

BEN: Let’s turn now to appropriate uses of AI tools to assist with writing performance reviews. If a manager were to give ChatGPT a list of an employee’s primary job functions, along with a numerical rating (say, from 1 to 5) to indicate performance in each area, is this an effective use of AI?

BLAISE: In short, absolutely not. Expecting ChatGPT to provide nuanced, detailed, and accurate output after providing it with limited information is setting yourself up for disappointment. Generic information in means generic information out.

BEN: I totally agree. This is going to result in a bland review devoid of the critical details needed to document the employee’s performance and support their growth. But, what if the manager gives ChatGPT a bulleted list of strengths and weaknesses along with brief examples. Could ChatGPT turn those notes into prose?

BLAISE: I think that’s a great application for generative AI. Instead of asking the tool to create information from nothing, you’re using it to reword, rephrase, and rework existing data. Even so, the more details you can provide ChatGPT, the better. Quality prompts come in three parts: give the AI model a role, assign it a task, and request a specific format for the output.

BEN: Further, AI tools should never be used on their own to evaluate an employee’s performance. We can use these tools to help us convert our ideas into a different format – notes to prose, for example – but the actual evaluative content needs to come from a human. Remember, the AI tool doesn’t know anything about your employees (unless you tell it) and is also likely to demonstrate bias when asked to make evaluative decisions.

BLAISE: So, to recap, AI detection tools alone are not absolute proof that someone has utilized generative AI to write a performance review, an email, or anything else. Ultimately, you’ll need to speak with the employee directly.

BEN: Definitely! But instead of confronting, I suggest an honest, judgment-free conversation. You can applaud the employee for adopting new technology and trying to increase efficiency, setting the table for a coaching conversation about safe and ethical uses of AI. You can also share your organization’s AI policy (contact HR Source for a sample!) and discuss how it applies to this situation. Overall, you want employees to feel comfortable coming clean about their AI experiments and asking for guidance.

Thanks for joining me today, Blaise! If HR Source members want to learn more about the AI tool landscape and safe use, where can they turn?

BLAISE: For starters, I recommend watching our Beyond the Buzz: What Is AI and How Do I Use It? webinar, available now on HR Source’s Online Video Training Store. During that 90-minute session, we break down the latest AI tools and discuss how to start using AI effectively and ethically.

And, as always, HR Source members can give us a call on the HR Hotline at 800-448-4584 or email us through the HR Hotline Online.

Do you have a workable policy or clear parameters defining the use of AI in your workplace?  Let’s start the conversation on the HR Exchange and see where we all fall.