There’s something interesting about a company being able to tell the public it tests its artificial intelligence for discrimination, while the results of certain tests remain protected from disclosure in a lawsuit alleging that the technology discriminates.

That’s the situation in the ongoing case against Workday, whose automated hiring software can determine which applicants ever reach a human recruiter.

In a discovery order filed May 29, U.S. Magistrate Judge Laurel Beeler ruled that Workday did not have to turn over certain internal AI bias-testing materials requested by the plaintiffs. The court found those materials were protected by attorney-client privilege because Workday’s attorneys had curated the data and used the testing to provide legal advice.

The plaintiffs argued that the information should be available during discovery. They also argued that Workday had waived its privilege by publicly discussing its bias-testing program. The court disagreed, finding that acknowledging the existence of testing wasn’t the same as relying on the protected test results to defend the lawsuit.

I can understand both sides of that argument.

Attorney-client privilege serves a legitimate purpose. Companies need to be able to consult their attorneys, investigate potential legal problems, and receive advice without assuming every confidential communication will eventually become public.

If that protection disappears whenever a company examines a possible problem, there’s a risk that organizations will become more reluctant to look for problems in the first place.

But consider the position of someone who believes an automated hiring system unfairly rejected their application. They don’t know exactly how the system evaluated them, what information it considered, or whether the company already identified potential problems through its own testing. Much of what they would need to understand the decision is controlled by the organization they’re challenging.

Now they’re told that some of the company’s internal testing is protected. I have a problem with organizations asking people to trust systems they can’t meaningfully examine, particularly when those systems influence decisions that affect someone’s livelihood.

I’m not suggesting companies should lose attorney-client privilege just because they use artificial intelligence. But there’s a difference between protecting confidential legal advice and establishing that a system is reliable enough to make decisions about people. One doesn’t establish the other.

An employer might reasonably rely on assurances from a software vendor that its hiring tools have been tested for bias. But what does the employer actually know about those tests? Has it examined the methods and limitations, or is it accepting the vendor’s assurances because the software is widely used?

If an organization is going to rely on automated systems to make consequential employment decisions, it needs enough information to understand what those systems are doing. That responsibility doesn’t disappear because the vendor conducted an internal audit or because certain materials are protected by law.