CIOs have long struggled with AI reliability issues, given problems with training data, model interpretations, and inconsistent data weighting delivering various levels of bias. IBM officials at next week’s shareholder meeting will have to address those concerns directly, as they face a shareholder motion demanding increased visibility into how it manages AI bias, a thorny issue that also affects all of the other major AI players. 

The shareholder resolution is demanding that IBM “issue a report, within the next year, on the methods used to eliminate bias from the Company’s artificial intelligence (AI) models, Including an assessment of the risk that seeking to avoid disparate impact in outputs will undermine the accuracy of, and trust in, those outputs.”

IBM’s official response to the resolution asks shareholders to reject the proposal. “Since releasing its first Granite model, IBM has been transparent with its data management and training procedures via technical reports, model cards, and other model documentation,” the company said. “The IBM models are open source In order to foster transparency. Moreover, IBM publicly provides the information sought by this proposal in its submissions to Stanford University’s Foundation Model Transparency Index (FMTI).”

FMTI is a benchmarking initiative that looks at how transparent companies are about their foundation models, measuring disclosure across areas like data sources, training methods, evaluation metrics, risks, and governance practices, to help stakeholders understand how responsibly and openly these models are developed and deployed.

IBM’s response added, “information related to mitigating bias that the proponent requests is largely already publicly available for consideration by stockholders.” Therefore, it argued, preparing such a report “would not provide new meaningful information and it is not in the best interests of IBM stockholders, as it will divert management’s attention and would be an inefficient use of corporate resources.”

Beyond its argument that it already provides such AI bias transparency, the company pointed to its customers’ ability to fine-tune their models to resolve any specific bias concerns.

“IBM models are smaller and targeted towards enterprise clients and use cases,” it said. “These models are not general purpose, consumer-facing models. Therefore, our open-source models are built in a manner that allows our clients to build an AI solution that will address their specific needs. IBM developed several methods to allow clients to address bias issues that may arise as they train the AI system. In other words, IBM not only provides the building blocks for its clients’ AI solutions, but also provides the tools to help more clients address bias.”

An industry-wide problem

Analysts and consultants generally found IBM’s position correct, but most pointed to the AI bias issue as an industry-wide problem impacting all of the major AI providers and all of their enterprise users. 

Sanchit Vir Gogia, chief analyst at Greyhound Research, said, “IBM’s stance deserves to be taken seriously, but not at face value. The company is right to say that bias mitigation, fairness frameworks, and governance controls are already built into its AI systems. That is not in dispute. In fact, compared to much of the market, IBM has been more deliberate than most in turning responsible AI from a set of principles into something operational.”

But, he added, “when IBM points out that customers can and should address bias through fine-tuning and governance, it is quietly acknowledging a limitation. It is admitting that whatever happens at the model layer is not the end of the story. It is only the beginning of it.”

Manish Jain, a principal research director at Info-Tech Research Group, saw the IBM position as correct, but also as the latest example of large vendors shifting responsibility for AI accuracy onto their enterprise customers. 

“I see IBM’s board’s stance being broadly consistent with industry practice, which is doing everything to shift the responsibility of removing bias towards customers,” Jain said. “In fact, many independent software vendors (ISVs) are also taking a similar position and saying to their customers, ‘We’ll provide the compass, you chart the course.’ Unfortunately, accountability is the victim. Regulatory guidelines, independent audits, standardized benchmarks, in addition to clearer disclosures, are extremely important to ascertain this accountability.”

Noah Kenney, principal consultant for Digital 520, had similar feelings about IBM’s response. 

The shareholder demand for more transparency “is asking the right question for the wrong reason,” Kenney said. “The proponent frames disparate-impact correction as a threat to accuracy, but the real issue is that IBM, and every major model provider, is measuring bias at the output layer when most of it originates upstream. You cannot fairness-tune your way out of a training data problem.”

He noted, “IBM’s response is accurate on the facts. FMTI scores, model cards, FairIQ, Equi-tuning, FairReprogram, and the Granite transparency posture are all real, and more than most of their peers publish. The gap is not disclosure. The gap is that the industry has converged on post-hoc mitigation as the dominant paradigm, and post-hoc mitigation has diminishing returns once a model is trained.”

Mike Leone, VP/principal analyst at Moor Insights & Strategy, pointed out that IBM is doing a better job than most AI vendors in terms of bias transparency, and that the industry needs to address the issue globally. 

“IBM discloses more of its AI bias and transparency work than most vendors. IBM has built specific bias mitigation methods into the stack rather than just talking about bias at a high level. A new annual report would mostly repeat what’s already out there,” Leone said.

“I truly don’t think eliminating bias is possible, and that’s not an IBM problem,” he added. “The whole market is operating the same way in that any model trained on human-generated data carries the biases of whoever made it, which is exactly the same as humans would do. What vendors can do, IBM included, as they do a bunch of this already, is measure it, disclose it, monitor it after deployment, and give customers tools to adapt. I’m in the camp that anyone promising to completely eliminate bias is telling you what you want to hear.”

‘Unbiased’ can’t be defined

Part of the answer to the AI bias problem is technological, but there is an underlying fundamental issue of bias that cannot possibly be defined. 

Carmi Levy, an independent technology analyst, observed, “the very definition of the word, unbiased, simply doesn’t exist, because what might seem unbiased or perfectly fair to one stakeholder might be perceived as wildly biased or unfair by another.”

Within that context, he noted, “the notion of eliminating any and all forms of bias from the AI equation is unrealistic. At best, vendors should be aiming for mitigation instead of outright elimination. They might also want to devote more resources toward transparency. Although sharing too much can compromise their competitive market position, there’s no reason why a carefully balanced and communicated messaging strategy can’t alleviate stakeholder concerns over bias without giving competitors undue advantage.”

Complete bias removal impossible

The AI bias issue is sometimes subtle, as it chooses which of the relevant details it should use in answers and in what sequence. But in other instances, such as when racial and gender prejudices are reinforced by AI working for human resources, the bias can potentially appear quite blatant. California, for example, wants to force AI vendors to prove that they have strong bias safeguards. 

However, said Gartner VP analyst Nader Henein, “completely removing bias is impossible, which is why almost every piece of AI regulation focuses on AI systems that make decisions that will impact people’s lives or people’s livelihoods and introduce obligations such as human oversight.”

For example, he said, “a recruitment application that sorts applicants by the suitability to a job role should be used by a trained recruiter who understands that this is AI, that it can make mistakes, and they are responsible to oversee the AI system, much in the same way that you oversee an entry level employee taking on sensitive work, the difference being that oversight is permanent.”

Chris Hood, an independent AI strategist and former head of Google’s strategy and transformation, also said that IBM’s position is legitimate, but it’s not enough.

“IBM’s position is technically defensible and practically insufficient,” Hood said. “Publishing bias mitigation reports and giving customers fine-tuning options are reasonable steps. They are also steps that address the symptoms rather than the architecture. IBM is describing what it does to manage bias. The harder question is whether bias in foundation models is manageable at all, or whether it is structural.”

He noted that the models learned from human-generated content, which carries “every historical imbalance, cultural assumption, and factual error humans have ever produced at scale. You can audit it, weight it, and filter it. You cannot eliminate it. The data is what it is.”

Potentially more of an issue is the personal bias that every user brings to every AI interaction. “A geopolitically charged question asked by someone in the United States and the same question asked by someone in another country will be interpreted differently, evaluated differently, and potentially produce different outputs. This layer is almost impossible to govern at the model level because it lives in the interaction, not the training,” Hood noted.

He added: “IBM recommending shareholders reject this proposal while pointing to existing efforts is a reasonable governance posture. It is also a posture that treats bias as a solved problem rather than a managed one. The difference matters enormously as agents move from answering questions to making decisions.”

IBM has agreed to settle a complaint from the US Justice Department around its initiatives to diversify its workforce and to encourage hiring of underrepresented groups, contrary to a presidential directive. The federal contractor also agreed to pay the government roughly $17 million.

The pressure from the Trump administration to eliminate workforce diversification efforts, typically known as DEI (Diversity, Equity, and Inclusion) programs, has persuaded many companies, including Meta, Google, Amazon, Salesforce, Intel, OpenAI, Tesla and Zoom, to publicly back away from those diversification efforts. A few companies, including Apple, Microsoft, Nvidia and Oracle, have held firm in favor of DEI, for the most part. 

The government’s official position states that age, race, sexual preference, and gender should have zero impact on hiring decisions. Diversification proponents counter that workforce composition will stay stagnant unless explicit efforts are made to diversify.

Focus of settlement

The Justice Department settlement focused mostly on IBM’s role as a government contractor.

The government filing said IBM made “false claims” and “false statements” to the government regarding hiring practices in connection with IBM’s government contract work.

“As a federal contractor, IBM was required to comply with anti-discrimination requirements as set forth in Title VII of the Civil Rights Act of 1964,” the settlement said, adding that IBM “discriminated against employees during employment and applicants for employment because of race, color, national origin, or sex, and failed to treat employees during employment without regard to race, color, national origin, or sex.”

Beyond hiring practices, the government also opposed hiring goals that encouraged diversity, including “developing race and sex demographic goals for business units and taking race, color, national origin, or sex into account when making employment decisions to achieve progress towards those demographic goals” and using those same criteria to offer “certain training, partnerships, mentoring, leadership development programs, educational opportunities or resources, and/or similar opportunities only to certain employees.”

The agreement also said that the deal “is neither an admission of liability by IBM nor a concession by the United States that its claims are not well founded” and added that IBM agreed to the settlement “to avoid the delay, uncertainty, inconvenience and expense of protracted litigation.”

Acting US Attorney General Todd Blanche issued a statement saying, “racial discrimination is illegal, and government contractors cannot evade the law by repackaging it as DEI.”

IBM did not respond to an email seeking comment.

Companies can work around biases

Bryan Howard, the CEO of recruiting strategy consulting firm Peoplyst, said he would encourage enterprises to simply move their workforce diversification efforts earlier in the recruitment process. 

“There’s a big difference between candidate pool and the selection process,” Howard said, suggesting that there are no federal rules limiting outreach choices. If, for example, a company wanted to increase workforce representation for a particular group, then the job notice should be focused on universities and other places where that group is well represented.

“Expand your pool and do not contract it. Fish in the ponds where those people are,” Howard said. “Increase diversity by simply recruiting from diverse sources.”

Howard also said the government position leverages last year’s US Supreme Court decision in Ames v. Ohio Department of Youth Services, where the court held that reverse discrimination is illegal. 

Complicating diversification efforts today are two popular recruiting/hiring tools pushed by HR: Using genAI to filter a massive number of applicants and only present a small handful to the hiring managers to choose from; and referral programs in which employees are offered cash incentives if they recommend job candidates who are eventually hired.

AI’s bias is to seek job candidates whose profiles most closely resemble that of the current workforce. In other words, AI wants to learn everything it can about who the company has hired before, to help it determine the attributes to look for. 

Referral programs, Howard said, also tend to attract people with the same characteristics as the existing workforce. Even though those referral hires tend to stay with the company longer, “if you have a population that is already skewed and that is the population recruiting, the existing bias will likely continue.”

Settlement could hurt recruitment efforts

Consultant Brian Levine, executive director of FormerGov, said it is difficult to interpret the settlement as anything other than opposing DEI efforts. 

The US Justice Department, where Levine once worked as a federal prosecutor, ”has issued a multi-million dollar penalty for company policy that seemed to be intended to encourage diversity,” he said. “As with Anthropic, in this new world, sometimes organizations may be forced to choose between ‘the law’ as it is currently being interpreted by some, and a good faith effort to positively influence society, or at least to minimize societal harm.”

Levine said some enterprises may try to overcompensate to keep the current administration happy.

“Fearing financial penalties, some companies that work with the federal government will now choose to ensure their DEI program is fully dismantled,” Levine said. “Other companies may choose to cease working with the federal government and/or may choose to keep, or even double down, on their DEI program. If Anthropic is any indication, these latter companies may ultimately be rewarded in the market.”

Flavio Villanustre, CISO for the LexisNexis Risk Solutions Group, added that this settlement might end up hurting tech recruitment efforts. 

“I think that this will force organizations to reframe their DEI programs to not upset the DOJ, which could have an impact on hiring of individuals in certain classes and could result in overall less diversity,” Villanustre said. “Diversity is an important part of building resilient, successful organizations, so this could have a broader impact than just the one at hiring time.”