Ethical AI Solutions: An Insight with Cal Al-Dhubaib from Pandata

Businesses neglecting to implement AI ethically are likely to face significant repercussions as regulations catch up with rapid technological advances. The proposed AI Act in the EU mirrors the enforcement principles of GDPR, but it introduces even steeper penalties, reaching up to €30 million or 6% of an organization’s annual revenue. Various other nations, including China and an increasing number of U.S. states, are moving toward similar regulatory practices.

Pandata specializes in human-centered, explainable, and trustworthy AI. Based in Cleveland, this company is dedicated to providing AI solutions that offer a competitive advantage for enterprises while adhering to ethical guidelines. Recently, AI News had a discussion with Cal Al-Dhubaib, the CEO of Pandata, to delve deeper into the significance of ethical AI solutions.

AI News: Can you briefly describe what Pandata does?

Cal Al-Dhubaib: Pandata assists organizations in designing and developing AI and machine learning solutions, concentrating on heavily regulated sectors such as healthcare, energy, and finance. We prioritize the implementation of trustworthy AI and bring extensive experience in handling sensitive data and high-risk applications. Our clientele includes esteemed organizations like Cleveland Clinic, Progressive Insurance, Parker Hannifin, and Hyland Software.

What are some primary ethical challenges associated with AI?

CA: The past five years have seen notable shifts, particularly in our capability to swiftly train and deploy intricate machine-learning models utilizing unstructured data like text and images. This evolution has brought forth two main challenges:

• Defining ground truth has become increasingly complex. For instance, summarizing an article with AI might yield multiple ‘correct’ interpretations.
• The models themselves are more sophisticated and challenging to interrogate.

The most significant ethical concern we encounter is that our models may fail in ways we cannot predict, leading to numerous instances of physical harm or biases related to race and gender.

How crucial is “explainable AI”?

CA: As model complexity has grown, the field of explainable AI has expanded. Sometimes, simpler models are employed to clarify the workings of more advanced models that excel at specific tasks. Explainable AI is vital in two key scenarios:

1. When an audit trail is essential to substantiate decisions made.
2. When expert human decision-makers must act based on the outcomes produced by an AI system.

Are there scenarios where AI should be avoided by companies?

CA: Initially, AI was primarily the realm of data scientists. However, as the technology becomes widely adopted, we find ourselves collaborating with diverse stakeholders, including user experience designers and business leaders. Interestingly, fewer than 25% of professionals consider themselves data literate (HBR 2021), often leading to mismatched expectations regarding AI’s capabilities. I suggest three key rules:

• If a task can be explained through clear procedural steps or rules, investing in AI may not be warranted.
• In cases where a task is inconsistently executed by qualified experts, it is unlikely that AI will effectively identify uniform patterns.
• Exercise caution with AI systems that significantly impact human quality of life—whether financially, physically, mentally, or otherwise.

Should AI regulations be tightened or relaxed?

CA: In some respects, regulatory measures are sorely needed. Regulations have struggled to keep pace with innovations. As of 2022, the FDA has reclassified over 500 AI-based software applications as medical devices. The forthcoming EU AI Act, expected to roll out in 2024-25, will establish explicit guidelines for AI applications that impact human life. Similar to how GDPR ushered in significant changes in data privacy, the EU AI Act will compel organizations to adopt a more disciplined approach to model deployment.