Are governing AI properly?

To ensure that AI is governed effectively, we recommend the following practices:

Conduct AI Impact Assessments: Similar to DPIAs, these evaluate the potential risks of your AI systems and help you identify biases, privacy issues, error rates, and impacts on rights. The EU and others may mandate these for high-risk AI use cases, so get ahead by doing them now.

Data management for AI: Ensure any personal data used in developing or training AI is permitted and protected. Anonymise or pseudonymise wherever possible. Maintain a record of what data went into training and under what lawful basis.

Bias mitigation: Actively test your algorithms for bias or disparate outcomes. Use diverse training data and consult domain experts. If issues are found, retrain or put constraints in place. Document these efforts – under future AI laws you might need to prove you took steps to avoid bias.

Transparency and explainability: Strive to make your AI’s functionality explainable to users. Even if the algorithm is complex, provide users with understandable reasons for outputs, e.g. “Our algorithm suggests a higher risk of X because it noted [factor1] and [factor2] in your data.” Also, label AI interactions clearly (don’t make a chatbot pretend to be human).

Human-in-the-loop: For any significant decisions or recommendations (especially those affecting health or eligibility for services), consider having a human review or an override mechanism. Many regulations encourage or require human oversight for high-impact automated decisions. Even if not required, it’s a good safety practice.

Monitoring and iteration: Once an AI feature is live, monitor its outputs and user feedback. Set up a process to handle complaints or corrections (like if a user says the AI’s suggestion was wrong or harmful). Continuously improve the model or rules as needed.

Define roles and responsibilities: Establish who “owns” AI governance in your organisation. Some companies set up an AI Ethics Committee; others assign the Chief Data Officer or similar to oversee AI compliance. Involve multidisciplinary perspectives – technical, legal, clinical (if it’s health advice), etc.

In short, you should treat your AI with the same rigour as you would a core product offering – because to regulators and users, it is just that. The goal is to reap AI’s benefits (personalization, scalability, efficiency) without undermining privacy, fairness, or safety. The companies that succeed in doing this will stand out as trustworthy innovators in the health tech field. Those that rush AI features to market without these safeguards may quickly find themselves in regulatory hot water or at the wrong end of a viral news story about an AI mishap.

Contact us to find out more.