AI TRiSM : Navigating the Maze of Data+AI Security with Confidence

The future belongs to Artificial Intelligence, or so one could safely assume after the advent of Generative AI (GenAI).Large Language Learning Models (LLMs) are proliferating across the globe, bringing transformations across industries and processes.

As GenAI gains global traction, it is important to recognize that inherent risks and challenges are also emerging in parallel. To illustrate it better, let’s take a quick look at a poll by Gartner conducted during a webinar on risks concerning GenAI.

The poll illustrated that data privacy is one of the foremost concerns when it comes to GenAI-associated risks. 42% of the respondents felt that there is a genuine data privacy concern as people usually lack insights into the AI models, such as what models exist in their environment, the data used for training the models, or its compliance with global regulations. Similarly, Hallucinations (14%) and Data Security (13%) were cited as the other most critical concerns. A fair percentage of the respondents also voiced their concerns about the models or output data being biased and unfair.

Unchecked AI - A Looming Catastrophe

When AI is not controlled appropriately, it could lead to a massive disaster. Let’s take a look at the real-world harms of AI to paint a better picture. In 2019, the then-Dutch Prime Minister and his entire cabinet had to resign from their seats after an AI algorithm went haywire. The Dutch taxation authority developed a self-learning AI algorithm to create risk profiles and fight fraud concerning childcare benefits. Due to the discriminative AI algorithm, thousands of parents were falsely accused of fraud, which consequently devastated many families.

The ChatGPT ban by a major consumer electronics company was yet another prominent incident that jolted corporations into recognizing the need for controlled AI usage. The generative AI was barred by the organization when a few of its personnel were found to be feeding sensitive codes as a prompt to the AI.

Hardly a month goes by when news like “AI gone wrong” doesn’t make the headlines. This raises a series of questions, starting with the primary concern- what are the critical gaps that lead to uncontrolled AI?

The Crucial Risks Arising from Artificial Intelligence

Organizations face a number of risks and challenges that hinder the safe adoption of AI. Let’s discuss some of the primary risks:

AI Model Proliferation

Large organizations aren’t limited to a single AI model to accelerate their operations and growth. In fact, a single organization may be using a number of LLMs, either directly deployed by the developers or accessed through a third-party (SaaS) application. It becomes challenging for organizations to keep a single inventory of all the AI models, let alone a catalog of Shadow AI. The Shadow AI is a bunch of ad-hoc or unsanctioned AI systems that exist in the environment but without proper IT governance. The lack of visibility into existing AI models in the environment puts AI governance, data security, privacy, and compliance at serious risk.

Inherent Risks in AI Models

Organizations currently don’t have a standard AI risk assessment framework. This makes it difficult for teams to get an accurate picture of the risks inherent in their AI models. With no concrete way to accurately assess risks, AI models tend to face issues like toxicity, bias, hallucination, and discrimination, to name a few.

Security Gaps in Models

AI models are different from traditional data systems. At a time, a single model can have a considerable volume of compressed information stored in it. The integrity or the security of the model can severely be compromised if organizations fail to make sure appropriate security or access controls are established in and around the models. Security gaps can render AI models incapable of safeguarding against manipulation, data leakage, or other malicious cyber threats.

Unprotected Training Data

Apart from AI models, it is increasingly important to protect the data that is flowing into the AI models, i.e., the training data. AI models require training data to work efficiently and deliver accurate results. It is important that teams should have a clear understanding of what data needs to be allowed for training the AI model and what type of data should be restricted, such as sensitive data. When teams lack the insights into which data is being leveraged to train the model, concerns may arise around access entitlements. Needless to say, inadequate access entitlements further result in potential sensitive data leakage and unauthorized access to training data.

Unguarded AI Agents and Prompts

Security, governance, and privacy controls shouldn’t be limited to AI models or training data alone. In fact, these controls should be extended to the prompts and agents. It is critically important that organizations place guardrails around AI prompts and agents, as leaving these critical areas could open doors to harmful interactions with the models, putting users’ safety and ethical principles at risk.

Ever-Changing Regulatory Maze

There are tons of AI regulations that are coming through. North America recently introduced the AI governance framework or a set of guidelines in the form of an AI Executive Order. Similarly, the EU AI Act is yet another comprehensive AI law that has turned heads when it was first introduced. Similarly, new regulations and standards will be coming and receiving amendments as the technology further advances. To comply with these regulations or standards concerning AI, it is imperative for organizations to have complete visibility around their AI models, training data, prompts, access entitlements, and processing policies. Without these crucial insights, it would be difficult to establish appropriate controls, let alone ensure compliance.

What is an AI TRiSM Approach?

AI TRiSM stands for Trust, Risk, and Security Management. It is a comprehensive framework that enables organizations to ensure “AI model governance, trustworthiness, fairness, reliability, robustness, efficacy, and data protection,” as defined by Gartner. In fact, as per Gartner, organizations that operationalize secure and trustworthy AI will achieve a 50% increase in their AI adoption and business goal attainment.

A 5-Step AI TRiSM Approach to AI Governance

Fortunately, there are ways that enterprises looking to enable the safe use of AI can integrate AI models into their data landscape while meeting legal requirements, upholding ethical standards, and driving positive business outcomes. Here’s how incorporating AI governance into a central Data Command Center enables the safe use of AI.

Discover & Catalog AI Models

To get insights into AI models and establish appropriate controls, it is vital for organizations to have a clear picture of the models or systems that exist in their environments. Organizations must first start with the discovery of all AI models across public clouds, private clouds, and the SaaS landscape. The aim of this discovery is to catalog all the models under one roof, especially Shadow AI. During the discovery and cataloging phase, discovery teams should catalog the models that exist not only in their own developer ecosystem but also in third-party systems, such as those used in SaaS applications.

Assess & Classify Risks

Organizations need to have a standard risk rating template. The template should cover various AI risks like toxicity, discrimination, hallucination, bias, copyright infringement, or model efficiency to provide teams with a detailed picture of the risks surrounding AI models. The assessment template should further include AI models used by vendors. This assessment can help organizations determine the impact of the models used by the vendors on the organization’s AI landscape. Vendor assessment may cover aspects like vendor’s AI models or training data handling measures, security controls, compliance policies, etc.

Map & Monitor Data+AI Flow

The next step is to understand the full context or relationship between AI models and systems with data flows, processes, and sources. Organizations must map the models to their associated data sources, processing paths, data flows, vendor systems or applications, risks, and compliance obligations. The objective of AI model and data mapping is to track the journey of the data across the AI ecosystem. Consequently, teams can proactively uncover AI governance, security, privacy, and compliance risks.

Establish Data+AI Controls