What is AI TRiSM and Why It’s Essential in the Era of GenAI

The launch of ChatGPT in late 2022 was a watershed moment for AI, introducing the world to the possibilities of GenAI. After OpenAI made ChatGPT available to the general public, its adoption was unprecedented, reaching 100M users faster than any technology ever.

This powerful new technology promises to have a significant impact on the global economy, on the scale of $4.5 trillion in annual global GDP, according to McKinsey, or up to $7 trillion, according to a Goldman Sachs’ estimate.

Organizations are understandably eager to leverage AI to capture those trillions.  And while “Consumer AI”, such as ChatGPT is based on troves of data scraped from the internet, many organizations wish to build “Enterprise AI”, leveraging their own proprietary data to increase the accuracy and relevance of outputs in their own business contexts, while creating competitive differentiation.

A key impediment to deploying Enterprise AI, is ensuring that proper governance and security controls are in place to protect sensitive data and align with relevant regulations. According to Gartner, “AI brings new trust, risk and security management challenges that conventional controls do not address”.  Samsung discovered this when its engineers leaked source code. Successful prompt injections have been demonstrated against almost all the popular consumer LLMs. Without oversight and controls, AI can output harmful content or violate intellectual property rights in a way that leads to brand damage or litigation exposure.  On top of that, a slew of new regulations have created confusion for organizations trying to comply with various governments.

What is TRiSM?

Gartner first introduced the concept of AI TRiSM in 2023, with its report AI Trust and AI Risk: Tackling Trust and Risk in AI Models. It  stands for Artificial Intelligence Trust, Risk and Security Management.

• Trust refers to AI outputs that are reliable, consistent and grounded in truth.  Trust has a major impact on the effectiveness of AI systems and is therefore critical to adoption.
• Risk refers to the potential for negative outcomes in the event of AI gone awry- outcomes like regulatory action, intellectual property violations, brand damage or accidental leakage of sensitive data.
• Security Management refers to the requirement to secure the expanded surface area of AI from malicious actors.

Since then, Gartner has built the concept of TRiSM into a full-fledged framework.  This framework helps organizations develop a comprehensive and risk-informed approach to organizing people, process and technology around data and AI assets in such a way that accelerates innovation while avoiding negative outcomes such as regulatory action, reputational damage or leakage of sensitive data.  Gartner predicts that “by 2026, organizations that operationalize artificial intelligence (AI) transparency, trust and security will see their AI models achieve a 50% improvement in terms of adoption, business goals and user acceptance.”

The TRiSM framework consists of multiple layers, each layer depending on and building upon the layer beneath it.

Traditional technology protection and AI infrastructure and stack layers sit on the bottom.  All infrastructure and applications must be secure for any enterprise system to be secure of course. These two layers are largely traditional cybersecurity capabilities applied to the AI stack and while important are not net new capabilities.  For organizations seeking to deploy AI, the focus of TRiSM is primarily on the three higher levels.

1. Information Governance
2. AI Runtime Inspection and Enforcement
3. AI Governance

Information Governance is the foundational layer upon which AI TRiSM capabilities are based.  Without rock-solid Information Governance capabilities, trust, risk and security can be severely compromised and even the most sophisticated guardrails can not be expected to make up for incomplete Information Governance.  AI is a powerfully democratizing force, helping users and systems traverse huge swaths of data, magnifying any gaps in Information Governance.

Organizations want to use their proprietary data in AI applications and workflows as a source of competitive advantage. Information Governance helps them do so while protecting sensitive information from loss or misuse, ensuring access entitlements are properly enforced, and complying with critical regulations. Fine-grained control of all data used to train or fine-tune AI models, provide context for user queries in a RAG system or feed AI agentic systems is required to protect enterprise data. Done well, Information Governance can organize data in useful ways to accelerate innovation.

AI Runtime Inspection and Enforcement sits atop Information Governance and refers to the ability to inspect all AI events in real-time, providing continuous assurances and evaluation of performance, reliability, security or safety metrics. AI Runtime Inspection and Enforcement goes beyond prompt level “guardrails”, to monitoring of all AI inputs and outputs with risk scoring for probability of adverse events that can be immediately triaged, remediated or outright blocked. AI Runtime Inspection and Enforcement depends upon information governance to provide detailed access controls and usage policies for all data used in AI systems.

Gaps in AI Runtime Inspection and Enforcement significantly reduce the likelihood that organizations will  be able to respond to threats proactively or deliver safe, trusted outputs in a reliable manner.

AI Governance sits at the apex of the framework, and seeks to provide a unified view of AI objects across the enterprise to facilitate the management of Trust, Risk and Security.  AI Governance does this by providing both pre- and post-deployment visibility and traceability of all AI used in an organization for policy enforcement and regulatory compliance, improving adoption while mitigating risk. Good AI governance accelerates innovation by ensuring the reusability of safe and trustworthy AI objects and data sets. AI Governance aligns with AI Ops and validates controls established in the Information Governance and AI Runtime Inspection and Enforcement layers.

Gaps in AI Governance result in an incomplete view of AI risks and inadequate controls, including the risks posed by shadow AI. Furthermore, gaps in AI Governance exacerbate the burden of regulatory compliance and the risk of incurring regulatory penalties.

TRiSM is not a tool or one-time project but rather an operating model that technology and business leaders should implement and make their own in order to compete safely in the era of Gen AI.  Responsibility for TRiSM falls on the organizations adopting AI in the enterprise and is not the sole burden of the model providers or hyperscalers.  As such, organizations should not rely solely on features of their core AI infrastructure and tools, such as “guardrails”.

In fact, Gartner explicitly states in their Market Guide for AI TRiSM that "Enterprises must retain independence from any single AI model or hosting provider to ensure scalability, flexibility, cost control and trust, as AI markets rapidly mature and change."

TRiSM Must Address New Paradigms

According to Gartner, “AI brings new trust, risk and security management challenges that conventional controls do not address”. The era of Gen AI represents three colossal shifts in how organizations manage and govern data and AI systems:

1. From governance of structured to unstructured data

An estimated 80%-90% of data is unstructured. As compared to structured data, governing unstructured data, like documents, emails, and media files, is distinctly challenging because it lacks a predefined format, making it significantly harder to discover, classify, understand context, and consistently apply security or compliance policies at any significant scale. The sheer volume and variety further complicate efforts to manage risk and extract value effectively.

2. Maintaining control when data moves through AI pipelines

Even when data is well-governed in source systems, organizations face a significant challenge in maintaining control throughout AI pipelines. As data flows through AI pipelines, movement and processing obscure data lineage and make auditing difficult, increasing security vulnerabilities and the risk of non-compliance. Source files that move through a pipeline may lose track of entitlement controls.

3. Governing models that are opaque to users and data scientists

Once data is exposed to a model in a learning phase, that data will be transformed into model weights internal to the model. Models create their own abstractions from training data that are opaque to users and data scientists. There is a significant loss of control and understanding of what happens inside a model. Any data exposed to AI could be output at unexpected times and in unexpected ways at some future point.

Gartner recommends that organizations undertaking AI TRiSM efforts develop or partner to deliver AI catalogs, data maps and continuous monitoring capabilities.

First is an AI catalog. Organizations must establish a complete inventory of AI entities used in the organization, including models, agents, and applications. All AI must be accounted for, including off-the-shelf and third-party applications. Models and agents that have been built or fine-tuned with enterprise data or contextualized via Retrieval Augmented (RAG) systems must also be accounted for.

Second is an AI data map. Each of those systems needs an explicit and detailed mapping of the data they utilize and have access to, including all processing, aggregation, and transformation steps they might undergo in an AI pipeline, all the way back to the source system. An AI data map is vital to gaining a complete view of risks and deploying adequate controls.

Lastly, a real-time continuous monitoring capacity for providing continuous assurance and system evaluation is required. Measures for trust, performance, etc, must be developed and systems should be regularly tested against them both offline and in a continuous, real-time manner.

In order to properly architect a TRiSM framework, organizations need to address the following technical requirements for Information Governance, AI Runtime Inspection and Enforcement and AI governance.

Information Governance Technical Requirements

The purpose of Information Governance technologies in TRiSM is to restrict AI and user access to only relevant and properly permissioned data throughout the lifecycle. In a study conducted by ISMG in partnership with Microsoft titled First Annual Generative AI Study, the top concern about the use of AI, cited by 80% of business leaders and 82% of cybersecurity professionals, was the potential leakage of sensitive data. A comprehensive approach to information governance is therefore critical in securing an organization's sensitive information and providing a solid foundation for TRiSM efforts.

Technology solutions for Information Governance must address key challenges that organizations face when trying to secure their data. First is discovery. According to a recent survey from Omdia, only 11% of organizations can account for 100% of their data. This issue is exacerbated by a patchwork of tools across hybrid and multi-cloud environments, giving fragmented views of enterprise data. Information governance solutions must be able to scan environments and bring back intelligence about the structured and unstructured data that exists throughout an organization.

The second challenge Information Governance technologies must address is the classification of sensitive data. Solutions that rely on keywords and manual tagging are insufficient for the volume and variety of unstructured data that organizations typically store. Solutions that merely “sample” data sets to determine if they contain sensitive data are also insufficient because PII or other sensitive data can sometimes be buried deep in unstructured data in unexpected places. Robust solutions conduct a deep scan of all data and automate classification with a high degree of accuracy and specific labels (such as PII, IP, password, etc) by analyzing context around potentially sensitive data.

The third challenge is overpermissioning. The Sysdig 2023 Cloud-Native Security and Usage Report found that 90% of granted permissions are not used. This suggests that users often have access to data they don’t need and possibly shouldn’t access. AI makes that data much more accessible to end users. Information Governance solutions should be able to identify overpermissioned users and data sets and enforce policies restricting access.

Lastly, as data is moved from a source system to be staged for use in AI, critical context is often lost. Entitlements, classifications, ownership and residency, and other information that is critical to managing risk and compliance can get lost. Information Governance solutions should preserve that metadata for use in AI Runtime Inspection and Enforcement as well as audits.

Beyond those common challenges, Information Governance technologies should clearly map data utilized and accessed by AI by capturing data provenance across complex pipelines that can include aggregations, processing and movement from source systems. Information Governance solutions must also establish data retention policies to aid in data minimization efforts and regulatory compliance.

A good Information Governance solution curates clean, sanitized data for AI that has security built-in from the beginning, not as an afterthought. Fine-tuning, RAG-powered solutions, or agents can leverage permissions, labels and other critical context that can be enforced at runtime. Done well, Information Governance accelerates the development of safe AI by making the right data easily available for use while making sure sensitive data is not exposed to the wrong users or systems, providing a solid foundation for AI Runtime Inspection and Enforcement.

AI Runtime Inspection and Enforcement Technical Requirements

The purpose of AI Runtime Inspection and Enforcement technology is to monitor for and address risks and threats as they unfold at runtime. Organizations need assurances that AI model outputs can be trusted, risks are mitigated and that systems are secure. Prevention is the goal; discovering a cyber attack or sensitive data leakage after it has happened does little to help. Therefore, AI Runtime Inspection and Enforcement technologies must first and foremost have real-time observability into AI events. An AI event is a discrete interaction or state change occurring within an AI system or workflow, involving one or more key components: human users, autonomous or semi-autonomous AI agents, AI models, and the data being accessed, processed, or generated. Because any of these interactions can be attack points for cyber threats or failure points for data protections, they must be addressed. Merely monitoring prompts for known “jailbreak” attempts, for example, is insufficient.

A short list of AI events must be monitored:

• The user submits a prompt
• A prompt is engineered/modified
• Agent receives user request and formulates a query for a model
• Agent retrieves context data (e.g., from a vector database)
• Agent sends prompt/query and context data to a model
• Model performs inference
• Model accesses training data or external tools/APIs
• Model generates a response or output data
• Agent receives model response
• Agent formats and delivers response to the user
• Detection of sensitive data in a prompt or response
• Policy violation detected (e.g., prompt injection attempt, harmful content generation)
• Update to a model's configuration or fine-tuning data

The first challenge is visibility into these various systems. Visibility across multiple models, applications, databases, pipelines, etc requires a high degree of interoperability, lest organizations suffer from fragmented or incomplete views of their AI events. McKinsey’s 2025 Study“ The state of AI: How organizations are rewiring to capture value” revealed that only 27% of organizations deploying AI actually monitor all outputs. AI Runtime Inspection and Enforcement solutions must integrate with many different systems and tools.

The second challenge in the AI Runtime Inspection and Enforcement layer of AI TRiSM is the number and variety of AI events that must be monitored. AI systems generate vast amounts of data. This becomes more pronounced as organizations develop and deploy more AI use cases, connecting different models to different data sets and applications or potentially creating a multi-agent system where each agent executes different sub-tasks in complex end-to-end business processes.  Human operators simply can not inspect all events in real-time to spot risks and threats.  Inspection of AI events at scale requires a high degree of automation.

The third challenge is that AIs are non-deterministic, meaning they take “fuzzy inputs” and create probabilistic outputs. From IBM, “How Observability is Adjusting to Generative AI”, "Unlike traditional software, LLMs produce probabilistic outputs, meaning identical inputs can yield different responses. This lack of interpretability—or the difficulty in tracing how inputs shape outputs—can cause problems for conventional observability tools. This "black box" phenomenon highlights a critical challenge for LLM observability. While observability tools can detect problems that have occurred, they cannot prevent those issues because they struggle with AI explainability—the ability to provide a human-understandable reason why a model made a specific decision or generated a particular output.” Explicit rules-based approaches to monitoring are likely to prove insufficient. Robust AI runtime inspection & enforcement tech should be flexible, context-aware and able to discern the meaning and intent of various actions, ie, robust AI runtime inspection & enforcement should utilize AI to inspect events for potential issues.

Lastly, is the adaptability problem.  AI systems, threats, and how people use them change constantly. Controls need to adapt just as fast. Static rules quickly become obsolete, and designing controls that can reliably handle the unpredictable or novel outputs and uses of AI is an ongoing challenge. AI runtime inspection & enforcement should be easily modifiable or have some capacity to learn and improve automatically.

Effective AI runtime inspection & enforcement facilitate the deployment of safe, trusted AI by building upon the Information Governance layer, utilizing labels to protect sensitive data, detecting drift from an established baseline of “normal” activity and detecting specific risks and threats in real-time to be remediated and fed back to sentinels in order to continuously learn and improve AI safety and security posture.

AI Governance Technical Requirements

The primary purpose of AI governance technologies is to provide a unified view of AI across the enterprise to facilitate trust, risk and security management. AI governance technologies must map the relationships between all AI models, agents, applications data and relevant policies to facilitate enterprise goals for TRiSM as well as compliance with regulations. Rapid identification of vulnerabilities or policy violations for swift remediation is how organizations manage TRiSM proactively.

Here, the familiar challenge of scale presents itself in a new way. The variety of tools, data sets and models across a fragmented landscape makes governance difficult.  Establishing and tracking data provenance with an explicit purpose documented for every bit and byte of data across complex environments proves to be prohibitively laborious without the right tools.  Managing versioning of models with documentation about their vulnerabilities, biases, etc., adds a layer of complexity. McKinsey’s study shows that organizations are rapidly deploying models in multiple domains.

To add to the challenges of complexity and scale, AI Governance needs themselves are evolving. Since the EU AI Act was formally adopted in 2024, there have been a slew of new regulations proposed or formally adopted in Japan, Brazil, Korea, and other countries, including many state-level regulations in the USA, like Colorado and California. As attacks evolve, security frameworks do as well, such as the OWASP top ten, NIST AI and others. To stay compliant with regulators and safety frameworks, AI Governance programs will need to meet changing needs.

The first and most critical requirement for AI governance is discovery and cataloging. AI governance tech must be able to scan enterprise environments to discover AI anywhere it is being used in the enterprise. Catalogs should lay out all models and artifacts with helpful information, such as model cards that describe model capabilities, characteristics, potential vulnerabilities and biases. Organizations should be able to produce a complete AI bill of materials and have a complete lineage for all assets. The EU AI Act, for example, requires the classification of AI systems into risk tiers based on data utilized, the intended purpose of the system and information regarding potential bias in data.  If regulators wish to see evidence, tracing data back to the source will be key.

Gartner recommends AI governance both pre- and post-deployment.  Pre-deployment, AI governance should provide visibility, traceability, as well as any required approvals or attestations. Once AI is deployed, continuous assurances via scanning and testing are necessary, as well as documentation of any policy violations.  Automated testing is highly recommended. Some solutions may include AI explainability functionality or red teaming and include full data lineage and provenance in data+AI maps that can be used for detailed audits.

Because many of the regulatory requirements are overlapping, a complete TRiSM stack composed of comprehensive AI governance tech sitting on top of robust AI Runtime Inspection and Enforcement and a foundational layer of Information Governance, can help an organization to meet varying regulatory requirements without a great deal of one-off effort. Many AI governance technologies will assist or automate the production of regulatory reports and link to evidence that may be needed in case of an audit.  AI governance should also facilitate the development and deployment of safe AI by identifying assets that can be safely re-used and re-configured.

Using AI to SafeGuard AI: Guardian Agents

In “Guardians of the Future: How CIOs Can Leverage Guardian Agents for Trustworthy and Secure AI,” Gartner recommends developing Sentinels and Operators to keep enterprise data safe. Sentinels are basically AI information posture management agents that provide the foundation for real-time Operatives. Sentitles effectively act to ensure preparedness, providing the environmental context and situational awareness needed by Operatives to identify risks and threats.  Sentinels assess the environment to identify vulnerabilities and apply policies offline to ensure the protection of sensitive data. Sentinels can filter, mask and redact sensitive data as it enters the AI ecosystem. They also establish a baseline of “normal” that will become important at runtime to detect deviations.

Operator agents assess and remediate specific risks such as data leakage, bias, legal or security risk. To do so, engines that execute policy enforcement should be able to process different policies simultaneously at run time in a single unified engine so as to enforce policy with specificity and without potentially conflicting analyses that might arise from siloed tools.  Operators should block the event for policy violation, for example, if a user enters a prompt that contains PII.  More robust solutions can identify anomalous events to be triaged, or route events for remediation or based on which policy they violate.  For example, prompt injection attempts might be routed to cybersecurity, while outputs that contain sensitive data may be autoremediated via masking or redaction, while the event is flagged for review by governance teams.

Sentinels and Operators work together synergistically to create a robust system of controls where Sentinels maintain a well-prepared environment and establish a baseline from which Operators can detect deviations.  By leveraging the controls established by Sentinels offline, Operators can monitor, detect and remediate issues in real-time before passing back to sentinels information about those threats for Sentinels to learn and improve security posture.

Effective AI Runtime Inspection and Enforcement solutions are able to meet all these challenges to detect risks and threats across all AI events, and they are able to enforce policy or remediate issues.  It is this ability to act that makes them truly effective at runtime. In “ Guardians of the Future: How CIOs Can Leverage Guardian Agents for Trustworthy and Secure AIOperator Agents,” Gartner recommends the development of “Operator Agents” to work hand-in-hand with Sentinel agents detailed in the previous section.

Summary and Benefits of TRiSM

The rapid proliferation of Generative AI, exemplified by ChatGPT's explosive growth, presents immense opportunities but also introduces significant new trust, risk, and security challenges that conventional controls cannot adequately address Issues like sensitive data leaks, successful prompt injections, intellectual property violations, and a complex, evolving regulatory landscape highlight the urgent need for robust oversight. To navigate this, Gartner introduced AI TRiSM – a framework designed to ensure AI outputs are trustworthy, risks are managed, and the expanded AI attack surface is secured

AI TRiSM provides a layered approach, focusing on three key AI-specific technology functions built upon traditional security and infrastructure.  Foundational is Information Governance, which ensures comprehensive discovery, classification, access control, and context preservation for all data (especially unstructured) used in AI pipelines, mitigating risks magnified by AI's ability to traverse vast datasets. Information Governance is the foundational layer for TRiSM efforts to build upon. “For many organizations, weak information governance is emerging as the major obstacle to wider GenAI rollouts”. This is because information governance efforts may come from siloes like security or identity and access management that were never intended to provide complete information governance holistically.  Gaps in Information Governance will quickly result in vulnerabilities and therefore must be secured before AI Runtime Inspection and Enforcement and AI governance can be expected to be effective. According to Gartner, “Information governance fractures the surface, causing a renewed immediate need for a unified approach across different organizations for protecting information used by AI.”

AI Runtime Inspection & Enforcement then provides automated monitoring and control across all AI events to proactively detect and remediate threats like data leakage or policy violations.  At the apex sits AI Governance, providing unified visibility and traceability across all enterprise AI assets.

Ultimately, AI TRiSM is not merely a compliance checklist or a single tool, but a necessary operating model for enterprises aiming to leverage AI safely and effectively. A robust and well-integrated TRiSM stack delivers very tangible benefits – Gartner predicts that “by 2026, organizations that operationalize artificial intelligence (AI) transparency, trust and security will see their AI models achieve a 50% improvement in terms of adoption, business goals and user acceptance.”  Financial and reputational damage from the potential leakage of sensitive data or output of harmful content can be incalculable.   New regulations such as the EU AI Act provide further incentive for companies to invest resources in AI TRiSM.

While governance is frequently seen as a limitation on development speed, TRiSM  maintains speed over the long term by helping organizations build and deploy without incurring significant debt that will later have to be remedied for example exposing sensitive data to a model.

Implementing AI TRiSM is no longer optional for enterprises serious about leveraging AI. It's fundamental to sustainable success.

Ready to build a more trusted, secure, and resilient AI strategy?

Securiti.ai is helping our customers implement TRiSM programs of their own that accelerate the development of safe AI while protecting enterprise data and managing compliance with many regulation frameworks.  Securiti.ai meets all the criteria for each layer of the TRiSM framework proposed by Gartner, beginning with robust Information Governance.  Securiti provides adaptable AI TRiSM solutions that address customers' immediate needs and evolve with their expanding requirements throughout their journey.

Click here to Learn more about how Securiti can help

Frequently Asked Questions (FAQs)