New Zealand’s Privacy Commissioner Issues Guidance on AI Usage

Contributors

Anas Baig

Product Marketing Manager at Securiti

Muhammad Faisal Sattar

Sr. Manager, Product Legal & Global Data Compliance

FIP, CIPT, CIPM, CIPP/Asia

Published November 3, 2023 / Updated December 12, 2023

Listen to the content

On September 21, 2023, New Zealand's Office of the Privacy Commissioner (OPC) published guidance on Artificial Intelligence and the Information Privacy Principles (IPPs). This guidance expands upon the OPC’s initial set of expectations around AI use, published on May 25, 2023. The purpose of the guidance is to assist New Zealanders who use AI tools in complying with the Privacy Act 2020 as it relates to the usage of AI.

The guidance explains how AI tools function, provides real-world examples and sets out several questions to think about concerning privacy concerns. It also explains how AI relates to the 13 IPPs in the Privacy Act. The Privacy Act applies when collecting, using, or sharing personal information and using AI tools.

According to the Privacy Commissioner, AI tools pose unique privacy challenges as AI enables unique ways of collecting and combining personal information, making it more challenging to see, comprehend, and justify using personal information. Organizations should exercise extra caution when feeding personal information into AI technologies since it’s still unclear how these algorithms produce a specific result.

Personal information under the Privacy Act includes information like a person's name, address, contact details, or photographs. It can also include technical metadata like map coordinates, Internet protocol addresses, or device identifiers related to a person. It also includes information about a person that is inaccurate or made up, including fake social profiles and deepfake images.

Overview of the Updated Guidance

The guidance takes a broad approach to AI systems and their potential privacy impacts as AI is still a developing area, where experts disagree on how capable current systems are and how this will develop over time.

The guidance suggests that AI users in New Zealand should align with IPPs at every stage of the AI process. The guidelines specifically state that organizations utilizing AI tools should:

  • Realize that privacy is a starting point for responsible use of AI tools, and the best time to start privacy work is at the beginning.
  • Think carefully about the use-case before relying on exciting new tools to solve it, and be confident that you understand potential privacy risks.
  • Have senior leadership approval based on full consideration of risks and mitigations.
  • Review whether a generative AI tool is necessary and proportionate given potential privacy impacts, and consider whether you could take a different approach.
  • Conduct a privacy impact assessment before using these tools, including what data sources they were trained on and how relevant and reliable they are for your purposes.
  • Be transparent, telling people how, when, and why the tool is being used.
  • Consider Māori perspectives and engage with them about potential risks and impacts to the taonga of their information.
  • Develop procedures about accuracy and access by individuals to their information.
  • Ensure human review before acting on AI outputs to reduce risks of inaccuracy and bias.
  • Ensure that the AI tool does not retain or disclose personal information.

Understanding the potential risks will enable you to use privacy policies to govern your AI tools and ensure privacy statements set clear expectations.

Consider the IPPs When Using AI Tools

The 13 IPPs, which govern how agencies must handle personal information, are the fundamental component of the Privacy Act. The IPPs govern the activities of collecting, using, and sharing personal information.

The IPPs apply whether you're developing your own AI tools, utilize AI tools to aid in decision-making, or have team members who utilize AI informally at work. They also apply when overseas organizations give AI tools to New Zealanders. You have to take your privacy obligations into account in each situation.

The IPPs provide guidelines for handling personal data, including how to collect it (IPPs1-4), use and protect it (IPPs5–10), and share it (IPPs 11–12). There are also specific requirements for unique identifiers (IPP13). Key questions to ask include:

Is the training data behind an AI tool relevant, reliable, and ethical?

AI tools reproduce patterns from their training data. Agencies are generally required to gather personal information directly from the individual it concerns (IPP2) and to disclose all information they collect, including how it will be used (IPP3). Additionally, agencies have to ensure that personal information obtained is fair and legal and does not unnecessarily pry into private matters, especially when it comes to obtaining data from minors (IPP4).

AI tools replicate patterns observed in their training data. Therefore, an organization cannot know whether a tool incorporates personal information obtained in a way that violates IPPs 1–4 unless it has an in-depth understanding of the training data and the design methods used to create it. Additionally, any gaps or biases may limit accuracy (IPP8).

Organizations must specify why they collect personal data and only use and disclose it for those reasons (IPPs 10 and 11). This implies that organizations must carefully assess the reasons behind their information collection needs and ensure they only obtain the information necessary to meet those requirements.

An organization must disclose at the time of data collection if it intends to use personal information for AI tool training. Training data is the foundation of AI tools, and if an organization is offering a service, like a chatbot or a phone line, it needs to explicitly inform users about this and provide them an option to opt out of having their information used for these purposes.

Additionally, organizations need to be confident they are using personal information in ways that fit the purpose for which it was collected. Reusing information for training may go against this (IPP10).

How are you keeping track of the information you collect and use with AI tools?

A person has the right to request access to and correct any information held about them by an agency (IPP6 and IPP7). Prior to implementing an AI tool, the Commissioner asserts that organizations must establish processes for handling requests from individuals requesting access to and correcting their personal data. Prior to using an AI tool during the procurement phase, you may want to think about the following:

  • Are you confident you can provide information about a person to them if they ask for it?
  • Are you confident that you can correct personal information?
  • How often are models you rely on updated? Can you correct AI outputs in a timely way?
  • How will you verify the identity of an individual requesting their information?

AI capabilities also make it easier for people to mimic other people realistically. Thus, organizations must be extra cautious when confirming the identity of someone requesting sensitive information.

How are you testing that AI tools are accurate and fair for your intended purpose? Are you talking with people and communities with an interest in these issues?

Agencies that possess personal information are required under IPP8 to take reasonable measures to verify that the data is accurate, updated, complete, relevant, and not misleading before using or disclosing it. This prompts the question, what "reasonable steps" can organizations take to ensure that AI technologies will adhere to the concept of accuracy?

An organization should conduct privacy impact assessments and evaluate every stage of an AI tool's lifecycle, which might involve examining the training process that developed it and engaging with the community, including Māori, to understand and uphold fairness and accuracy. For instance, discussing Māori about the potential risks and impacts to the taonga of their information.

What are you doing to track and manage new risks to information from AI tools?

Organizations must safeguard personal information, prompts, and training data from theft, unauthorized access, and other misuse (IPP5). This includes using cybersecurity measures, such as two-factor authentication.

As a result, organizations will have to decide if they can utilize AI tools without sharing back data or if they can rely on contractual clauses that prevent the provider from using the input data for training. Additionally, they will require privacy breach response strategies addressing the possible risks of using AI tools.

In Conclusion

The Commissioner suggests that the safest course of action is to avoid putting personal information into an AI tool if an organization is unsure about it and ensure that everyone in the organization complies.

We all rely on individuals and organizations accepting accountability for their actions within the larger framework, necessitating the importance of being proactive about privacy to better control risk and use AI tools more efficiently. Additionally, organizations should ensure that the training data is obtained and handled in a manner that complies with data privacy regulations, AI laws, and ethical standards.

Analyze this article with AI

Prompts open in third-party AI tools.
Join Our Newsletter

Get all the latest information, law updates and more delivered to your inbox



More Stories that May Interest You
Videos
View More
Rehan Jalil, Veeam on Agent Commander : theCUBE + NYSE Wired: Cyber Security Leaders
Following Veeam’s acquisition of Securiti, the launch of Agent Commander marks an important step toward helping enterprises adopt AI agents with greater confidence. In...
View More
Mitigating OWASP Top 10 for LLM Applications 2025
Generative AI (GenAI) has transformed how enterprises operate, scale, and grow. There’s an AI application for every purpose, from increasing employee productivity to streamlining...
View More
Top 6 DSPM Use Cases
With the advent of Generative AI (GenAI), data has become more dynamic. New data is generated faster than ever, transmitted to various systems, applications,...
View More
Colorado Privacy Act (CPA)
What is the Colorado Privacy Act? The CPA is a comprehensive privacy law signed on July 7, 2021. It established new standards for personal...
View More
Securiti for Copilot in SaaS
Accelerate Copilot Adoption Securely & Confidently Organizations are eager to adopt Microsoft 365 Copilot for increased productivity and efficiency. However, security concerns like data...
View More
Top 10 Considerations for Safely Using Unstructured Data with GenAI
A staggering 90% of an organization's data is unstructured. This data is rapidly being used to fuel GenAI applications like chatbots and AI search....
View More
Gencore AI: Building Safe, Enterprise-grade AI Systems in Minutes
As enterprises adopt generative AI, data and AI teams face numerous hurdles: securely connecting unstructured and structured data sources, maintaining proper controls and governance,...
View More
Navigating CPRA: Key Insights for Businesses
What is CPRA? The California Privacy Rights Act (CPRA) is California's state legislation aimed at protecting residents' digital privacy. It became effective on January...
View More
Navigating the Shift: Transitioning to PCI DSS v4.0
What is PCI DSS? PCI DSS (Payment Card Industry Data Security Standard) is a set of security standards to ensure safe processing, storage, and...
View More
Securing Data+AI : Playbook for Trust, Risk, and Security Management (TRiSM)
AI's growing security risks have 48% of global CISOs alarmed. Join this keynote to learn about a practical playbook for enabling AI Trust, Risk,...

Spotlight Talks

Spotlight 00:00
AI Agents Unleashed? The Future of Identity & Data Access Security
Watch Now View
Spotlight 00:00
IDC Research: Why Data, AI, Identity and Access Security Must Come Together
Watch Now View
Spotlight 59:11
Data Controls for AI: Findings from the 2026 GigaOm DSPM Research
Watch Now View
Spotlight 1:02:06
Consent by proxy: When AI agents start deciding for us
Watch Now View
Spotlight 1:00:41
Future-Proofing for the Privacy Professional
Watch Now View
Spotlight 50:52
From Data to Deployment: Safeguarding Enterprise AI with Security and Governance
Watch Now View
Spotlight 11:29
Not Hype — Dye & Durham’s Analytics Head Shows What AI at Work Really Looks Like
Not Hype — Dye & Durham’s Analytics Head Shows What AI at Work Really Looks Like
Watch Now View
Spotlight 11:18
Rewiring Real Estate Finance — How Walker & Dunlop Is Giving Its $135B Portfolio a Data-First Refresh
Watch Now View
Spotlight
Choosing the Right DSPM: An Industry Analyst’s Perspective
Watch Now View
Spotlight 13:38
Accelerating Miracles — How Sanofi is Embedding AI to Significantly Reduce Drug Development Timelines
Sanofi Thumbnail
Watch Now View
Latest
KSA PDPL in 2026 View More
KSA PDPL in 2026: What Organizations Need to Know Now
KSA's Personal Data Protection Law (PDPL) has been fully enforceable since 14th September 2024. Over the past two years, organizations have been navigating overlapping...
EU Digital Sovereignty View More
EU Digital Sovereignty is Moving from Policy to Procurement: What Organizations Need to Know
Digital sovereignty in Europe is moving from policy ambition toward something much more concrete: a framework that can determine how cloud services are assessed,...
View More
What Is Unstructured Data Discovery? A Practical Guide
Unstructured data discovery finds sensitive data in files, chats, and cloud storage. Securiti explains how it works, common challenges, and best practices.
Data Anonymization View More
What Is Data Anonymization
What is data anonymization? Securiti’s latest blog explains how it works, key techniques, and how it differs from pseudonymization. Read to learn more.
Saudi PDPL Compliance Toolkit View More
Saudi PDPL Compliance Toolkit
Turn Saudi PDPL requirements into action with practical checklists, a maturity self-assessment, and a 90-day plan for privacy and compliance leaders.
From Compliance to Continuous Assurance View More
From Compliance to Continuous Assurance: The Next Era of Saudi Data Protection
PDPL compliance didn’t end with the deadline. Learn how to comply with SDAIA enforcement requirements, govern cross-border transfers, and prepare your data for AI.
Data Minimization, Data Minimization for AI, Safe AI, AI Readiness, ROT Data, AI Governance, Data Governance, Data and AI Security, AI Risk Management, Sensitive Data View More
Minimize Before You Scale
Discover why redundant, obsolete, and trivial (ROT) data raises AI-era risks and how to minimize them. Explore practical fixes and business outcomes.
The PDPL Operational Lifecycle View More
The PDPL Operational Lifecycle
Find it. Fix it. Prove it. Repeat. Explore the 10-stage PDPL operational lifecycle, key deadlines, and the evidence regulators expect.
View More
Take the Data Risk Out of AI
Learn how to prepare enterprise data for safe Gemini Enterprise adoption with upstream governance, sensitive data discovery, and pre-index policy controls.
View More
Navigating HITRUST: A Guide to Certification
Securiti's eBook is a practical guide to HITRUST certification, covering everything from choosing i1 vs r2 and scope systems to managing CAPs & planning...
What's
New