KHAGAN AYDIN
Writing

Before You Paste Company Data into an AI Chatbot

What happens to the text you type, what has already gone wrong, and a policy a team can follow from Monday.

· 6 min read

Every company now has employees using AI chatbots at work, whether or not there is a policy. They paste in code to find a bug, a contract to summarize, a spreadsheet to explain. Most of them have never asked where that text goes.

This spring gave us several clear answers.

What has already happened

Samsung, March to May 2023. Within about three weeks of allowing engineers to use ChatGPT, Samsung recorded three incidents. One engineer pasted source code to find a bug. Another pasted code to optimize a test process. A third pasted the transcript of a meeting to turn it into minutes. All of that text left the company and sat on another company’s servers. In early May, Samsung restricted the use of generative AI tools on company devices.

OpenAI, March 2023. A bug in an open source library used by ChatGPT showed some users the titles of other users’ conversations. OpenAI also reported that, during a nine-hour window, payment details of about 1.2% of ChatGPT Plus subscribers active at the time could have been visible to others, including name, email, billing address and the last four digits of the card. The lesson is not that one vendor is careless. Anything you send to a service lives in that service, with that service’s bugs.

Italy, March and April 2023. On March 31, Italy’s data protection authority ordered OpenAI to stop processing the data of people in Italy, and OpenAI blocked ChatGPT there. The authority cited the lack of a legal basis for using personal data for training, the lack of age checks and poor notice to users. Access returned on April 28, after OpenAI added privacy controls and disclosures. Regulators are watching how personal data enters these systems, and that includes data your employees paste in.

Where your text goes

The rules differ by product and change often, so read the current terms of any tool you approve. As of this month, for OpenAI:

  • ChatGPT, the consumer app. Conversations may be used to improve the models unless the user turns off chat history, an option added on April 25. With history off, conversations are still kept for 30 days for abuse monitoring, then deleted.
  • The OpenAI API. Since March 1, data sent through the API is not used to train models unless the customer opts in. It is kept for up to 30 days for abuse monitoring.

In both cases, the text leaves your network and is stored by a third party for some period. For many kinds of data, that alone breaks a contract, a regulation or a promise you made to a customer.

Sort the data first

Most teams do not need a ban. They need a clear line between what can go in and what cannot.

Never paste

  • Customer or employee personal data: names with account details, health, financial or identity information
  • Passwords, API keys, tokens, private keys and connection strings
  • Proprietary source code, or code covered by a client contract
  • Unreleased financial results, deals, pricing and plans
  • Legal matters, investigations and HR cases
  • Security findings, penetration test reports and incident details

Paste with care, after removing identifying details:

  • Internal documents that are not confidential
  • Code snippets you have reduced to the generic problem
  • Data you have replaced with placeholders

Fine to paste

  • Public information
  • Your own writing that contains no confidential facts
  • General questions about tools, languages and concepts

Practical steps

1. Give people an approved option. A ban without an alternative moves the same work to personal phones and personal accounts, where you see nothing. An internal tool built on the API, or a business plan with clear data terms, keeps the work where you can manage it.

2. Turn off training where you can. For the consumer app, that means chat history off. For the API, the default already excludes training. Write down which setting each approved tool uses.

3. Remove identifying details before sending. Replace real names, account numbers and hostnames with placeholders. The model rarely needs the real value to help.

Before: Customer Maria Lopez (acct 448812) says her card ending 4421
        was charged twice by our billing service at billing-prod-02.

After:  Customer [NAME] (acct [ID]) says her card ending [CARD] was
        charged twice by our billing service at [HOST].

4. Put a safety net in your internal tool. If you build your own wrapper around the API, check the text before it leaves your network. Pattern matching will not catch everything, but it catches the most common and most damaging mistakes.

import re

PATTERNS = {
    "EMAIL": r"[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}",
    "CARD": r"\b(?:\d[ -]?){13,16}\b",
    "SSN": r"\b\d{3}-\d{2}-\d{4}\b",
    "AWS_KEY": r"\bAKIA[0-9A-Z]{16}\b",
    "PRIVATE_KEY": r"-----BEGIN [A-Z ]*PRIVATE KEY-----",
}

def redact(text):
    found = []
    for label, pattern in PATTERNS.items():
        if re.search(pattern, text):
            found.append(label)
            text = re.sub(pattern, "[" + label + "]", text)
    return text, found

clean_text, found = redact(user_text)
if "PRIVATE_KEY" in found or "AWS_KEY" in found:
    raise ValueError("Secrets detected. Remove them before sending.")

Redact personal data automatically, and refuse outright when a secret appears. A key that reached a third party must be rotated, even if nothing bad seems to have happened.

5. Log what goes out. An internal tool should record who sent what, when and to which service. If something leaks, you will need that record.

6. Teach with examples, not rules alone. Show people a real-looking example of each “never paste” category. Most leaks are not careless people. They are busy people who did not recognize that the text was sensitive.

Questions to ask a vendor before you approve a tool

Before a tool goes on the approved list, get written answers to these:

  • Is our data used to train or improve your models? Is that off by default, and can we confirm it in the contract?
  • How long do you keep prompts and answers, and can we request deletion?
  • In which countries is our data stored and processed?
  • Which other companies process our data on your behalf?
  • Do you have an independent security report, such as SOC 2 Type II, that we can read?
  • Can we use our own single sign-on, and can we remove access when someone leaves?
  • Do we get logs of who used the tool and when?
  • How and how quickly will you tell us about a breach that affects our data?

A vendor that cannot answer these clearly is telling you something important.

A one-page policy you can adapt

AI assistant use policy

1. Approved tools: [list]. Do not use personal accounts for work data.
2. Never enter: personal data, credentials or keys, client or
   proprietary code, unreleased financials, legal or HR matters,
   security findings.
3. Replace names, account numbers and hostnames with placeholders
   before sending anything else.
4. Treat every answer as a draft. You are responsible for checking
   it before it is used.
5. If you entered something you should not have, tell [security
   contact] the same day. Early reports are not penalized.
6. Owner: [name]. Reviewed every 6 months.

Point five matters more than it looks. People who fear punishment hide mistakes, and a hidden leak of a key or a customer list does far more damage than one reported in the first hour.

AI assistants are useful enough that people will keep using them. The job is not to stop that. It is to make sure that what leaves the company is what the company is willing to share.