AI security: don't leak keys, user data or code to AI tools
AI security when coding: never paste keys, user data or company code into a chat box. A secret-hiding filter, ways to block leaks and what to do after one.
AI security starts with one question before every paste: does this snippet hold a secret? Three things need protecting: keys and passwords, the data of other people, and code that does not belong to you. Was your last paste clean? Let's check.
What is scary about this log snippet?
In the previous post, I used an AI to review code while still checking every comment myself. That post ended on a small habit that can do big damage: when in a hurry, people paste a whole piece of log or a config file into the chat box and forget that it contains something that should never leave the machine.
I rebuilt a situation so you can see it. The Telegram bot of the expense tracker app reports an error, and you want to ask an AI to take a look. The log below is one I built to illustrate, and every key, password, name and phone number in it is fake. Try reading it like someone who is about to press "paste":
[2026-11-13 21:04:11] ERROR sendMessage failed
FetchError: request to https://api.telegram.org/bot123456789:AAHxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/sendMessage failed, reason: connect ETIMEDOUT
at ClientRequest.<anonymous> (node_modules/node-fetch/lib/index.js:1491:11)
Running config: BOT_TOKEN=123456789:AAHxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx DATABASE_URL=postgres://moc_admin:Fake-Password-12345@db.example.com:5432/expenses
User: Nguyen Van An (0901 000 001, an.nguyen@example.com) sent "lunch 45k"
Running commit: 3f2a9c1 (2026-11-13), order 1041In those six lines there are three things that should not go anywhere. The token of the bot appears twice, once in the request address and once in the config line. The reason it sits in the address is that the Telegram Bot API requires every call to have the form https://api.telegram.org/bot<token>/<method name>, and many HTTP libraries print the whole address into the error message.
The second is the database connection string, which holds both the user name and the password. The third is the name, phone number and email of one user. The last line is harmless: a commit ID, a date and an order ID.
What is worth thinking about is that nobody leaked anything on purpose. The config line is in the log because the bot prints it at startup, and the person pasting only meant to take "the whole error, for enough context". That very phrase, "for enough context", is where things often go wrong.
Three things not to give to an AI, and why
The first is keys and passwords: API keys, tokens, database connection strings, service-role keys. Once it is in the chat box, you no longer control it. It passes through the servers of a third party, it may sit in the history of the conversation, and the AI may copy it into the code it suggests. An AI also sometimes suggests writing the key straight into the code "to try it out", and then you forget to remove it. In my view that is even more dangerous than pasting, because then the key goes straight into Git.
The second is the data of other people: names, phone numbers, emails, the spending history of real users. This is no longer your own business, because they did not agree to have their data go to a tool they do not know about. When you need an AI to help with debugging, replace it with fake data of the same shape (a person called "A", the number 0900000000).
The third is code and documents that do not belong to you: the code of your company, of a client, or anything under a confidentiality agreement. Every AI tool and every plan has its own policy about storing and using the data you send, and a personal plan is often different from a business plan. I cannot speak for each provider, so what you should do is read the data terms of exactly the plan you use, and ask your company whether there is a list of approved AI tools before you paste code into any tool.
Hide the secrets before you paste
The way I do it is a small filter that runs on the text before you paste it. It looks for things with familiar shapes and replaces them with a label such as [HIDDEN:bot-token]. The AI still sees the structure of the log, but not the values. You need Node.js 22.18 or newer, because from that version Node runs .ts files directly, and nothing else needs to be installed. The folder layout:
hide-secrets/
package.json
filter-before-pasting.ts
scan-folder.ts
src/secrets.ts
src/sample-log.ts
src/secrets.test.ts{
"name": "expense-ai-10",
"private": true,
"type": "module",
"scripts": { "test": "node --test src" }
}The file below holds the fake values. I assemble them from several pieces at run time instead of writing the whole string, because the secret scanner of GitHub will raise an alarm if it sees a whole string that looks like a real token in a code repository, even when it is fake:
// FAKE values to try the filter. Assembled from several pieces at run time, so this file does not contain a whole string that looks like a real key
// (the secret scanner of GitHub would raise an alarm if it did).
export const FAKE_BOT_TOKEN = `123456789${":"}AAH${"x".repeat(32)}`;
export const FAKE_DB_PASSWORD = `Fake${"-"}Password${"-"}12345`;
// A rebuilt log of a BOT ERROR, to illustrate. The HTTP library also prints the request address into the error message,
// and the address of the Telegram Bot API contains the token.
export const SAMPLE_LOG = `[2026-11-13 21:04:11] ERROR sendMessage failed
FetchError: request to https://api.telegram.org/bot${FAKE_BOT_TOKEN}/sendMessage failed, reason: connect ETIMEDOUT
at ClientRequest.<anonymous> (node_modules/node-fetch/lib/index.js:1491:11)
Running config: BOT_TOKEN=${FAKE_BOT_TOKEN} DATABASE_URL=postgres://moc_admin:${FAKE_DB_PASSWORD}@db.example.com:5432/expenses
User: Nguyen Van An (0901 000 001, an.nguyen@example.com) sent "lunch 45k"
Running commit: 3f2a9c1 (2026-11-13), order 1041`;Now the main part: seven rules, and each rule is a regex for one kind of secret. The function hideSecrets runs the rules one by one, replaces the matched spot with a label, and records which line and what kind. It only records the first four characters of the value, enough for you to recognize which one it is without printing the whole secret again:
// Find and hide what you should not paste into the chat box of an AI: keys, passwords and personal information.
// This is a NET for the shapes it already knows, not a shield: a secret with an unusual shape slips through.
export type Kind = "url-with-password" | "bot-token" | "aws-key" | "jwt" | "assigned-secret" | "email" | "phone-number";
export type Finding = { kind: Kind; line: number; preview: string };
// Sample values such as "<token>", "your_key", "changeme" (they often sit in .env.example), or a spot that is already hidden, are not real secrets.
const PLACEHOLDER = /^(<.*>|your[_-].*|changeme|x{3,}|\*{3,}|env\(.*|\[HIDDEN:.*)$/i;
// The order matters: the specific rules (bot-token, URL) run before the general rule ("token=...").
const RULES: { kind: Kind; pattern: RegExp }[] = [
{ kind: "url-with-password", pattern: /\b[a-z][a-z0-9+.-]*:\/\/[^\s:@/]+:[^\s@/]+@[^\s/]+/gi },
{ kind: "bot-token", pattern: /(?<!\d)\d{8,10}:[A-Za-z0-9_-]{35}(?![A-Za-z0-9_-])/g },
{ kind: "aws-key", pattern: /\bAKIA[0-9A-Z]{16}\b/g },
{ kind: "jwt", pattern: /\beyJ[\w-]{8,}\.eyJ[\w-]{8,}\.[\w-]{8,}/g },
{ kind: "assigned-secret", pattern: /\b\w*(?:api[_-]?key|[_-]key|secret|token|password)\w*\s*[:=]\s*["']?([^\s"',;]{8,})/gi },
{ kind: "email", pattern: /\b[\w.+-]+@[\w-]+(?:\.[\w-]+)+\b/g },
{ kind: "phone-number", pattern: /(?<!\d)(?:\+84|0)(?:[\s.]?\d){9}(?!\d)/g }, // the Vietnamese format: +84 or 0, then 9 digits
];
export function hideSecrets(text: string): { clean: string; findings: Finding[] } {
const findings: Finding[] = [];
let clean = text;
for (const { kind, pattern } of RULES) {
const before = clean;
clean = before.replace(pattern, (match: string, ...args: unknown[]) => {
const position = args[args.length - 2] as number; // the second-to-last parameter of the replace callback is the match position
const value = kind === "assigned-secret" ? (args[0] as string) : match;
if (PLACEHOLDER.test(value)) return match;
findings.push({ kind, line: before.slice(0, position).split("\n").length, preview: `${value.slice(0, 4)}…` });
return match.replace(value, `[HIDDEN:${kind}]`);
});
}
return { clean, findings: findings.sort((a, b) => a.line - b.line) };
}There are a few decisions worth explaining. The order of the rules matters: the specific rules (bot token, URL with a password) run before the general rule name=value, otherwise the line BOT_TOKEN=... would be hidden twice. The bot token is recognized by its usual shape (8 to 10 digits, a colon, 35 characters), and I use (?<!\d) instead of \b at the start because in the Telegram address the token is stuck to the word bot, so there is no word boundary. The variable PLACEHOLDER skips values such as your_token_here or changeme, because they often sit in the file .env.example, and if they were not skipped the filter would raise an alarm even for harmless files. The phone rule uses the Vietnamese format (+84 or 0 followed by nine digits) because that is the data of this series; for another country you would change that one line.
Run the filter on the sample log:
// Hide the secrets in a piece of text before you paste it to an AI. Run: node filter-before-pasting.ts [file]. With no file, it uses the sample log.
import { readFileSync } from "node:fs";
import { hideSecrets } from "./src/secrets.ts";
import { SAMPLE_LOG } from "./src/sample-log.ts";
const text = process.argv[2] ? readFileSync(process.argv[2], "utf8") : SAMPLE_LOG;
const { clean, findings } = hideSecrets(text);
console.log(`--- Text after hiding\n${clean}\n--- Hid ${findings.length} places`);
for (const f of findings) console.log(`line ${f.line}: ${f.kind} (starts with ${f.preview})`);node filter-before-pasting.ts--- Text after hiding
[2026-11-13 21:04:11] ERROR sendMessage failed
FetchError: request to https://api.telegram.org/bot[HIDDEN:bot-token]/sendMessage failed, reason: connect ETIMEDOUT
at ClientRequest.<anonymous> (node_modules/node-fetch/lib/index.js:1491:11)
Running config: BOT_TOKEN=[HIDDEN:bot-token] DATABASE_URL=[HIDDEN:url-with-password]/expenses
User: Nguyen Van An ([HIDDEN:phone-number], [HIDDEN:email]) sent "lunch 45k"
Running commit: 3f2a9c1 (2026-11-13), order 1041
--- Hid 5 places
line 2: bot-token (starts with 1234…)
line 4: url-with-password (starts with post…)
line 4: bot-token (starts with 1234…)
line 5: email (starts with an.n…)
line 5: phone-number (starts with 0901…)Compare with the original log. Both tokens, the connection string, the email and the phone number are hidden, while the commit ID 3f2a9c1, the date, the order ID 1041 and "45k" are intact, because they are not secrets and the AI needs them to understand the error. But you can see two things that are not hidden: the name "Nguyen Van An" and the server address db.example.com. The filter only recognizes shapes, and a person's name has no fixed shape. For those things, your own eyes are still the last filter.
Scan a folder, and block at the source
Hiding each snippet before pasting is manual work, and people often forget it exactly when they are in a hurry. So I add two more layers that run without you remembering. The first layer scans a whole folder to see whether any secret is still sitting in a file, and checks whether .env is excluded by .gitignore. The script builds a sample project in the temp folder by itself (with one deliberate mistake: the .env line is forgotten in .gitignore) so you can run it right away:
// Scan a folder for secrets that are still sitting in files, and check that .env is excluded by .gitignore.
// Run: node scan-folder.ts <folder>, or node scan-folder.ts --demo (it builds a sample project in the temp folder).
import { existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, statSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join, relative } from "node:path";
import { hideSecrets } from "./src/secrets.ts";
import { FAKE_BOT_TOKEN } from "./src/sample-log.ts";
const SKIP = new Set(["node_modules", ".git", ".next"]);
const isEnvFile = (name: string) => /^\.env(\..+)?$/.test(name) && name !== ".env.example";
function* walk(folder: string): Generator<string> {
for (const name of readdirSync(folder)) {
if (SKIP.has(name)) continue;
const path = join(folder, name);
if (statSync(path).isDirectory()) yield* walk(path);
else yield path;
}
}
function buildSampleProject(): string {
const root = mkdtempSync(join(tmpdir(), "sample-project-"));
mkdirSync(join(root, "src"));
writeFileSync(join(root, ".env"), `BOT_TOKEN=${FAKE_BOT_TOKEN}\n`);
writeFileSync(join(root, ".env.example"), "BOT_TOKEN=your_token_here\n");
writeFileSync(join(root, ".gitignore"), "node_modules/\n"); // the .env line is forgotten: a common mistake
writeFileSync(join(root, "src", "run-bot.ts"), `// Contact: an.nguyen@example.com\nconst token = "${FAKE_BOT_TOKEN}"; // pasted temporarily to try it\n`);
return root;
}
const folder = process.argv[2] === "--demo" ? buildSampleProject() : process.argv[2];
if (!folder) throw new Error("Usage: node scan-folder.ts <folder> | --demo");
let problems = 0;
for (const path of walk(folder)) {
const name = path.split(/[\\/]/).pop()!;
if (isEnvFile(name)) continue; // a .env file may hold secrets; it only must not reach Git
for (const f of hideSecrets(readFileSync(path, "utf8")).findings) {
problems++;
console.log(`${relative(folder, path)}:${f.line} ${f.kind} (starts with ${f.preview})`);
}
}
const gitignore = existsSync(join(folder, ".gitignore")) ? readFileSync(join(folder, ".gitignore"), "utf8") : "";
if (existsSync(join(folder, ".env")) && !/^\.env\b/m.test(gitignore)) {
problems++;
console.log("WARNING: there is a .env file but .gitignore does not exclude it, so git add . would put the token into the repo");
}
console.log(problems === 0 ? "Nothing suspicious found (but this is only a net)." : `${problems} places need attention.`);node scan-folder.ts --demosrc\run-bot.ts:1 email (starts with an.n…)
src\run-bot.ts:2 bot-token (starts with 1234…)
WARNING: there is a .env file but .gitignore does not exclude it, so git add . would put the token into the repo
3 places need attention.Those three places are exactly the three things I put in on purpose: an email in a comment, a token written straight into run-bot.ts ("pasted temporarily to try it"), and a .env file that is not excluded. On Linux or macOS the path will have a / instead of a \. The file .env.example that holds your_token_here is not reported, and the content of the .env file is not scanned, because it is allowed to hold secrets, it only must not go into Git.
The Tech Notes blog project does something similar with the command npm run verify:governance. That script runs over every file that Git would commit and reports an error if there is a real .env file or a line that matches this expression (I copied it word for word from its code):
const secretPattern = /(?:api[_-]?key|secret|token|password)\s*[:=]\s*["'](?!env\()[^"'\s]{12,}["']/i;The part (?!env\() is an exception with a reason: the file supabase/config.toml has values of the form env(NAME), which are references to environment variables and not secrets, and without that exception the check would report false errors. But when I tried this expression on the line SUPABASE_SERVICE_ROLE_KEY="..." (with a fake value), it did not catch it, because the variable name does not contain api_key, secret, token or password. My filter above also missed exactly that spot the first time I wrote it, and only a test showed me; the version you see above has [_-]key added to the list of names. The lesson: whatever filter you write, try it with the real variable names of your own project.
The second layer is to keep an AI tool from being able to read secret files from the start. For Claude Code, the settings documentation has a permissions section with a deny list, and the example there blocks reading the .env file. In the file .claude/settings.json of the project, that line looks like this:
{
"permissions": {
"deny": ["Read(./.env)", "Read(./.env.*)"]
}
}I should state the limit plainly: while researching, I saw some bug reports that users sent to GitHub saying that this blocking rule is not always applied (for example with an absolute path, or when the JSON file has a syntax error so the rule is skipped), and I have not verified that. So I treat it as one layer among many, not the only layer. You should test it by asking the tool to read the .env file and seeing whether it refuses. The last layer is GitHub itself: the documentation on push protection says that when you push code containing a kind of secret that they recognize, GitHub blocks that push.
What do you do after a leak?
Suppose the bot token is already in the chat box, or already in Git. The first step is not to erase the traces but to make that token useless. The order below matters:
- Revoke the token at once. For a Telegram bot, according to the guides I read, you open @BotFather, choose
/mybots, choose the bot, go to the API Token item and click Revoke current token. I do not have a real bot to try this in the post, so follow the current instructions of Telegram. - Create a new token and update the places that use it (environment variables on the server, on Vercel, in CI). Store it in the proper place for secrets, not in the code.
- Look at the usage history to see if there was any strange activity during the time the token was exposed.
- Only after that clean up the Git history, if the secret was committed.
Revoking must come first, because deleting that line from Git is not enough: the old copy stays in the history, in the clones and forks, and in any place where the string was copied. Only when the token is revoked do those copies lose their value. GitHub has a separate page on the risks of a leaked secret if you want to read more.
Tests for the filter, and what it cannot do
I wrote seven tests for hideSecrets (in the demo repo): the sample log gives exactly five findings and no original value is left after hiding, ordinary things (a commit ID, a date, an order ID, an amount, a phone number with too few or too many digits) are not hidden by mistake, the sample values in .env.example are not reported, three ways of writing a phone number, a sample AWS key and a JWT, and the fact that SUPABASE_SERVICE_ROLE_KEY is hidden after the patch. The last test records one known limit: a secret with an unusual shape and no hinting name slips through.
node --test src/secrets.test.ts# tests 7
# pass 7
# fail 0Understand this filter correctly. It helps when you are in a hurry, and it does not help when you do it on purpose or when the secret has an unusual shape. It does not know that "Nguyen Van An" is a real person, and it does not know that the code you are about to paste belongs to your company. The best protection is still the habit: do not put secrets in places where they are easy to copy, which means keeping keys in environment variables and not writing them into code files, and always think about where this snippet will go before you paste.
What if the app itself has an AI feature?
So far the AI has only been my assistant when writing code. But the expense tracker app can use AI by itself: a user messages "lunch 45k", and the bot classifies the expense. Then the data of real users goes to an AI service every time they send a message, and everything above becomes a question about your app and not only about how you work. That is for the later posts.
You can follow the whole journey in Build a side project with AI. If you want to recall how to give the right context to an AI when debugging, the post on debugging with AI explains it, and the questions about using AI while programming are in the interview question bank.