Shipping AI to users is different from using it in your editor. This lesson covers the minimum bar for responsible developer-built features.
Data you should not send blindly
Assume prompts may be logged or used for quality review depending on vendor terms and your settings:
- Passwords, API keys, OAuth tokens, session cookies.
- Government IDs, health records, and financial account numbers.
- Unredacted production stack traces with internal hostnames.
Minimize — redact, tokenize, or summarize before the model sees payloads.
Prompt injection
If a user-controlled string reaches the model and influences actions (tools, SQL, shell), an attacker can embed instructions like "ignore prior rules."
Mitigations:
- Separate instructions from user content with clear delimiters; still not foolproof but helps auditing.
- Allow-lists of tools; never pass raw SQL or shell from model output without validation.
- Human confirmation for irreversible or high-risk operations.
Transparency
Tell users they are interacting with AI, especially if outputs affect decisions (hiring, credit, moderation). Link to how data is used and how to report mistakes.
Outputs are not promises
Disclaimers in UI matter less than guardrails: validate structured output, run server-side checks, and surface confidence honestly when you cannot verify.
Key takeaways
- Treat prompts as sensitive as HTTP bodies going to a third party — because they are.
- Assume malicious text in user input; design tool access accordingly.
- Pair model outputs with automated checks and clear UX about limitations.