Common Prompt Engineering Mistakes and How to Fix Them

Mistake 1: Vague Task Descriptions

The problem: Instructions like "analyze this email" or "help with this customer" give the model no clear success criteria. The model interprets the task differently each time, producing inconsistent output format, depth, and focus.

The fix: Replace vague verbs with specific actions. "Analyze this email" becomes "Extract the customer's primary request in one sentence, identify the product area affected, and recommend which team should handle it." Every word in the task description should be concrete enough that two people reading it would produce the same output structure.

Test: If you cannot objectively score the output as correct or incorrect, your task description is too vague.

Mistake 2: No Output Format Specification

The problem: Without a defined output format, the model chooses its own structure each time. One request might return a paragraph, the next a bullet list, the next a JSON object. Downstream systems cannot parse inconsistent formats.

The fix: Always specify the exact output format. For programmatic consumption: "Return ONLY valid JSON with fields: {category: string, priority: integer 1-5, summary: string max 30 words}. No text before or after the JSON." For human consumption: "Respond in 2-3 sentences. First sentence states the answer. Second sentence provides evidence. Third sentence (optional) gives context."

Test: Run your prompt 10 times on the same input. If the output format varies, you need stricter format specification.

Mistake 3: Missing Failure Mode Instructions

The problem: When the model encounters an input it cannot handle (ambiguous, off-topic, missing information), it guesses rather than acknowledging uncertainty. These guesses look confident to downstream systems and cause silent failures.

The fix: Explicitly define what to do when uncertain. "If the email does not clearly state a request, return {category: 'unclear', needs_clarification: true, suggested_question: 'What specific issue can we help you with?'}." For every category of failure (missing data, ambiguous input, out-of-scope request), provide explicit fallback behavior.

Test: Feed your prompt intentionally ambiguous or out-of-scope inputs. If it does not fail gracefully, add failure mode instructions.

Mistake 4: Contradictory Rules

The problem: "Always answer the customer's question" conflicts with "Never discuss topics outside your domain." "Be concise" conflicts with "Include all relevant details." The model resolves contradictions unpredictably, sometimes following one rule and sometimes the other.

The fix: Review your prompt for conflicting instructions and resolve them with explicit priority rules. "If the customer asks an off-topic question, prioritize staying in domain over answering their question. Redirect them politely." Establish a clear hierarchy: safety rules override helpfulness rules, which override format rules.

Test: Write inputs that trigger both sides of a potential conflict and check that the model consistently picks the correct priority.

Mistake 5: Testing Only on Easy Examples

The problem: The prompt works perfectly on 5 hand-picked examples during development but fails on real-world inputs that are longer, messier, more ambiguous, or contain edge cases the developer did not anticipate.

The fix: Build a test set from REAL production data (anonymized if needed). Include: the easiest 20% (sanity check), the average 50% (baseline accuracy), the hardest 20% (edge cases), and intentionally adversarial 10% (trying to break the prompt). Your accuracy target should be measured against this representative sample, not against cherry-picked examples. See How to Test and Iterate on AI Prompts.

Mistake 6: Over-Constraining the Response

The problem: So many rules and restrictions that the model produces robotic, unnatural output. Every sentence follows the same pattern because creativity is eliminated by over-specification. Common in customer service chatbot prompts where the desire for safety leads to responses that sound obviously automated.

The fix: Distinguish between hard constraints (rules that must never be violated: policy accuracy, topic boundaries, safety) and soft guidance (preferences that allow variation: sentence structure, vocabulary choices, greeting styles). Hard constraints use words like "never" and "always." Soft guidance uses words like "prefer" and "typically." This gives the model room to sound natural within safe boundaries.

Mistake 7: Ignoring Token Costs at Scale

The problem: A 2,000-word system prompt that includes extensive examples, lengthy explanations, and redundant instructions. Works fine in testing. At 10,000 requests per day, it costs $150+ daily just for the system prompt input tokens, and much of that content is not improving output quality.

The fix: After your prompt reaches target accuracy, run an ablation test: remove sections one at a time and re-test. If removing a section does not reduce accuracy, it was consuming tokens without providing value. Most production prompts can be trimmed 30-50% after initial development without accuracy loss. Also use prompt caching (available in Claude and GPT APIs) to reduce repeated processing of static system prompt portions.

Mistake 8: Not Handling Multi-Issue Inputs

The problem: A customer email mentions a billing error AND asks how to use a new feature. The prompt extracts one issue and ignores the other. Or the classification picks one category when both apply.

The fix: Explicitly address multi-issue inputs: "If the text contains multiple distinct issues, identify the PRIMARY issue (the one they seem most concerned about or mention first) and flag multi_issue: true with a brief note about the secondary issue. For routing purposes, use the primary issue category." Alternatively, if your system can handle multiple issues, return an array of issues rather than a single classification.

Mistake 9: Examples That Contradict Instructions

The problem: Your few-shot examples demonstrate one behavior while your written instructions describe another. The model follows the examples because they are concrete demonstrations, ignoring the abstract instruction. For instance, your instructions say "respond formally" but your examples use casual language.

The fix: Audit every example against every instruction. Each example should be a perfect demonstration of all your rules simultaneously. If you change an instruction, update every example that it affects. Consider examples as the "ground truth" of your prompt, because that is how the model treats them.

Mistake 10: No Versioning or Change Tracking

The problem: You iterate on a prompt, improving one aspect, then a week later discover something that used to work now fails. Without version history, you cannot identify which change caused the regression or revert to a known-good state.

The fix: Store prompts in version control (git) or a prompt management system. Every change includes: what was changed, why, what test results showed before and after. When a regression appears, you can bisect through versions to find the breaking change. Treat prompts with the same rigor as source code because they have the same impact on system behavior.

Mistake 11: Assuming One Prompt Fits All Models

The problem: A prompt optimized for Claude is deployed on GPT (or vice versa) without testing. Different models respond differently to the same instructions. Formatting cues, level of detail needed, and instruction-following reliability vary between model families and even between versions of the same model.

The fix: Test every prompt on the specific model version it will run on in production. When switching models (for cost or performance reasons), re-run your full test set. Keep model-specific notes: "Claude follows formatting instructions more literally; GPT-4 needs fewer examples but benefits from more explicit constraints." Budget time for prompt adjustment when switching models. See How to Switch Between AI Models.

Mistake 12: Not Measuring What Matters

The problem: Measuring overall accuracy (85%!) without breaking it down by category. The overall number hides that the prompt is 99% accurate on easy cases and 40% accurate on the hard cases that actually matter. Or measuring accuracy without measuring other dimensions that affect production quality: response time, format compliance, tone appropriateness.

The fix: Define success metrics that match your actual business requirements. For each metric, set a minimum threshold. For customer service: accuracy per category, escalation rate (should not be too high or too low), response length consistency, policy compliance rate. For data extraction: field-level accuracy (not just overall), null accuracy (correctly identifying when data is absent), format compliance rate. Report metrics per category and per difficulty level, not just aggregate.