12KB
Reverse Engineer a Gold Standard
Version: 1.0.0 Status: ACTIVE
Purpose
Extract reusable structure, decision logic, constraints, and quality patterns from an existing high-quality artifact.
The goal is not merely to imitate the artifact.
The goal is to discover:
WHY DOES THIS WORK?
and convert that into something reusable.
Possible outputs include:
SKILL
PROMPT
TEMPLATE
CHECKLIST
REVIEW RUBRIC
CODING CONVENTION
WORKFLOW
1. Trigger Conditions
Use this skill when the user provides or identifies:
- an excellent document
- a strong code implementation
- a good report
- a successful workflow
- an effective prompt
- a high-quality design
- a useful script
- a previous output they want repeated consistently
- a reference implementation
Use this skill when the request is conceptually:
"Make future outputs work like this."
2. Core Principle
Do not merely copy:
CONTENT
Extract:
STRUCTURE
DECISION LOGIC
CONSTRAINTS
SEQUENCE
QUALITY SIGNALS
The reusable value is usually beneath the surface.
3. Four-Layer Analysis
Analyze the gold standard through four layers:
1. ROLE AND CONTEXT
2. STRUCTURAL LOGIC
3. BOUNDARIES
4. REUSABLE SCAFFOLD
4. Layer 1 — Role and Context
Identify what expertise and environment shaped the artifact.
Ask:
Who appears to be producing this?
Who consumes it?
What problem does it solve?
What assumptions does it make?
What domain knowledge does it rely on?
Example:
A strong weekly executive report may assume:
reader has little time
reader cares about decisions and risk
technical detail is secondary
metrics need context
These contextual assumptions matter.
5. Layer 2 — Structural Logic
Identify how the artifact is organized.
Look for:
sequence
hierarchy
decision order
information grouping
progression
dependencies between sections
Example:
A strong incident report may follow:
IMPACT
↓
TIMELINE
↓
ROOT CAUSE
↓
RECOVERY
↓
PREVENTION
That sequence may be more reusable than the wording.
6. Layer 3 — Boundaries
Look for what the artifact deliberately avoids.
Examples:
no unnecessary history
no speculation
no implementation detail
no jargon
no recommendations without evidence
no unverified metrics
Negative structure can be as important as visible content.
7. Layer 4 — Reusable Scaffold
Convert the useful pattern into parameters.
Example:
Gold-standard status report:
Project Alpha
Green
Deployment completed Friday.
2.3% error rate dropped to 0.4%.
Risk:
vendor certificate expires next month.
Decision:
approve certificate renewal.
Reusable scaffold:
PROJECT
STATUS
WHAT CHANGED
METRIC / EVIDENCE
RISK
DECISION REQUIRED
The project details disappear.
The structure remains.
8. Identify Fixed vs Variable Elements
Separate:
FIXED STRUCTURE
from:
VARIABLE CONTENT
Example:
Fixed:
problem
impact
root cause
corrective action
verification
Variable:
system name
error
timeline
environment
This distinction enables reuse.
9. Extract Decision Rules
Strong artifacts often contain hidden decisions.
Example code:
If file exists:
validate it
Else:
create it
The reusable lesson may be:
Check current state before performing creation.
Look beyond syntax.
10. Extract Ordering Rules
Ask:
Why does section A come before section B?
Example:
A deployment procedure may:
BACKUP
↓
STOP SERVICE
↓
DEPLOY
↓
START
↓
HEALTH CHECK
The order is part of the knowledge.
11. Extract Verification
Look for how the gold standard proves success.
Examples:
row counts
test output
health endpoint
checksum
build result
comparison
review checklist
Verification patterns often deserve explicit preservation.
12. Extract Error Handling
Look for:
what happens when input is invalid
what happens when dependency fails
how partial failure is handled
whether retry is safe
A gold-standard implementation may be valuable largely because of failure handling.
13. Extract Tone
For written artifacts, identify:
sentence length
formality
information density
amount of explanation
use of headings
use of examples
directness
technical depth
Do not reduce tone analysis to vague labels like:
professional
Use observable properties.
14. Extract Formatting
Formatting may carry meaning.
Examples:
risk appears before recommendations
numbers are placed beside claims
decisions are separated from background
code examples are minimal
Capture formatting only when it supports function.
15. Remove Accidental Details
Strip details that do not generalize.
Example:
Gold standard contains:
C:\Deploy\ProjectA
The generalized scaffold should probably use:
{{DEPLOYMENT_PATH}}
unless the path is a permanent project fact.
16. Preserve Essential Domain Rules
Do not generalize away details that are actually part of the rule.
Example:
If every Mail.dat workflow requires a specific file relationship, that relationship may be essential.
Generalization should remove accidental specifics, not domain truth.
17. Ask Why Repeatedly
For each strong element ask:
Why is this here?
What problem does it prevent?
Would quality decrease if removed?
If an element has no reusable function, do not necessarily preserve it.
18. Candidate Output Types
After analysis choose the best artifact.
Skill
Use when the gold standard reveals a procedure.
Prompt
Use when it reveals a repeatable AI task.
Template
Use when it reveals a reusable structure.
Checklist
Use when it reveals validation criteria.
Convention
Use when it reveals a stable coding or project pattern.
Rubric
Use when it reveals how quality is judged.
19. Test Generalization
Never assume the extracted structure is reusable.
Test it on:
A SECOND, SUBSTANTIALLY DIFFERENT EXAMPLE
If it only works on the original, it was copied rather than generalized.
20. Transfer Test
Ask:
Does the pattern survive when:
names change?
domain details change?
data volume changes?
another author uses it?
the scenario changes?
The stable elements are likely the real pattern.
21. Over-Generalization Test
Watch for scaffolds that become meaningless.
Weak:
1. Understand problem
2. Solve problem
3. Verify solution
This is technically general but not useful.
A reusable scaffold needs enough specificity to improve execution.
22. Under-Generalization Test
Watch for templates containing too many original details.
Weak:
Always write:
"KENT3821 certified mailing..."
when the goal is a general certified-mail workflow.
Replace accidental specifics with parameters.
23. Reverse Engineering Code
For code, inspect:
responsibility boundaries
control flow
error handling
data flow
naming
dependencies
interfaces
tests
logging
security checks
Do not simply reproduce syntax.
24. Reverse Engineering Documents
Inspect:
audience
section order
information density
evidence placement
decision placement
tone
length
formatting
Then determine which elements create quality.
25. Reverse Engineering Prompts
Analyze:
role
objective
inputs
procedure
constraints
output format
examples
tool use
failure handling
verification
Then test whether the prompt structure transfers to another task.
26. Reverse Engineering Workflows
For operational workflows, extract:
trigger
preconditions
sequence
decision branches
failure paths
verification
rollback
completion criteria
These often become excellent skills.
27. Reverse Engineering User Corrections
Repeated user edits to agent output can act as a gold standard.
Example:
The user repeatedly changes reports to:
remove opening paragraph
put decision first
shorten bullet list
show exact numbers
That repeated pattern may justify:
EXECUTIVE REPORT SKILL
or an existing skill update.
28. Compare Multiple Good Examples
When several gold standards exist:
COMPARE
Look for shared patterns.
Shared structure is stronger evidence than one example.
Example:
Three successful deployment scripts all:
perform preflight
create backup
stop service
deploy
verify health
rollback on failure
This strongly supports a deployment skill.
29. Compare Good vs Bad
A weak artifact can help reveal the pattern.
Ask:
What does the good example contain
that the bad example lacks?
or:
What does the bad example contain
that the good example deliberately avoids?
Contrast improves extraction.
30. Gold Standard Audit
Do not assume the reference is perfect merely because it is preferred.
Before extracting it, check for:
unsafe patterns
obsolete practices
accidental bugs
project-specific hacks
unnecessary complexity
A gold standard can still contain defects.
Preserve what is good, not blindly everything.
31. Output Contract
A reverse-engineering result should contain:
GOLD STANDARD PURPOSE
ROLE / CONTEXT
STRUCTURAL PATTERN
IMPORTANT BOUNDARIES
VARIABLE ELEMENTS
REUSABLE SCAFFOLD
TARGET ARTIFACT TYPE
TRANSFER TEST
32. Example — Executive Update
Gold standard characteristics:
short
status first
numbers support claims
risk is explicit
decision request is clear
Extracted template:
# {{PROJECT}}
Status: {{STATUS}}
## What Changed
{{FACTS}}
## Evidence
{{METRICS}}
## Risk
{{RISK}}
## Decision Needed
{{DECISION}}
33. Example — Code Pattern
Three existing services all:
validate inputs
open connection late
use parameterized command
close connection in cleanup
normalize errors
Possible output:
.ai/skills/data-access-service/SKILL.md
The skill should encode the procedure rather than copying one service class.
34. Example — Troubleshooting
A successful troubleshooting transcript repeatedly used:
capture exact error
identify layer
test connectivity
test authentication
test application behavior
This could become a diagnostic skill.
35. Relationship to Prompt Architect
Use Reverse Engineer when you have:
GOOD EXISTING OUTPUT
Use Prompt Architect when you have:
GOAL + REQUIREMENTS
Typical combination:
GOLD STANDARD
↓
REVERSE ENGINEER
↓
SCAFFOLD
↓
PROMPT ARCHITECT
↓
PRODUCTION PROMPT
36. Relationship to Skill Extractor
Reverse Engineer focuses on:
AN EXISTING HIGH-QUALITY ARTIFACT
Skill Extractor focuses on:
A PROCEDURE DISCOVERED THROUGH REAL WORK
They overlap but have different triggers.
37. Relationship to Mirror Audit
After extracting a pattern:
MIRROR AUDIT IT
Ask whether the pattern:
depends on hidden assumptions
has unsafe constraints
overfits the example
This prevents copying flawed conventions.
38. Versioning
Reusable artifacts extracted from gold standards should be versioned when they become important.
Example:
executive-update_v1.0.md
Later:
executive-update_v1.1.md
Track why the structure changed.
39. Self-Improvement
Improve this skill when:
- extracted patterns repeatedly overfit examples
- agents generalize too aggressively
- verification is not transferred
- domain truth is accidentally stripped away
- a better transfer test emerges
Do not make the extraction process unnecessarily academic.
The goal is reusable execution.
40. Anti-Patterns
Avoid:
COPY THE EXAMPLE
Avoid:
GENERALIZE UNTIL NOTHING USEFUL REMAINS
Avoid:
PRESERVE ACCIDENTAL DETAILS
Avoid:
ASSUME GOLD STANDARD MEANS PERFECT
Avoid:
SKIP TRANSFER TEST
41. Changelog
1.0.0
Initial active version.
Introduced:
- four-layer analysis
- fixed-vs-variable extraction
- decision-rule extraction
- boundary analysis
- verification extraction
- transfer testing
- over/under-generalization checks
- multi-example comparison
- good-vs-bad contrast
- gold-standard auditing