Technical Writer Cookbooks: Research Briefs for Seven Hard AI and Infrastructure Topics.
A staged index of seven evidence-led assignments for writers covering Tailscale, SkillOpt, agent harnesses, and trustworthy data analytics.
This series is a set of writing assignments, not a pile of prewritten conclusions. Each cookbook gives a writer a defensible thesis, a boundary around the topic, primary sources, a reproducible demonstration, failure modes, interview prompts, and a definition of done. The aim is to make research depth repeatable without pretending that a cheap first draft is a finished technical article.
The seven assignments
- Tailscale: explain the network, not just the VPN—3–4 trial hours; 4–5 production hours.
- SkillOpt: skills as trainable state—3–4 trial hours; 4–6 production hours.
- SkillOpt-Sleep and Claude dreaming—4–5 trial hours; 5–6 production hours.
- SkillOpt vs. GEPA vs. DSPy—4–5 trial hours; 5–6 production hours.
- What is an agent harness?—3–4 trial hours; 4–5 production hours.
- What is a meta-harness?—4–5 trial hours; 5–6 production hours.
- Why unlimited model tokens do not create trustworthy self-service analytics—hour budget not supplied; editor assignment required before work starts.
Cookbook A · Tailscale beyond the VPN label
Thesis
Tailscale is best explained as an identity- and policy-aware connectivity layer built on WireGuard, not as a conventional hub-and-spoke VPN. Show what the coordination plane does, what it does not carry, when peers connect directly, and when DERP relays become the fallback.
Scope
Cover node enrollment, the tailnet, keys, coordination, NAT traversal, DERP, MagicDNS, identity, and grants or ACLs. Explain control-plane/data-plane separation. Do not recommend production policy without a threat model or imply that WireGuard removes endpoint risk.
Terminology
- Tailnet: the private network formed by one Tailscale domain.
- Coordination server: distributes peer and policy information; it is not the normal payload path.
- DERP: an encrypted relay when a direct UDP path cannot be established.
- MagicDNS: tailnet name resolution.
- Grant: policy connecting subjects, destinations, and capabilities or ports.
Source map
Reproducible demo
- Enroll two disposable machines in a test tailnet and record OS plus
tailscale version. - Run
tailscale statusandtailscale ping <peer>; report direct or relayed without forcing the result. - Permit one test service and deny another; capture both results.
- Remove one node and show how revocation changes reachability. Scrub names, IPs, and policy from public evidence.
Pitfalls
- Calling coordination a traffic proxy.
- Equating encryption with unrestricted endpoint trust.
- Using “zero trust” without identity, policy, logging, and revocation.
- Calling a DERP path a failure rather than a designed fallback.
Fact-check checklist
- Separate coordination metadata from payload traffic.
- State that direct connectivity depends on network conditions.
- Validate examples against current policy syntax.
- Check volatile product-tier and pricing claims on publication day or omit them.
Outline
- The misleading simplicity of “mesh VPN.”
- Identity and keys before packets.
- Direct paths and DERP fallback.
- Policy as the operational surface.
- A two-node test with an intentional denial.
- Where Tailscale does and does not fit.
Interview questions
- Which access bottleneck motivated adoption?
- What still needs separate identity, endpoint, or secrets controls?
- How often do important paths use DERP?
- Who reviews policy changes and stale devices?
Acceptance rubric
Pass: accurate control/data-plane account, observed path test, policy denial, five primary sources, and limitations. Revise: no observed path or denial. Reject: generic VPN copy, unsupported security claims, or exposed infrastructure.
Hour budgets
Trial: 3–4 hours. Production: 4–5 hours. Trial delivers the source ledger, tested outline, and demo receipt.
Cookbook B · SkillOpt and trainable agent skills
Thesis
SkillOpt treats a natural-language skill document—not model weights—as trainable state. Its contribution is the disciplined loop around rewriting: scored rollouts, reflection, bounded edits, rejected-edit memory, and a held-out validation gate.
Scope
Explain the research engine, optimizer/target separation, and deployed best_skill.md. Cover how benchmarks and harnesses affect results. Do not merge SkillOpt-Sleep into the core method, imply every skill improves, or quote a result without its model, benchmark, harness, split, and metric.
Terminology
- Target: model or agent executing tasks with the candidate skill.
- Optimizer: model proposing edits from scored trajectories.
- Learning rate: textual edit budget.
- Selection split: held-out examples used for candidate acceptance.
- Harness: execution environment, such as direct chat or an agent CLI.
Source map
- Microsoft SkillOpt repository
- Project documentation
- SkillOpt paper
- Microsoft Research feature
- Release history
Reproducible demo
- Clone and pin the repository commit; use the documented Python version.
- Materialize a documented benchmark split in an isolated environment.
- Run and preserve one seed-skill baseline.
- Run a small optimization budget, then evaluate on the same held-out selection protocol.
- Publish the seed/selected diff, configuration, score, model identifiers, and costs.
Pitfalls
- Leakage across training, selection, and test.
- Changing model, harness, or evaluation with the skill.
- Reporting only the best run.
- Assuming cross-harness transfer.
- Treating a readable skill as efficacy evidence.
Fact-check checklist
- Tie every result to a paper table and experimental cell.
- Separate the research engine from SkillOpt-Sleep.
- Scope “zero inference-time model calls” to deployment of the selected skill.
- Pin commit and configuration.
- Account for optimization-time calls.
Outline
- The skill document as mutable state.
- The rollout-to-gate loop.
- Why held-out acceptance matters.
- A small before/after experiment.
- What benchmark tables prove.
- Where SkillOpt fits.
Interview questions
- Why bounded patches rather than whole rewrites?
- How is selection-set overfitting detected?
- Which cells remain flat?
- What evidence permits adoption?
Acceptance rubric
Pass: correct loop, pinned demo, artifact diff, held-out result, five primary sources, and limitations. Revise: README paraphrase or unpinned demo. Reject: “self-improving AI” claims without splits, gates, artifacts, and cost.
Hour budgets
Trial: 3–4 hours. Production: 4–6 hours. If paid access is unavailable, publish the pinned protocol and say the optimization was not run.
Cookbook C · SkillOpt-Sleep and Claude dreaming
Thesis
“Dreaming” is useful only after the mechanics are explicit. SkillOpt-Sleep turns recent agent experience into candidate guidance through harvest, mine, replay, consolidate, validate, and human-adopt stages. The gate and data boundary—not the metaphor—determine safety and usefulness.
Scope
Cover the standalone Sleep engine, transcript sources, backend separation, review-before-adopt, experience replay, and dream rollouts. Do not claim Claude changes its weights, learns inside an ordinary chat, or keeps all processing local when a real backend is selected.
Terminology
- Harvest: bounded read of supported session records.
- Mine: derive recurring testable tasks.
- Replay: re-run representative tasks for comparable outcomes.
- Dream rollout: repeated or varied task used for contrastive evidence.
- Adoption: explicit application of a staged proposal.
Source map
Reproducible demo
- Use a disposable project and synthetic, secret-free transcripts.
- Pin the version and run
skillopt-sleep dry-runwith the mock backend. - Inspect tasks and report; prove the target skill did not change.
- Run
python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves. - Stage and inspect a proposal, showing that adoption remains separate. Do not enable auto-adopt.
Pitfalls
- Calling the flow local while sending excerpts to a provider.
- Selecting on evaluation tasks.
- Confusing
mainfeatures with the PyPI release. - Auto-adopting without a reviewable diff and rollback.
Fact-check checklist
- Pin installed version versus
main. - Name network-calling backends.
- Pair results with task count, split, model, seed, and variance caveat.
- Prove stage/adopt separation.
- Keep “dreaming” as an observable-pipeline metaphor.
Outline
- What “Claude dreams” appears to mean.
- The actual nightly state machine.
- The outbound data boundary.
- A mock night.
- The held-out gate and human adoption.
- Where repetition helps and noise wins.
Interview questions
- What evidence makes a task recurring?
- Which transcript fields are excluded or redacted?
- What failed proposal tested the gate?
- How is adoption rolled back?
Acceptance rubric
Pass: observable pipeline, mock demo, provider boundary, version pin, gate/adopt separation, and honest scope. Revise: no inspected proposal or data-flow analysis. Reject: anthropomorphic learning or hidden network use.
Hour budgets
Trial: 4–5 hours. Production: 5–6 hours. Reserve one production hour for data-boundary and version review.
Cookbook D · SkillOpt vs. GEPA vs. DSPy
Thesis
These systems overlap but are not interchangeable: SkillOpt defines a training loop for agent skill documents; GEPA is a reflective evolutionary optimizer for textual parameters; DSPy is a programming framework whose compilers optimize modular LM programs—and DSPy can use GEPA.
Scope
Compare optimized object, abstraction, feedback, search/update, validation, runtime artifact, and evaluation budget. Include the DSPy–GEPA integration. Do not manufacture a universal winner or rank mismatched headline benchmarks.
Terminology
- Textual parameter: editable prompt, skill, code, or configuration.
- LM program: modular program with typed signatures and LM-backed modules.
- Actionable Side Information: GEPA diagnostic feedback used for reflection.
- Pareto frontier: candidates excelling on different subsets.
- Compile: DSPy optimization of prompts or demonstrations against examples and a metric.
Source map
- SkillOpt and its paper
- GEPA and its paper
- GEPA quickstart
- DSPy and its paper
- DSPy GEPA docs
Reproducible demo
- Freeze one tiny task, train/validation/test splits, metric, model, and metric-call cap.
- Represent it as SkillOpt skill, GEPA candidate, and DSPy module.
- Record baseline calls, time, artifact, and score.
- Optimize under the closest defensible budget; do not claim equality without normalized calls and examples.
- Evaluate once on untouched test and publish artifacts.
Pitfalls
- Comparing framework and optimizer as the same layer.
- Ignoring
dspy.GEPA. - Mixing models, datasets, or budgets.
- Hiding metric-design labor.
- Repeatedly inspecting test results.
Fact-check checklist
- Describe each system at the correct layer.
- Record version, model, dataset, split, and cap.
- Separate maintainer results from this demo.
- Show selected artifacts.
- Conclude with selection criteria, not a winner.
Outline
- The false three-way contest.
- Framework, optimizer, and artifact map.
- Three update loops.
- A controlled task.
- Economics and leakage.
- Decision guide.
Interview questions
- What artifact should survive optimization?
- What diagnostic feedback is available?
- How many metric calls are affordable?
- What evidence permits deployment?
Acceptance rubric
Pass: correct layers, controlled protocol, native sources, artifact examples, cost accounting, and decision criteria. Revise: feature matrix without comparable test. Reject: SEO ranking from mismatched claims.
Hour budgets
Trial: 4–5 hours. Production: 5–6 hours. Without paid evaluation, publish the protocol and mark results not run.
Cookbook E · What is an agent harness?
Thesis
An agent harness is the execution system around a model: it assembles context, exposes tools, controls the loop, constrains side effects, records state, evaluates outcomes, and handles recovery. The model proposes; the harness determines what can happen and what counts as done.
Scope
Define intake, context, tools, loop, sandbox, permissions, state, verification, recovery, and handoff. Contrast model, prompt, framework, MCP server, and CI. Do not claim a standards-body definition or define the whole organization as the harness.
Terminology
- Agent loop: observe, decide, act, evaluate.
- Tool surface: typed operations and resources.
- Policy boundary: independently enforced permissions and approvals.
- Receipt: durable proof of an action or gate.
- Recovery: retry, rollback, resume, or escalation after failure.
Source map
- Anthropic: Building effective agents
- OpenAI: Harness engineering
- OpenAI Symphony
- MCP specification
- SkillOpt
Reproducible demo
- Give the same model and prompt one disposable file-edit task in plain chat and a repository harness.
- Pin model, repository commit, and time budget.
- Record context, tools, permissions, actions, tests, retries, and artifact.
- Introduce a deterministic failing test or denied write and observe recovery.
- Compare completion evidence, not prose quality.
Pitfalls
- Calling a system prompt the harness.
- Equating tool visibility with permission.
- Calling work autonomous while a human verifies it.
- Counting a plausible answer as completion.
- Changing model and harness together.
Fact-check checklist
- Locate each component in a documented system.
- Separate model, protocol, orchestration, and external systems.
- Keep model and task constant.
- Demonstrate permission and failure behavior.
- Tie done to an external check or durable artifact.
Outline
- The model is not the agent system.
- Harness anatomy.
- MCP, sandboxes, and CI.
- Same model, different harness.
- Failure and recovery.
- A review checklist.
Interview questions
- Which failures are blocked by code?
- What state survives retries?
- Who decides completion?
- Which side effects need approval?
Acceptance rubric
Pass: bounded definition, component map, controlled comparison, failure case, primary sources, and completion criteria. Revise: taxonomy without a trace. Reject: model-centric explanation that ignores policy, state, verification, or recovery.
Hour budgets
Trial: 3–4 hours. Production: 4–5 hours. Spend production time on the failure/recovery trace, not more framework names.
Cookbook F · What is a meta-harness?
Thesis
A meta-harness is a governed system that observes and improves agent harnesses. It proposes bounded changes, runs comparable evaluations, promotes supported variants, monitors canaries, and rolls back regression. It operates on the system around the model rather than executing one task inside it.
Scope
Cover evidence intake, candidates, isolation, evaluation, promotion, canary, rollback, provenance, and budget. Explain recursive risk: the changer must be more constrained than the harness it changes. Do not present one self-editing prompt as a complete meta-harness.
Terminology
- Candidate: proposed prompt, tool, policy, context, routing, evaluation, or recovery change.
- Champion: approved harness configuration.
- Challenger: candidate tested against the champion.
- Promotion gate: independent evidence threshold.
- Canary: limited exposure before broad adoption.
Source map
Reproducible demo
- Freeze a three-task fixture and champion harness.
- Create one bounded challenger, such as a mandatory test before completion.
- Run both with the same model, inputs, limits, and clean environments.
- Apply a written promotion rule covering success, cost, and harmful side effects.
- Canary on one new task, simulate regression, and prove rollback restores the prior version.
Pitfalls
- The optimizer edits its evaluator or gate.
- Experiments contaminate shared state.
- Pass rate rises through unsafe cost or actions.
- A small selection set is overfit.
- The old configuration cannot be reconstructed.
Fact-check checklist
- Label this an operational definition, not a universal standard.
- Separate mutable and immutable components.
- Trace candidates to evidence and revision.
- Include performance, cost, safety, and variance.
- Prove rollback.
Outline
- Improving the executor.
- Champion/challenger loop.
- Mutable versus independent controls.
- One-variable experiment.
- Promotion, canary, rollback.
- Recursive governance.
Interview questions
- Which parameters may change automatically?
- Who owns immutable gates?
- How is contamination detected?
- Which negative control catches shortcuts?
Acceptance rubric
Pass: operational definition, control loop, pinned champion/challenger demo, independent gate, cost/safety measures, and rollback. Revise: architecture without executed promotion. Reject: self-improvement rhetoric without isolation or provenance.
Hour budgets
Trial: 4–5 hours. Production: 5–6 hours. Production funds rollback proof and skeptical review of gate independence.
Cookbook G · Why unlimited LLM tokens do not guarantee trustworthy analytics
Thesis
A finance leader can have unlimited frontier-model access and still lack self-service analytics. More inference cannot repair ambiguous metrics, inaccessible systems, unsafe permissions, weak semantic models, non-deterministic query generation, or missing reconciliation. The useful system constrains the model between a governed vocabulary and deterministic data execution.
Scope
Cover intent, semantic layers, governed metrics, SQL or tools, permissions, validation, reconciliation, provenance, and escalation. Use finance questions concretely. Do not reproduce private screenshots, claim a universal accuracy rate, or imply deterministic SQL makes business meaning deterministic.
Terminology
- Semantic layer: governed mapping from business terms to models and calculations.
- Deterministic execution: fixed query behavior after intent is resolved.
- Grounding: connection to authorized data and retrievable evidence.
- Reconciliation: comparison with authoritative totals or identities.
- Abstention: decline when intent, access, freshness, or evidence is insufficient.
Source map
- Anthropic's supplied analytics source
- Supplied video seed—verify title, speaker, date, and transcript before citing
- dbt Semantic Layer
- Google LookML
- Anthropic tool use
Reproducible demo
- Create a synthetic ledger with two revenue definitions, one late transaction, and restricted business-unit access; publish expected totals.
- Ask five finance questions against raw schema and governed semantic-contract conditions.
- Require interpreted metric, filters, query, sources, freshness, and result.
- Test ambiguity, denied access, stale data, and reconciliation; separate correct, safe abstention, and unsafe answer.
- Repeat questions to expose non-determinism.
Pitfalls
- Plausible metric invention with valid SQL.
- A semantic layer hiding stale data.
- Security applied after sensitive context exposure.
- Dropped filters, units, currency, or periods.
- Aggregate accuracy hiding costly finance errors.
Fact-check checklist
- Tie any supplied 21% claim to original method, denominator, and date.
- Separate model, semantics, database, and permissions.
- Expose query, definitions, freshness, and reconciliation.
- Report unsafe answers separately.
- Label vendor evidence.
Outline
- The wrong “more tokens” theory.
- Finance questions are semantic before SQL.
- Question to query to reconciliation.
- Raw schema versus semantic contract.
- Permission, freshness, and abstention.
- What a CFO should demand.
Interview questions
- Which metric disagreements cost most?
- Who owns and versions definitions?
- What must an answer show?
- Which questions always escalate?
- How are plausible errors detected?
Acceptance rubric
Pass: primary-source architecture, reproducible synthetic fixture, ambiguity and denial, provenance, reconciliation, and abstention. Revise: no repeated trials or expected outputs. Reject: private screenshots, leaderboard prose, or equating SQL validity with correctness.
Hour budgets
Trial: not supplied. Production: not supplied. The source conversation mentions additional dollar flexibility but no defensible hour conversion. The editor must set both caps before commissioning.
Shared definition of done
- The opening states one falsifiable thesis and names the reader.
- Every product, architecture, performance, security, and cost claim points to a dated source.
- The writer runs the cookbook's demonstration and records environment, version, command, result, and failure.
- The article distinguishes vendor claims, measured results, and the author's inference.
- No client names, private transcripts, credentials, internal paths, or unpublished operating data appear in the draft.
- A skeptical technical reviewer can reproduce the central example from the article alone.
Editorial sequence
- Commission. Confirm audience, angle, hour cap, and the one question the article must answer.
- Research. Build the source ledger before prose. Archive the checked date and version for every volatile source.
- Reproduce. Run the smallest demonstration that could disprove the proposed explanation.
- Interview. Use the cookbook questions to find operational texture, not to outsource fact-checking.
- Draft. Write from evidence; label inference and remove claims the evidence cannot carry.
- Gate. Apply the assignment rubric and send unresolved claims back for research.