Accepted · EMNLP 2026 · Industry Track

SkillHone↗

Skills improve.
The knowledge stays.

Turn real failures into tested improvements. SkillHone evolves complete Skill repositories and preserves every decision, so the next run builds on the last.

Codex · Claude Code · Cursor · Pi · ZCode

SKILLHONE / IN MOTION00:35

Real failure → tested repair → reviewable PR

GAIA
+15.8points
over the commercial deep-research agent
WebWalkerQA-EN
+3.2points
same agent, raw open-web setting
Whole-Skill improvement
Beyond the prompt
Instructions, tools, references and regression tests
Persistent decisions
Keep the reasoning
Diagnoses, revisions, evidence and outcomes

Paper results relative to its commercially backed deep-research baseline. SkillHone uses a raw open-web setting without a pre-integrated search stack. See the comparison and experimental context.

Start with the Agent you already use.

Give this prompt to Codex, Claude Code, Cursor, Pi or ZCode. The installation guide covers setup and verification.

Ask your coding Agent
Install SkillHone from https://github.com/Tencent/SkillHone, following docs/install/skillhone.md. Make the CLI and bundled Skills available to this Agent, use the documented --install-links=true Git installation command, and verify the installation.

Recording and reviewing Issues, PRs and Wiki records needs no model credential. Install and configure the optional DeepSeek Harness runtime when you want it to execute repairs.

Prefer the terminal?
npm install -g --install-links=true git+https://github.com/Tencent/SkillHone.git#main
skillhone --help
skillhone setup --agent codex

Replace codex with your Agent runtime. Follow the guide to configure models and verify a real repair-tool round trip.

A Skill can improve.
Its knowledge should accumulate.

A missing script, a broken reference, an outdated API: real work exposes gaps that a polished instruction file alone cannot solve. SkillHone connects those failures to complete, reviewable repairs.

Improve the complete Skill

Work on the repository that makes the Skill useful: its entry instructions, executable tools, reference material, assets and tests.

Keep what each attempt taught

Record the diagnosis, revision, evidence and outcome. Future Agents can inspect earlier fixes and rejected alternatives across sessions.

Review evidence with the change

Follow a failure through its Issue, regression test, repair run and local PR. Decide from the tests and diff, with a Wiki record to return to.

SKILL.mdscripts/references/assets/tests
Explore the longer research walkthrough

The original two-minute research introduction explains persistent decision history and cross-backbone transfer.

Original research demonstration

See what needs attention.
Review what changed.

The local workbench puts Skills and their repair records in one place.

  • Skills at a glance: current state, open Issues and per-Skill approval counts.
  • An approval inbox: inspect tests, changed files, commits and the repair trajectory.
  • History that stays: linked Wiki records preserve the reasoning behind earlier revisions.
Open the workbench installation guide
SkillHone development workbench showing a Skills list and two local PRs awaiting review
Development workbench with actual local records. Click to inspect the full screenshot. Local PRs belong to the Skill repository on your computer.

From a real failure to a repair you can review.

  1. Report

    Capture a reproducible failure with sanitized evidence in a local Issue.

  2. Reproduce

    Attach a focused regression test for the behavior that needs to change.

  3. Repair

    DeepSeek Harness works on an isolated branch and runs the linked tests.

  4. Review

    Inspect the local PR, its diff, tests and run before approving a merge.

  5. Remember

    Keep the linked outcome and Wiki record for the next Agent or session.

Follow a public example. The Phoenix tracing Skill referenced four documents that were never shipped. Inspect the original Issue #2567, the merged fix #2568, and the reproducible SkillHone walkthrough.

Two ways to keep improving.

Choose the workflow that matches the evidence you have.

Repair failures from real work

Start with a reproducible Skill defect.

Use runtime maintenance for missing files, broken scripts, stale instructions and API drift. Review the linked regression test and local PR.

Runtime maintenance workflow

Improve a measured capability

Start with a frozen evaluator in a separate repository.

Measure a baseline, iterate on candidates, validate the proposed improvement and retain the decision history. Evaluation data stays in the Eval repository.

Evaluation-driven workflow
The idea

A diff shows what changed. SkillHone records why.

Agent skills extend language-model agents with task-specific procedures, scripts, and references — but the tasks and environments they target continually change. Existing methods polish skills in bounded runs and retain only the final artifact, discarding the decision history that later agents need to interpret prior revisions, evaluations, and rejected alternatives.

SkillHone represents each development step as a structured decision record. When the environment shifts, later subagents can inspect the persistent history to decide whether a failure is new, whether a similar fix was already attempted, and why an alternative was rejected — preventing redundant edits and accidental rollbacks of working repairs.

01 — Artifact gap

Diff vs. decision

A diff says how files changed. A decision record links that change to the problem it targeted and the evidence used to evaluate it. Long-lived skills need the latter.

02 — Agent-facing harness

Portable across runtimes

SkillHone records diagnoses, revisions, evidence, and outcomes under separated roles — portable across Claude Code, Codex, and Hermes runtimes.

03 — Empirical gains

Bundles that transfer

Improves skill development across public benchmarks and internal deployments, with skill bundles that transfer across execution backbones with no re-optimization.

The harness

One dispatcher, two role-separated teams, one persistent history.

SkillHone splits each step into optimization and evaluation dispatches. The dispatcher is a message router that creates subagents on demand and records accepted outcomes — making the optimizer/evaluator split structural rather than prompt-imposed.

SkillHone architecture: optimization team and evaluation team connected through a dispatcher that writes a persistent decision history.

Figure 1. At each step, fresh role-bounded subagents are dispatched. Routed evidence, repository operations, and recorded outcomes form a persistent decision history that later agents reuse.

Dispatch patterns

Roles emerge from permissions, not prompts.

Each subagent is created on demand, assigned to a permission-bounded team, and granted only the actions it needs for that dispatch. The optimizer/evaluator split is enforced by what each team can read or write.

Inspect all research roles and permissions
Optimization

Proposer

Reads redacted reports and decision history; writes diagnoses. No access to probe targets or validators.

Optimization

Explorer

Searches external resources for reusable patterns when a revision needs them. No access to probe targets or validators.

Optimization

Developer

Edits the current skill and proposes typed revisions. No access to unredacted evaluation assets.

Optimization

Reviewer

Reviews pending changes using redacted evidence and prior decision records. No access to probe targets.

Optimization

Decider

Accepts or rejects candidate revisions according to recorded evidence. No access to unredacted evaluation assets.

Evaluation

Executor

Runs the current skill on probe items. May inspect oracle targets, validators, and traces. No skill-repo write access.

Evaluation

Diagnoser

Analyses outcomes and traces; updates evaluation diagnostics. No skill-repo write access.

Evaluation

Reporter

Produces redacted problem reports for the optimization side. No revision-decision authority.

Evaluation

Auditor

Checks probe metadata, traces, and redacted reports for evaluation-side consistency. No skill-repo write access.

Runtime

Dispatcher

Creates subagents and routes artifacts between repositories. No direct skill edits; no access to unredacted probe targets.

Main results

Higher benchmark averages in the paper's raw open-web setting.

Published in EMNLP 2026 Industry Track. The results below are from the paper's evaluation, with its stated baselines and settings.

Under the raw open-web setting, agents receive no pre-integrated search tools and must organise search, parsing, extraction, and recovery through portable skill bundles. SkillHone leads both averages, despite the baseline keeping its commercial search stack.

Setting System GAIA WebWalkerQA-EN
L1L2L3Avg. EasyMed.HardAvg.
Curated search deep-research agent 61.947.026.348.8 58.562.367.163.2
Raw open-web Existing-Skills 64.333.321.141.7 51.248.552.650.2
Skill-Creator 64.337.921.144.1 36.636.940.838.1
Hermes-SE 73.840.931.650.4 53.751.555.353.0
SkillHone (Ours) 76.266.731.664.6 53.769.268.466.4

Table 1. Main results. Bold marks the best score per column. SkillHone runs without curated search yet achieves the strongest averages.

GAIA accuracy under the development backbone (Qwen3.6-35B-A3B) and the transfer backbone (Claude Sonnet 4.6).

Cross-backbone transfer

With zero re-optimization on the new backbone, the SkillHone skill bundle transfers from Qwen3.6-35B-A3B (development) directly to Claude Sonnet 4.6 (transfer).

SkillHone reaches 72.4% on GAIA under Sonnet 4.6 — beating Hermes-SE by 10.2 pp, Existing-Skills by 15.7, and Skill-Creator by 24.4. The gain reflects the skill procedure, not fitting to one model.

The controls are doing real work.

No decision history keeps role-separated subagents but starts each step from the latest skill artifact alone. No role separation keeps decision history but lets a single subagent access the skill repo and unredacted evaluation assets jointly.

VariantGAIAWebWalkerQA-EN
SkillHone (full)64.666.4
w/o decision history51.2 (−13.4)55.5 (−10.9)
w/o role separation58.2 (−6.4)61.1 (−5.3)

Table 2. Ablation. Decision history is the larger lever.

Trajectory

Where persistent history earns its keep.

Probe-split trajectories for SkillHone vs. Hermes-SE across five optimisation iterations from a shared seed skill. SkillHone improves 30% → 70% while recovering from two regressed revisions through targeted follow-up edits. Hermes-SE accepts or rejects whole prompt candidates under a scalar validation signal.

Optimisation trajectories of SkillHone and Hermes-SE.

Figure 2. Later revisions can target the offending part of a change while retaining useful edits — only possible when prior decisions are still inspectable.

Iter 1 looks like “no progress” on pass rate — but timeouts dropped 4→0 and average solve time fell 3×. Only a harness that reads its own history can see that and keep going. — Iteration 1 observation, travel-qa
Deployment study

+18.8 pp average across seven internal scenarios.

Beyond public benchmarks, SkillHone is deployed on internal tool-mediated analysis scenarios with exact-match evaluation. It improves six of seven seeded skills; gains concentrate where the initial procedure leaves reusable analysis steps underspecified.

Counting
+30.0
Aggregation
+26.3
Structure parsing
+25.0
Density estimation
+23.1
Span retrieval
+21.5
Filtered ranking
+5.9
List filtering
+0.0
Average
+18.8

Table 3. Optimised minus seeded accuracy on recurring internal scenarios.

Cite

Cite the EMNLP 2026 paper.

ACL Anthology · arXiv · Download BibTeX

@inproceedings{li-hu-2026-skillhone,
  title = "{S}kill{H}one: A Harness for Continual Agent Skill Evolution Through Persistent Decision History",
  author = "Li, Zhiwei and Hu, Yong",
  editor = "Atanasova, Pepa and Haffari, Gholamreza and Wang, Minghan",
  booktitle = "Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track",
  month = oct,
  year = "2026",
  address = "Budapest, Hungary",
  publisher = "Association for Computational Linguistics",
  url = "https://aclanthology.org/2026.emnlp-industry.105/",
  pages = "1819--1829",
  ISBN = "979-8-89176-460-6"
}

A few practical questions.

What changes when I already have a Skill?

SkillHone starts from the existing repository and evidence of a failure or evaluation gap. A revision can change instructions, scripts, references, assets and tests, while preserving the decisions behind each attempt.

Which Agents can use it?

The local CLI is runtime-neutral. Codex, Claude Code, Cursor, Pi and ZCode are documented installation targets; other CLI-capable Agents can call the same workflow. DeepSeek Harness is the managed repair executor.

Does it merge or push changes automatically?

Review is the default. Automatic local merge is an explicit opt-in and requires linked tests to pass. Pushing is a separate user-authorized action. A copy-mode import also has an explicit synchronization step.

Do the paper's gains describe every repair?

The reported improvements belong to the paper's benchmark settings and baselines. For your own Skill, use its regression tests or a separate evaluator to measure the effect of a proposed change.

Acknowledgements

SkillHone uses DeepSeek Harness as its repair executor. It works on isolated branches and runs linked tests; SkillHone keeps the resulting local PR, evidence and decision history together.