🐛 fix(agent-signal): route hinted skill documents (#14895)

This commit is contained in:
Neko
2026-05-17 22:59:00 +08:00
committed by GitHub
parent 73fa3b1689
commit 6281ca4228
7 changed files with 10 additions and 5 deletions
@@ -23,6 +23,7 @@ export const systemPrompt = `You have access to an Agent Documents tool for crea
- By default, if the user does not explicitly specify otherwise, and the relevant Agent Documents tool is available for the task, prefer Agent Documents over Cloud Sandbox because it is easier for collaboration and multi-agent coordination.
- **createDocument**: create a new document with title + content. Use scope="currentTopic" only when the user asks to create a document in the current topic; otherwise omit scope for an agent-scoped document.
- Set hintIsSkill=true only when creating a document that contains reusable procedural knowledge, workflow instructions, tool usage guidance, or durable agent behavior. Leave ordinary notes unhinted.
- When the user asks to remember, save, or reuse a workflow, checklist, template, skill, or repeatable procedure for this agent or topic, prefer createDocument with hintIsSkill=true over user memory. This preserves scoped procedural knowledge without turning it into a global personal preference.
- Do not create or maintain managed skills directly; Agent Signal decides whether hinted documents become skills.
- **listDocuments**: list agent documents. Use scope="currentTopic" when the user asks about documents in the current topic. The default agent_documents_index hides web-crawled documents; pass sourceType="web" here to enumerate them, or sourceType="all" to see everything. Use this to resolve a title to a document ID before reading.
- **readDocument**: retrieve current content by document ID. This is the only way to read an agent document — there is no read-by-filename variant. Prefer format="xml" when you may edit content, because XML includes stable node IDs. If the response contains empty content, the document is genuinely empty; do not retry with a different format or filename.
@@ -24,6 +24,7 @@ Memory effort level: {{memory_effort}}
- Do **not** use memory tools for requests to create, update, refine, merge, consolidate, or store reusable skills, procedures, workflows, playbooks, checklists, agent capabilities, agent prompts, or agent documents.
- If the user asks for a "reusable skill", "future workflow", "PR review checklist skill", "agent capability", or similar operational artifact, leave it to the skill/document management path. Do not convert it into addPreferenceMemory, addExperienceMemory, or addContextMemory.
- The same boundary applies in Chinese. Requests about "复用 skill", "可复用流程", "review 流程", "检查清单", "下次参考这个流程", "保留这个流程", or "合并/更新清单" belong to skill/workflow management unless they also contain a separate personal preference.
- If recent evidence includes an agent document or tool outcome marked hintIsSkill=true, treat that as skill/document evidence, not memory evidence.
- Preference memory is only for durable user preferences about how the assistant should behave; it is not a replacement for executable or document-like procedures.
- When a message mixes a personal preference with a skill/procedure request, only persist the personal preference if it remains valuable after removing the skill/procedure content. Otherwise skip memory.
</routing_boundaries>
@@ -3,7 +3,7 @@
exports[`agent signal skill intent prompt > renders classifier messages 1`] = `
[
{
"content": "Classify skill intent for Agent Signal using semantic meaning and structured evidence. Return direct_decision only for explicit skill actions or implicit strong future-use procedural learning. Return accumulate for generic praise or weak approval. Return non_skill for user preference that does not belong to skill management. Do not author skills.",
"content": "Classify skill intent for Agent Signal using semantic meaning and structured evidence. Return direct_decision for explicit skill actions, agent documents or tool outcomes marked hintIsSkill=true, or implicit strong future-use procedural learning. When the evidence points to exactly one existing managed skill, classify the action as refine; when it points only to a hinted document or draft without an existing managed skill target, classify the action as create or promote/register. Return accumulate for generic praise or weak approval. Return non_skill for global user preference that does not belong to skill management. Do not author skills.",
"role": "system",
},
{
@@ -54,6 +54,7 @@ Rules:
- "skill" can fan out with "memory" when the feedback contains both a personal preference and a reusable workflow/template insight.
- Route to "skill", not "prompt", when the feedback asks to create, update, refine, merge, consolidate, deduplicate, or reorganize an existing reusable checklist, skill, template, workflow, playbook, or writing pattern.
- Route to "skill" for explicit requests to create or preserve a reusable operational artifact, even when the message is phrased as an imperative instead of a complaint.
- Route to "skill" when recent structured evidence includes an agent document or tool outcome marked hintIsSkill=true. Do not route that evidence to "memory" unless the feedback separately states a global user preference.
- Route to "skill" for future-scoped reuse of a concrete procedure, review checklist, troubleshooting sequence, migration-review order, deploy/rollback checklist, or repeatable working method.
- Treat "use this next time", "follow the previous order", "do it like that for this class of task", "reuse this procedure later", and similar phrasing as "skill" when recent context contains procedural steps.
- Do not route to "memory" merely because the feedback contains future-oriented language such as "future", "next time", "going forward", or "later". If the durable directive mainly preserves a repeatable procedure, checklist, or workflow, route to "skill" instead.
@@ -8,7 +8,7 @@ export interface AgentSignalSkillIntentPromptInput {
}
export const AGENT_SIGNAL_ANALYZE_INTENT_SKILL_INTENT_SYSTEM_ROLE =
'Classify skill intent for Agent Signal using semantic meaning and structured evidence. Return direct_decision only for explicit skill actions or implicit strong future-use procedural learning. Return accumulate for generic praise or weak approval. Return non_skill for user preference that does not belong to skill management. Do not author skills.';
'Classify skill intent for Agent Signal using semantic meaning and structured evidence. Return direct_decision for explicit skill actions, agent documents or tool outcomes marked hintIsSkill=true, or implicit strong future-use procedural learning. When the evidence points to exactly one existing managed skill, classify the action as refine; when it points only to a hinted document or draft without an existing managed skill target, classify the action as create or promote/register. Return accumulate for generic praise or weak approval. Return non_skill for global user preference that does not belong to skill management. Do not author skills.';
const createAgentSignalAnalyzeIntentSkillIntentPrompt = (
input: AgentSignalSkillIntentPromptInput,
@@ -31,8 +31,9 @@ const SELF_ITERATION_SYSTEM_ROLE = [
'Inspect the bounded nightly review context and use the provided self-iteration tools to read evidence or apply safe write operations.',
'Evidence ids and proposal keys are different namespaces: read topic/message/tool_call/agent_document evidence with getEvidenceDigest; read proposals with readSelfReviewProposal only when using proposalActivity.active[].proposalKey or keys returned from listSelfReviewProposals.',
'Never claim that a write happened unless a write tool result confirms it.',
'Use writeMemory for explicit durable user preferences such as response style, summary structure, or verification-reporting preferences. Do not turn those preference memories into skills or proposals.',
'Use createSkillIfAbsent only when evidence describes a reusable workflow and you can provide a non-empty skill name and full bodyMarkdown.',
'Use writeMemory only for user-level durable preferences that should survive across agents and topics, such as tone, reporting style, or verification expectations.',
'Do not use writeMemory for reusable workflows, checklists, templates, skill drafts, agent capabilities, or agent/topic-scoped procedures. Route those to skill actions or recordSelfFeedbackIntent(kind="skill").',
'Use createSkillIfAbsent when evidence describes a reusable workflow and you can provide a non-empty skill name and full bodyMarkdown. Treat agent_document evidence with hintIsSkill=true as strong skill evidence.',
'When review evidence supports an approval-gated change, call createSelfReviewProposal in this run; do not offer to draft it later.',
'For createSelfReviewProposal actions, use actionType exactly create_skill, refine_skill, consolidate_skill, or record_idea. For refine_skill include target.skillDocumentId and operation { domain: "skill", operation: "refine", input: { skillDocumentId, bodyMarkdown } }. For consolidate_skill include operation { domain: "skill", operation: "consolidate", input: { canonicalSkillDocumentId, sourceSkillIds, bodyMarkdown } }. Prefer recordSelfReviewIdea instead of proposal actions when the output is only a thought or question.',
'Stop after the useful self-iteration work is complete and summarize the confirmed outcome.',
@@ -71,7 +72,7 @@ export const createAgentSignalSelfIterationToolSystemRole = (
) =>
mode === 'review'
? 'Use resource read tools before writes when evidence is incomplete. Use getEvidenceDigest for topic/message/tool_call/agent_document evidenceRefs. Use readSelfReviewProposal only with proposal keys from proposalActivity.active or listSelfReviewProposals, never with evidence ids. Treat write tool results as the source of truth.'
: 'Use resource read tools before writes when evidence is incomplete. Direct-apply only safe memory or bounded skill updates. Record approval-gated or unsupported reflection output with recordSelfFeedbackIntent, and record non-actionable thoughts with recordReflectionIdea.';
: 'Use resource read tools before writes when evidence is incomplete. Direct-apply only safe user-level memory or bounded skill updates. Prefer skill actions over memory when evidence includes reusable workflows, templates, checklists, or agent_document hintIsSkill=true. Record approval-gated or unsupported reflection output with recordSelfFeedbackIntent, and record non-actionable thoughts with recordReflectionIdea.';
const createNightlyReviewPromptXml = (input: AgentSignalSelfIterationPromptInput) =>
toXml(
@@ -67,6 +67,7 @@ Rules:
- Use read-only tools to inspect same-turn document outcomes before guessing from document names or content shape.
- Treat same-turn document outcomes with hintIsSkill:true as strong evidence, not automatic authorization.
- For hinted ordinary documents, prefer create/register candidates unless exactly one existing managed skill is the target.
- If same-turn evidence contains createDocument with hintIsSkill:true and there is no resolved managed skill target, prefer "create" over "reject" when the document content is a reusable workflow, checklist, template, or procedural writing pattern.
- Use refine only with one resolved targetSkillRefs entry; use consolidate only with multiple resolved targetSkillRefs entries.
- Do not force refine or consolidate without targetSkillRefs.
- Do not infer skill intent from a filename, title, or SKILL.md-shaped content alone.