How skillsmith Detects Malicious Skills

Methodology · skillsmith.ch

skillsmith is a static-first security scanner for Claude Agent Skills (SKILL.md). This page explains what the engine actually checks, what a score means, and where static analysis ends.

Layer 1: Structural lint

The frontmatter (name, description) is validated: missing fields, oversized descriptions and malformed YAML are flagged. A skill that cannot even describe itself honestly rarely deserves trust.

Layer 2: Pattern detection

The core engine matches weighted patterns across several categories:

CategoryExamples
Prompt injection"ignore previous instructions" — and paraphrases like "set aside earlier guidance", detected by meaning-level signals rather than fixed phrases
Data exfiltrationwebhook endpoints (Discord, Slack, Telegram), instructions to forward collected data outward
Credential accessreferences to ~/.ssh, .env, keychains, browser profiles
Dropperscurl … | sh, PowerShell IEX(Invoke-Expression), staged download-then-run chains
Obfuscationbase64 payloads split into chunks, defanged URLs (hxxps://…[…]) plus instructions to reassemble them
Deception"do not tell the user", "hide this step", "this file is safe"
Typosquatting/homoglyphsCyrillic look-alike characters in names and URLs (linear homoglyph scan)

Each finding carries a risk weight. The totals map to four verdicts: clean (0), low (<8), medium (<20), high (≥20). Every result includes a plain-language explanation of each finding, so you can verify instead of trusting a number.

Layer 3: Dependency intelligence (OSV.dev)

If the skill pins package versions, they are checked against Google's OSV.dev vulnerability database. Findings feed into the same risk score.

Layer 4: Behavioral analysis (optional)

Static analysis cannot answer "what would an agent following this actually do?" For that, the behavioral sandbox lets an AI analyst simulate the skill step by step inside an isolated environment and report simulated actions, indicators of compromise and a severity verdict — catching attacks that never match any string pattern.

Honest limitations

A clean score is not a guarantee of safety. It means no known pattern matched. Static scanners are a triage aid — always read short files yourself; use the sandbox when stakes are high.

Try it

Paste any SKILL.md into the scanner, run behavioral analysis on it, or integrate the API into your pipeline — see the API documentation. skillsmith is open source; audit the scanner itself on GitHub.