Before install
Confirm the publisher identity and repository URL match the listing. Spoofed “official” SEO plugins already show up in noisy directories.
Skim LICENSE and SECURITY.md if they exist. No license + broad network permissions is a hard pass for production machines.
Check last release date against Claude Code’s recent breaking changes. Stale plugins fail in confusing ways.
During install
Record every permission prompt. If the plugin wants unrestricted filesystem write “for convenience,” ask what file scope it truly needs.
Run the documented hello-world command on a throwaway repo first. If docs skip a minimal example, assume support will be painful.
Note cold-start time and token burn on a small task. A skill that burns a fortune to rewrite a README is not “powerful” — it is inefficient.
After first real task
Did outputs include verifiable claims (URLs, file paths, diffs) or vibes?
Did it respect your stack (Next.js, WordPress, monorepo) or spray generic advice?
Would you trust it unsupervised overnight? If not, keep it in “supervised only” until the scorecard improves.
Ship your own plugin? Reverse the checklist
Authors should pass the same bar: crisp install, honest permissions, current docs, and sample commands that work on day one. Pair this with the listing quality guide so marketplace visitors see what auditors will eventually measure.