commit 6099353dbc3cdd93064ddadcc1d693cb56a81c7e Author: iskra Date: Wed Jun 24 07:30:49 2026 +0000 F3199: publish bug-class-detectors + backtest-integrity skills (skills.sh + .well-known) diff --git a/.well-known/agent-skills/index.json b/.well-known/agent-skills/index.json new file mode 100644 index 0000000..cd9bb00 --- /dev/null +++ b/.well-known/agent-skills/index.json @@ -0,0 +1,18 @@ +{ + "skills": [ + { + "name": "bug-class-detectors", + "description": "Find bug CLASSES, not instances. A pack of 37 root-cause detectors (hardcoded-universe, schema-by-name, units-mismatch, stale-config landmines, fail-open config, double-count metrics, swallowed-exceptions-around-writes, dead-backend, duplicate-file-divergence, clone-class, and more) that scan a codebase for whole categories of defects and emit CI auto-fix PRs. A Snyk/Sonar alternative focused on architectural bug classes. By Elite Agentic Solutions (EAS).", + "files": [ + "SKILL.md" + ] + }, + { + "name": "backtest-integrity", + "description": "Audit any trading backtest for overfit before you trust it. Runs the Lopez-de-Prado integrity battery: purged & embargoed combinatorial-CV (CPCV), Probability of Backtest Overfitting (PBO), Deflated Sharpe Ratio (DSR) with the FULL trial count, sample-uniqueness down-weighting, and triple-barrier labels. Turns 'it backtested great' into a calibrated honesty verdict. By TradeHouse (Bloomberg-terminal-of-prediction-markets).", + "files": [ + "SKILL.md" + ] + } + ] +} \ No newline at end of file diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..be74a82 --- /dev/null +++ b/LICENSE @@ -0,0 +1,3 @@ +MIT License +Copyright (c) 2026 Elite AI Empire +Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND. diff --git a/README.md b/README.md new file mode 100644 index 0000000..9688db2 --- /dev/null +++ b/README.md @@ -0,0 +1,27 @@ +# Empire Agent Skills + +Open, portable agent skills distilled from the **Elite AI Empire** product engines. +Install into Claude Code, Codex, Cursor, OpenCode, or any supported agent: + +```bash +npx skills add eliteaiempire/agent-skills +# or a single skill +npx skills add eliteaiempire/agent-skills --skill bug-class-detectors +``` + +## Skills + +| Skill | What it does | Hosted upgrade | +|---|---|---| +| **bug-class-detectors** | Find bug CLASSES not instances — 37 root-cause detectors + CI auto-fix PRs (Snyk/Sonar alt). | https://bugclass.eliteaiempire.com | +| **backtest-integrity** | Lopez-de-Prado overfit battery (purged-CPCV, PBO, deflated-Sharpe, uniqueness, triple-barrier) — trust a backtest only if it survives. | https://th-validator.eliteaiempire.com | + +Each skill is the open, portable logic; the hosted engines add CI integration, dashboards, +auto-fix PRs, and (for backtest-integrity) 16-venue prediction-market settlement + L2 depth. + +## Discovery + +This repo also serves a `.well-known/agent-skills/index.json` discovery index, so it can be +resolved by any agent-skills client by domain as well as by `owner/repo`. + +MIT licensed. By the Elite AI Empire (EAS + TradeHouse). diff --git a/skills/backtest-integrity/SKILL.md b/skills/backtest-integrity/SKILL.md new file mode 100644 index 0000000..c4bc135 --- /dev/null +++ b/skills/backtest-integrity/SKILL.md @@ -0,0 +1,48 @@ +--- +name: backtest-integrity +description: "Audit any trading backtest for overfit before you trust it. Runs the Lopez-de-Prado integrity battery: purged & embargoed combinatorial-CV (CPCV), Probability of Backtest Overfitting (PBO), Deflated Sharpe Ratio (DSR) with the FULL trial count, sample-uniqueness down-weighting, and triple-barrier labels. Turns 'it backtested great' into a calibrated honesty verdict. By TradeHouse (Bloomberg-terminal-of-prediction-markets)." +license: MIT +homepage: https://th-validator.eliteaiempire.com +tested_with: claude-code v2.x +--- + +# Backtest-Integrity + +A great-looking backtest is the default outcome of searching enough strategies — not +evidence of edge. This skill applies the **Lopez-de-Prado** integrity battery so you only +trust a result that survives it. + +## When to use +- Before promoting a trading strategy / signal / persona to live capital. +- When a backtest Sharpe looks "too good" — quantify the overfit probability. +- Reviewing someone else's backtest claim. + +## The battery +1. **Purged + embargoed CV / CPCV** — remove train/test leakage from overlapping labels; an + un-purged CV inflates Sharpe by leaking future info across the split boundary. +2. **PBO (Probability of Backtest Overfitting)** — across combinatorial train/test splits, + how often does the in-sample-best config underperform out-of-sample? PBO > 0.5 = the + selection is no better than noise. +3. **Deflated Sharpe Ratio (DSR)** — deflate the observed Sharpe by the number of trials + ACTUALLY searched (the whole grid, not the survivors). Undercounting trials is the #1 + way DSR lies — count the full search space. +4. **Sample-uniqueness weighting** — concurrent/overlapping labels are not independent + observations; down-weight them or N is fiction. +5. **Triple-barrier labels** — label by which of {profit-take, stop, time} is hit first, + not fixed-horizon returns. + +## How to run +1. Take the strategy's returns/labels + the SIZE of the search that produced it (number of + configs/params/personas tried — be honest, include the discarded ones). +2. Run purged-CPCV → PBO. Run DSR with trials = full search space. Apply uniqueness weights. +3. Emit the verdict: `SUPPORTED` (survives) / `OVERFIT_RISK` (PBO high or DSR ≤ 0) with the + numbers and what would change the verdict. + +## Output +`{ PBO, DSR, deflated_sharpe, trials_counted, uniqueness_factor, verdict }` + a plain-English +"would I bet real money on this?" line. + +## Upgrade +The hosted **Overfit Validator** (free lead-magnet) + Research (hypothesis→p-value) + Copilot +are at **https://th-validator.eliteaiempire.com**, backed by 16-venue prediction-market +settlement + L2 depth. This skill is the portable integrity logic. diff --git a/skills/bug-class-detectors/SKILL.md b/skills/bug-class-detectors/SKILL.md new file mode 100644 index 0000000..e746c5b --- /dev/null +++ b/skills/bug-class-detectors/SKILL.md @@ -0,0 +1,77 @@ +--- +name: bug-class-detectors +description: "Find bug CLASSES, not instances. A pack of 37 root-cause detectors (hardcoded-universe, schema-by-name, units-mismatch, stale-config landmines, fail-open config, double-count metrics, swallowed-exceptions-around-writes, dead-backend, duplicate-file-divergence, clone-class, and more) that scan a codebase for whole categories of defects and emit CI auto-fix PRs. A Snyk/Sonar alternative focused on architectural bug classes. By Elite Agentic Solutions (EAS)." +license: MIT +homepage: https://bugclass.eliteaiempire.com +tested_with: claude-code v2.x +--- + +# Bug-Class Detectors + +Most static analyzers find bug *instances* — one null-deref, one injection. **Bug-class +detectors find the whole CLASS**: every place the same root-cause pattern recurs, so fixing +one fixes all and the class never regresses. This is the EAS BugClass engine +(`bugclass.eliteaiempire.com`), distilled into a portable skill. + +## When to use +- Auditing an unfamiliar codebase for systemic defects (not lint noise). +- Before a refactor or a release gate — catch the *category* of bug. +- In CI — emit auto-fix PRs that close a class atomically. + +## The 37 detected classes + +| ID | Class | What it catches | +|---|---|---| +| C1 | HARDCODED_UNIVERSE_LIST | A hardcoded universe/entity roster that silently drops a registered member when one is add | +| C2 | SCHEMA_BY_NAME | A data source validated/dispatched by name without asserting its schema (column/field shap | +| C3 | UNITS_MISMATCH | Money/quantity arithmetic (size_usd*price, qty by side) without an explicit unit/side corr | +| C4 | STALE_PW_CONFIG_LANDMINE | An inline/hardcoded secret (systemd Environment=, literal pw) that diverges from the live | +| C5 | FAIL_OPEN_NAIVE_CONFIG | A kill/arming/safety/feature gate whose backing store is missing/malformed and it defaults | +| C6 | METRIC_CONFLATION_DOUBLE_COUNT | Aliased/duplicate universe identity in capital/PnL math (double-count source), or corrupt | +| C7 | ROTATION_MISS | A long-lived consumer captures a secret at startup and never re-reads it after rotation. | +| C8 | SWALLOWED_EXCEPTION_AROUND_WRITE | A bare/catch-all except swallows a redis/db/file WRITE with no logging -> silent data loss | +| F2613 | UNIVERSE_FEE_FALLTHROUGH | A registry universe hits the generic ~2% fee fallback (silent mispricing); also peer share | +| F2614 | DUPLICATE_FILE_DIVERGENCE | A filename present in two source trees (e.g. /root/scripts and /root/shared-components) ha | +| F2649a | METRIC_MISREAD_DENOMINATOR_DRIFT | A ratio is read against the wrong denominator (coverage_frac over ALL markets vs traded ma | +| F2649b | BUILT_BUT_DARK_PRODUCER | A producer is built + default-OFF but has NO systemd unit and NO arming trigger -> sits da | +| F2650 | PERSONA_CLONE_CLASS | A signal-weight vector is byte-identical across >1 universe (same weights applied to diffe | +| F2654 | DEAD_BACKEND | A backend (container, host:port, ssh tunnel, CH table) moved/died post-migration; writers/ | +| F2602 | STALE_CRED_INHERITED_ENV | Active file uses a stale HARDCODED pw as a live connection (or a stale inherited env) -> 4 | +| F2665 | SIGNAL_RACE_FOREIGN_WRITE | An owner recomputes+overwrites its namespace each cadence; a FOREIGN writer to the same ca | +| F2666 | WRONG_ENGINE_WRONG_LAUNCHER | The launcher default engine/launcher mismatches the corpus/runtime shape (raw vs compacted | +| F2602b | LEAKED_TOKEN_IN_GIT_CONFIG | A credential (PAT/API token/password) embedded in a .git/config remote URL (https://user:T | +| F2681a | UNINITIALIZED_NAME | A name is used (Load) that is never bound in its scope and is not a builtin/import/global/ | +| F2681b | WRONG_LAYER_EDIT | A two-layer file (canonical source vs a sync/build-mirrored COPY) is edited on the COPY si | +| F2681c | LOOKAHEAD_LEAK_DECISION_TIME_JOIN | A backtest/replay signal-injection path selects a time-series point with a NON-as-of acces | +| F2681d | LIVE_STATE_CLAIM_UNVERIFIED | A memory/handoff/report file asserts a service/URL/file is missing/broken/dead/404 (or a c | +| F2681e | WRONG_BOX_INVENTORY_CLAIM | A capability/data is checked on the WRONG host/path/box (hub when the feed is apps-box-own | +| F2686 | PATCH_INJECTED_AT_WRONG_POSITION | An automated patch fragment is inserted at a position that breaks the module: a statement | +| F2687b | MISSING_TOOLING_SILENT_FALLBACK | A script/unit silently degrades when a tool/binary/import is absent: command -v X || skip- | +| F2688 | VENUE_TYPE_ARCHETYPE_MISMATCH | A persona whose archetype class is structurally impossible for its universe venue_type (se | +| F2694 | METRIC_MATURITY_CONFLATION | A monitoring/comparison reads an event-time-stamped table whose rows arrive with write-lag | +| F2694_REDIS_UNNAMED_CLIENT | Unnamed redis client (unix-socket caller un-attributable) | fc-redis unix-socket clients that never CLIENT SETNAME share addr=/tmp/fc-redis.sock:0 so | +| F2694_GITLAB_CIDR_DRIFT | GitLab webhook CIDR drift (silent auto-deploy break) | Hetzner fw rule 11082658 pins GitLab SaaS hook egress CIDRs (GCP us-east1 34.74.90.64/28+3 | +| F2685_WRONGLAYER | WRONG_LAYER_EDIT_F2402_SYNC (detector v2) | Edits made ONLY to the apps-box copy of an F2402-enforced file are reverted by the hub->ap | +| F2711_FLAG_NAMESPACE_CONFLATION | FLAG_NAMESPACE_CONFLATION | A freshness-watched service idles/skips/early-exits its MAIN LOOP on a flag F that is ALSO | +| C9 | HEARTBEAT_KEY_NAMESPACE_MISMATCH | A watchdog/monitor checks the freshness (get/ttl/exists) of a heartbeat key whose namespac | +| F228c_HEARTBEAT_KEY_DRIFT | HEARTBEAT_KEY_DRIFT_MANIFEST | Live/manifest variant: given watched_key + writer grep pattern, confirms the watched heart | +| CLONE_PERSONA | Cross-universe persona clone (byte-identical signal vector) | A persona signal-weight vector byte-identical across >1 universe, or carrying foreign-venu | +| VENUE_SIGNAL_MISMATCH | Foreign-venue / destroyer signal on a universe | A persona carries a signal structurally impossible for its venue_type (book-micro on binar | +| RETENTION_EATS_REPLAY_WINDOW | Offload/retention job re-darkens data a consumer is actively inflating | A retention/offload job (RETAIN_DAYS=4) and a long-window consumer (replay since-days=17) | +| F2765 | HARDCODED_FALLBACK_CRED_IN_SOURCE | A real credential hardcoded as a string literal in source, typically as the fallback of an | + +## How to run +1. Identify the language/stack and the highest-risk classes for it (config-driven systems → + stale-config / fail-open; data pipelines → units-mismatch / double-count / schema-by-name; + distributed systems → rotation-miss / swallowed-exception-around-write / dead-backend). +2. For each class, grep/AST-scan for the pattern signature (see class descriptions). Report + every hit as `CLASS_ID @ file:line` with the minimal fix. +3. Cluster hits by class. Propose ONE architectural fix per class (not N point-fixes). +4. If in CI with write access, open a single PR per class that fixes all instances. + +## Output +A per-class findings table: `class → instances → root cause → one fix that closes the class`. + +## Upgrade +The full hosted engine (live CI integration, auto-fix PRs, dashboard, 100s of repos) is at +**https://bugclass.eliteaiempire.com** — this skill is the open, portable detector logic.