Epic: Massive Testing Improvement #101

New issue

Closed

opened 2026-02-26 14:48:17 +00:00 by doxios · 0 comments

doxios commented

2026-02-26 14:48:17 +00:00

Collaborator

Motivation

Cobot's test suite is in rough shape:

17 failing tests across persistence, scheduled, and web plugins
2 test collection errors (nostr, telegram — missing deps at import time)
14 plugins with zero tests (including security, cron, wallet, filedrop, subagent)
Runtime API mismatches going undetected (#97 run_loop_sync, telegram call_extension)

We need a systematic push to fix broken tests, close coverage gaps, and prevent API drift.

Current State

Metric	Value
Total tests	300
Passing	283
Failing	17
Collection errors	2 (nostr, telegram)
Plugins WITH tests	13/27
Plugins WITHOUT tests	14

Failing Tests

test_persistence.py — 10 failures (persistence plugin API changed)
test_scheduled.py — 5 failures (subagent + heartbeat plugin API changed)
web/test_plugin.py — 2 failures (lifecycle tests)

Collection Errors

nostr/tests/test_plugin.py — ModuleNotFoundError: pynostr
telegram/tests/test_plugin.py — ModuleNotFoundError: telegram

Untested Plugins

cron, filedrop, heartbeat, security, wallet, subagent, compaction, logger, loop, ollama, persistence (has core tests but plugin tests broken), cli, knowledge

Known Runtime Bugs

#97: cli.py calls non-existent run_loop_sync() — broke Alpha for a week
Telegram plugin calls self._registry.call_extension() which no longer exists — crashes poll loop

Stories

Story 1: Fix 17 failing tests (persistence, scheduled, web)
Story 2: Fix test collection errors (nostr, telegram — guard optional imports)
Story 3: Fix telegram call_extension API mismatch
Story 4: Add tests for security plugin
Story 5: Add tests for cron, heartbeat, subagent plugins
Story 6: Add tests for filedrop, wallet plugins
Story 7: Add tests for compaction, logger, loop, ollama plugins
Story 8: Add CI smoke test to catch API mismatches (import all plugins, verify key methods exist)

Definition of Done

All existing tests pass (0 failures, 0 collection errors)
Every plugin has at least basic tests
CI catches API mismatches before they reach production
Test coverage report available

## Motivation Cobot's test suite is in rough shape: - **17 failing tests** across persistence, scheduled, and web plugins - **2 test collection errors** (nostr, telegram — missing deps at import time) - **14 plugins with zero tests** (including security, cron, wallet, filedrop, subagent) - **Runtime API mismatches** going undetected (#97 `run_loop_sync`, telegram `call_extension`) We need a systematic push to fix broken tests, close coverage gaps, and prevent API drift. ## Current State | Metric | Value | |--------|-------| | Total tests | 300 | | Passing | 283 | | Failing | 17 | | Collection errors | 2 (nostr, telegram) | | Plugins WITH tests | 13/27 | | Plugins WITHOUT tests | 14 | ### Failing Tests - `test_persistence.py` — 10 failures (persistence plugin API changed) - `test_scheduled.py` — 5 failures (subagent + heartbeat plugin API changed) - `web/test_plugin.py` — 2 failures (lifecycle tests) ### Collection Errors - `nostr/tests/test_plugin.py` — `ModuleNotFoundError: pynostr` - `telegram/tests/test_plugin.py` — `ModuleNotFoundError: telegram` ### Untested Plugins `cron`, `filedrop`, `heartbeat`, `security`, `wallet`, `subagent`, `compaction`, `logger`, `loop`, `ollama`, `persistence` (has core tests but plugin tests broken), `cli`, `knowledge` ### Known Runtime Bugs - #97: `cli.py` calls non-existent `run_loop_sync()` — broke Alpha for a week - Telegram plugin calls `self._registry.call_extension()` which no longer exists — crashes poll loop ## Stories - [ ] Story 1: Fix 17 failing tests (persistence, scheduled, web) - [ ] Story 2: Fix test collection errors (nostr, telegram — guard optional imports) - [ ] Story 3: Fix telegram `call_extension` API mismatch - [ ] Story 4: Add tests for security plugin - [ ] Story 5: Add tests for cron, heartbeat, subagent plugins - [ ] Story 6: Add tests for filedrop, wallet plugins - [ ] Story 7: Add tests for compaction, logger, loop, ollama plugins - [ ] Story 8: Add CI smoke test to catch API mismatches (import all plugins, verify key methods exist) ## Definition of Done - All existing tests pass (0 failures, 0 collection errors) - Every plugin has at least basic tests - CI catches API mismatches before they reach production - Test coverage report available