Menu

#318 test(ci): verify real plugin entry-point discovery

closed
nobody
None
2026-08-10
2026-08-09
Anonymous
No

Originally created by: jasperdingg

Pull Request

Slice

  • Repo: doberman-core
  • Feature / Slice: [#194]
  • Plan reference: issue [#194]

What this PR does

  • Installs the example plugin in a temporary virtual environment.
  • Verifies the plugin is discovered and its rule works.
  • Cleans up the plugin after testing.
  • Runs automatically in the existing CI test suite.

Tests added (run in CI)

  • pytest tests/integration/test_real_plugin_install_discovery.py -v 1 passed
  • pytest -n auto, 2448 passed, 5 skipped
  • ruff check .
  • ruff format --check .
  • lint-imports

Public-release safety (doberman-core only)

  • [x] Contains no private, enterprise, customer, or secret data
  • [x] Core works without the enterprise package

Security checklist

  • [x] Errors fail the test
  • [x] No secrets or private data added
  • [x] No production guardrail behavior changed
  • [x] No enterprise imports added

Edge cases covered / Deviations from plan / Risks introduced

  • Supports Windows and POSIX.
  • Prevents pip settings from changing the installation location.
  • Always cleans up the temporary installation.
  • No CI workflow change was needed.
  • I used some AI for planning / review

Related

Tickets: #194

Discussion

  • Anonymous

    Anonymous - 2026-08-10

    Originally posted by: fu351

    This is exactly what [#194] wanted — a real install, not a mock. You build an actual venv, pip install -e the example plugin in a subprocess, then spawn a separate interpreter that calls discover_rules() for real and asserts the plugin's ExampleRule shows up and returns Verdict.AUTH / sensitive_path_access. Exercising the PIP_TARGET edge case under --isolated is a nice touch. The mutation check confirms it isn't vacuous: point RULE_GROUP at a nonexistent group and it goes red with found []. Green on all three CI legs.

    Two small follow-ups, neither blocking:

    • pip install -e pulls the build backend (hatchling) and metadata from PyPI, so the test needs network. A one-line comment near the fixture saves a future "why is this flaky" hunt on an air-gapped runner.
    • The docstring in tests/unit/test_examples_plugin_guardrail.py still says discovery is "covered by the example's own tests" — this PR is what actually closes that gap now, so that cross-reference could point here instead.

    Merge-ready from my side. Thanks for pairing it with [#175] — a nice steady pair of PRs.

     

    Related

    Tickets: #175
    Tickets: #194

  • Anonymous

    Anonymous - 2026-08-10

    Ticket changed by: fu351

    • status: open --> closed
     
  • Anonymous

    Anonymous - 2026-08-10

    Originally posted by: fu351

    Merged — thanks! A real install-and-discover test catches the entry-point regressions the mocked unit tests can't, which is exactly what [#194] was after (now closed). Two tiny follow-ups whenever you like: a one-line note that the test needs network for the pip install, and the now-stale "covered by the example's own tests" docstring in test_examples_plugin_guardrail.py.

     

    Related

    Tickets: #194


Log in to post a comment.