feat(capabilities): add governed BeeScan HTTP probing

[beesyst]

Summary

Integrate BeeScan as an optional BeeAgent module and implement one real, explicitly authorized, host-governed HTTP probe using ProjectDiscovery httpx.

• Independent operator approval and exact-scope enforcement.
• Verified provider executable and fixed command profile.
• Network namespace, nftables and process isolation.
• Bounded execution, cleanup, output parsing and evidence artifacts.
• Real HTTP Observations using BeeScan It4 contracts.
• GitHub CI with an explicit core test profile.

Existing ROP and BeeDrill dispatch and default configuration remain unchanged.

Related issue

Closes #301

Depends on beesyst/beescan#8 and its reviewed immutable revision.

Iteration

• Iteration: BS-1 — BeeScan Module Integration and Governed HTTP Probe
• Goal: One real, safe HTTP probe with validated Observations.
• Change level: security-sensitive.

Scope

Included: optional BeeScan installation, module registration, host approval, provider pinning, restricted egress, fixed HTTP probe, CLI, artifacts, integration tests, CI and operational documentation.

Excluded: generic scanner execution, crawling, Nuclei, Findings, AI, BeeUI, database and changes to Solana execution.

Changes

• config/settings.yml / core/settings.py — disabled-by-default registration and validated approval.
• config/start.py — BeeScan bootstrap/probe CLI.
• core/beescan_policy.py — approval and execution policy.
• core/beescan_http_runner.py — isolated provider execution and destination enforcement.
• core/beescan_probe.py — strict output parser, handoff and artifact lifecycle.
• tests/test_beescan_probe.py — security and contract regressions.
• integration_tests/test_beescan_lab.py and fixtures — explicit real-provider laboratory gate.
• .github/workflows/ci.yml — PR/main CI.
• Documentation and limited BeeDrill cleanup-test race correction.

The final delivery must include a reproducible BeeScan dependency pin and corresponding CI installation.

Config / Contract impact

• config/settings.yml contract changed
• Fail-fast settings validation
• Module integration extended
• Artifact contract extended
• Capability/integration contract extended
• CLI/runtime behavior extended
• Documentation updated

New configuration

modules.registry[].enabled controls module loading.

beescan.probe_enabled and beescan.approval independently control network execution.

New artifacts

bootstrap_health.json, module_result.json, http_probe_status.json, http_observation.json and bounded evidence JSON.

Verification level

Base checks

• Full local pytest reported
• Real CLI/laboratory smoke reported
• Logs and artifacts inspected
• Hosted PR CI — pending

Quality / security

• SAST assessment
• Provider SCA applicability assessment
• Authorized laboratory DAST
• Adversarial/property tests
• IAST not required for this fixed-profile gate

Test details

Latest reported results before the final dependency/CI adjustment:

• BeeAgent: 2332 passed, 1 skipped.
• Real lab: 65 passed.
• BeeScan: 337 passed.
• Ruff and Pyright: passed, 0 type errors.
• Build and diff hygiene: passed.
• Bandit: documented pre-existing advisory findings, not represented as a green result.

Full and clean-checkout checks must be rerun after the immutable BeeScan dependency pin is added.

Manual scenarios checked

• Approved HTTP 200.
• Original-target HTTP 302 without redirect following.
• Forbidden IP, port and IPv6 egress.
• Proxy bypass attempts.
• DNS rebinding.
• Excluded and expired approval.
• Missing or corrupted provider.
• Missing isolation/firewall.
• Timeout and worker cleanup.
• Malformed or oversized provider output.
• Contradictory host/host_ip with no false-success artifacts.

Artifacts

Verified complete Observation, status and evidence linkage for real authorized lab runs.

Failed/refused runs retain bounded status artifacts without successful Observations or evidence references.

Security review

• Host UID and live approval govern authority.
• Module declarations never authorize execution.
• Provider version, SHA-256 and fixed arguments are enforced.
• Default-deny egress restricts actual destination connections.
• Redirects, proxies and unauthorized provider options cannot expand scope.
• Process isolation and resource budgets fail closed.
• Strict provider metadata validation prevents contradictory success.
• Response bodies, unrestricted headers, credentials and raw scanner logs are excluded from public artifacts.

The pinned httpx 1.12.0 Go 1.27.2 rebuild has a documented profile-specific advisory applicability review. This does not certify arbitrary provider options or future builds.

Checklist

• Approved feature scope preserved
• Defaults and existing module states preserved
• Config and authorization fail closed
• CLI and artifact lifecycle tested
• Security boundaries verified locally
• ROP/BeeDrill regressions reported
• Documentation updated
• Immutable BeeScan dependency and clean-checkout CI verified
• Independent final BeeAgent review completed
• Hosted PR CI passed

Limitations / follow-ups

• CI must install the reviewed BeeScan It4 dependency deterministically.
• The BeeScan pin must remain reachable after the companion PR is squash-merged.
• Actual GitHub CI must pass before merge.
• Production deployments require separately configured operator approval and the exact verified provider executable.

Notes

BeeAgent version remains 0.72.0.

BeeScan stays disabled by default, ROP stays disabled, and BeeDrill remains enabled. No automatic network authority is introduced.