Refine public comment moderation and pin DeepSeek via OpenRouter

.dockerignore

@@ -1,4 +1,5 @@
 .dockerignore
+.env*
 compose.yaml
 Dockerfile
 ngbot

.env.example

@@ -26,11 +26,12 @@ NGBOT_BUILD_DATE=unknown           # Compose build metadata; set to an RFC 3339
 
 ## LLM settings
 NG_LLM_GEMINI_API_KEY=             # Gemini API key; required for reactor when NG_LLM_API_TYPE=gemini
+NG_LLM_OPENROUTER_API_KEY=          # OpenRouter key; required when NG_LLM_API_TYPE=openrouter
 NG_LLM_OPENAI_API_KEY=             # OpenAI API key; required for reactor when NG_LLM_API_TYPE=openai
 NG_LLM_API_KEY=                    # Legacy fallback for the selected provider when its dedicated key is empty
 NG_LLM_API_MODEL=                  # Optional model override; empty uses the provider default
 NG_LLM_API_URL=https://api.openai.com/v1 # LLM API base URL (used when NG_LLM_API_TYPE=openai)
-NG_LLM_API_TYPE=openai             # LLM provider {openai,gemini}; Gemini implicit caching is automatic
+NG_LLM_API_TYPE=openai             # LLM provider {openai,gemini,openrouter}; Gemini implicit caching is automatic
 NG_LLM_REQUEST_TIMEOUT=45s         # Classification request deadline
 
 ## Spam control settings

AGENTS.md

@@ -91,6 +91,7 @@ For detailed architecture, see [docs/CODEBASE_MAP.md](docs/CODEBASE_MAP.md).
 
 ### External Services
 - Telegram Bot API.
+- **OpenRouter Classification**: The OpenRouter adapter uses `deepseek/deepseek-v4.1-flash` with request-level `provider.only=["deepseek"]`, disabled fallbacks and required parameter support. Preserve the shared public-comment spam policy, critical bot `start`/`startapp` handling, and valid `0`/`1`-only completed responses. Usage/generation telemetry must exclude message and reasoning texts and credentials; missing cache/cost fields are unknown, not measured zero. Keep native OpenAI/Gemini adapters available.
 - LLM APIs (OpenAI-compatible and Gemini).
 - Banlist API (lols.bot).
 - **Join-Captcha WebApp** 🔒: The gatekeeper join-captcha WebApp server speaks plain HTTP and **MUST** run behind a TLS-terminating reverse proxy. Its listen address is configured via `GatekeeperWebApp.ListenAddr`. In the Docker deployment it binds `0.0.0.0:8080` inside the container (mapped to `127.0.0.1:18080` on the host); the default **must NOT** be changed to loopback or the container port mapping breaks.

README.md

@@ -113,11 +113,12 @@ See [.env.example](.env.example) for a quick reference. `NGBOT_*` variables conf
 | | `NG_GATEKEEPER_WEBAPP_MAX_CONCURRENT` | Maximum in-flight Mini App requests | `32` | Integer greater than zero; `0` is invalid |
 | | `NG_GATEKEEPER_WEBAPP_REQUESTS_PER_MINUTE` | Per-client Mini App request limit | `120` | Integer greater than zero; `0` is invalid |
 | | `NG_LLM_GEMINI_API_KEY` | Gemini credential; required when `reactor` uses Gemini | | Preferred over the legacy key |
+| | `NG_LLM_OPENROUTER_API_KEY` | OpenRouter credential; required when using OpenRouter | | Preferred over the legacy key |
 | | `NG_LLM_OPENAI_API_KEY` | OpenAI credential; required when `reactor` uses OpenAI | | Preferred over the legacy key |
 | | `NG_LLM_API_KEY` | Legacy credential fallback for the selected provider | | Used only when its dedicated key is empty |
-| | `NG_LLM_API_MODEL` | Optional LLM model override | Provider-specific | Any valid OpenAI or Gemini model |
+| | `NG_LLM_API_MODEL` | Optional LLM model override | Provider-specific | OpenAI/Gemini overrides; OpenRouter requires `deepseek/deepseek-v4.1-flash` |
 | | `NG_LLM_API_URL` | OpenAI-compatible API base URL | `https://api.openai.com/v1` | Used when `NG_LLM_API_TYPE=openai` |
-| | `NG_LLM_API_TYPE` | LLM provider | `openai` | `openai`, `gemini` |
+| | `NG_LLM_API_TYPE` | LLM provider | `openai` | `openai`, `gemini`, `openrouter` |
 | | `NG_LLM_REQUEST_TIMEOUT` | Maximum duration of one classification request | `45s` | Any positive duration string |
 | | `NG_SPAM_LOG_CHANNEL_USERNAME` | Channel for spam logging | | Any valid channel username |
 | | `NG_SPAM_DEBUG_USER_ID` | User allowed to run diagnostics in private chat | `0` | Telegram user ID |
@@ -225,3 +226,25 @@ Feel free to add feature requests in issues.
 Structured debug logs include `author_kind`, `author_id`, chat/message IDs, `trust_phase`, skip reason and classification outcome without message text. Existing provider usage logs retain token counts. Daily SQLite KV statistics use `stats:<chat_id>:<YYYY-MM-DD>:<metric>` keys: `author_check_initial`, `author_check_renewal`, `author_check_edit`, `author_check_pending_case`, `author_check_report`, `author_trust_granted`, and `author_trust_skipped`. Check counters include attempts; `llm_checked` counts successful message/report classifications, and existing `spam_confirmed` / `false_positive` counters record resolved cases. Compare equivalent traffic windows after rollout; local tests do not establish the cause or savings of a production incident.
 
 Context cleanup runs through the existing periodic bounded retention job. Checked-message bindings contain only identity and timing metadata, and remain until the chat is removed. They survive trust expiry, renewal, resets and context deletion so that an old checked message cannot lose edit protection during a later trust period.
+
+## Public comment classification and OpenRouter accounting
+
+The initial check and reported-message check share one policy. Public-channel subscribers do not have to join the discussion group. Spam requires promotional/referral redirection or an invitation to unspecified income/work. News sources, documentation, ordinary social profiles, detailed vacancies (including iGaming recruitment), contextual recommendations, criticism and jokes remain allowed. Telegram bot `start`/`startapp` payloads require scrutiny: unexplained personal or opaque referral-like codes are spam, while clearly contextual document/function links and technical examples are allowed. Quoting a scam to discuss or warn about it is allowed. Existing author trust, banlists, CAPTCHA, voting and per-chat settings remain separate.
+
+Select `NG_LLM_API_TYPE=openrouter`, set `NG_LLM_OPENROUTER_API_KEY`, and leave the model empty or set `deepseek/deepseek-v4.1-flash`. Each request permits only the official `deepseek` provider, disables provider fallbacks and requires parameter support. The OpenRouter endpoint is fixed to `https://openrouter.ai/api/v1`; the OpenAI base URL is unrelated. Requests use low reasoning effort with a 2,048-token completion budget, including reasoning. Truncated, refused, malformed or unexpected-provider responses cannot become spam verdicts. DeepSeek implicit prefix caching needs no cache-management calls. Stable policy and global examples precede changing context and candidate data.
+
+At log level `4` (info) or higher, `OpenRouter usage metadata` records normalized/native token counts, cached tokens, cache writes when reported, reasoning, cost and cost breakdowns, generation/request/upstream IDs, actual provider, finish reasons and timings. Numeric usage fields are preserved even when the SDK has no typed field. A background queue of at most 32 metadata lookups retries `/generation` for up to ten minutes, because OpenRouter publishes these records after a delay. Separate generation records join the initial usage record by generation ID; shutdown cancels pending lookups. Unavailable statistics do not invalidate a valid verdict. No prompt, completion text, reasoning text, credentials or external-user identity is logged. Missing cache/cost observations are distinguished from measured zeros. Cumulative ratios cover completed requests since process start; raw records remain subject to Compose log rotation (five 10 MiB files).
+
+Aggregate any retained time window without adding costs from usage and generation twice:
+
+```sh
+docker compose logs --no-log-prefix --since 24h ngbot | ./scripts/openrouter-stats.py
+```
+
+`cache_hit_request_rate` measures requests with any reported hit among cache-observed requests; `cached_input_token_rate` measures cached input tokens among cache-observed input tokens. Reasoning tokens are part of completion tokens and must not be added again. Reported cache discounts and actual billed costs are separate; no assumed list-price savings are substituted for observed billing. Generation IDs deduplicate repeated log records. Export metadata logs before rotation for longer reporting windows.
+
+The opt-in semantic regression uses the real pinned provider and incurs API charges:
+
+```sh
+NGBOT_RUN_LIVE_OPENROUTER_MODERATION=1 go test ./internal/handlers/moderation -run '^TestLiveOpenRouterPublicComments$' -count=1 -v
+```

cmd/ngbot/main.go

@@ -20,6 +20,7 @@ import (
 	"github.com/iamwavecut/ngbot/internal/adapters"
 	"github.com/iamwavecut/ngbot/internal/adapters/llm/gemini"
 	"github.com/iamwavecut/ngbot/internal/adapters/llm/openai"
+	"github.com/iamwavecut/ngbot/internal/adapters/llm/openrouter"
 	"github.com/iamwavecut/ngbot/internal/bot"
 	"github.com/iamwavecut/ngbot/internal/config"
 	"github.com/iamwavecut/ngbot/internal/db"
@@ -203,6 +204,7 @@ func main() {
 	config.RegisterSecret(cfg.LLM.APIKey)
 	config.RegisterSecret(cfg.LLM.GeminiAPIKey)
 	config.RegisterSecret(cfg.LLM.OpenAIAPIKey)
+	config.RegisterSecret(cfg.LLM.OpenRouterAPIKey)
 
 	log.SetFormatter(&config.NbFormatter{})
 	log.SetOutput(os.Stdout)
@@ -456,7 +458,12 @@ func buildRuntime(ctx context.Context, cfg *config.Config, errChan chan<- shutdo
 		errChan,
 	)
 
+	var llmComponent lifecycle.Component
+	if component, ok := llmAPI.(lifecycle.Component); ok {
+		llmComponent = component
+	}
 	runtime := lifecycle.NewRuntime(
+		llmComponent,
 		service,
 		banService,
 		spamControl,
@@ -546,12 +553,15 @@ func maskConfiguration(cfg *config.Config) *config.Config {
 	maskedConfig.LLM.APIKey = maskSecret(cfg.LLM.APIKey)
 	maskedConfig.LLM.GeminiAPIKey = maskSecret(cfg.LLM.GeminiAPIKey)
 	maskedConfig.LLM.OpenAIAPIKey = maskSecret(cfg.LLM.OpenAIAPIKey)
+	maskedConfig.LLM.OpenRouterAPIKey = maskSecret(cfg.LLM.OpenRouterAPIKey)
 	return &maskedConfig
 }
 
 func configureLLM(cfg *config.Config, logger *log.Entry) (adapters.LLM, error) {
 	apiKey := cfg.LLM.APIKeyForProvider()
 	switch cfg.LLM.Type {
+	case config.LLMProviderOpenRouter:
+		return openrouter.NewOpenRouter(apiKey, cfg.LLM.Model, logger)
 	case config.LLMProviderOpenAI:
 		return openai.NewOpenAI(
 			apiKey,

cmd/ngbot/main_test.go

@@ -19,6 +19,7 @@ import (
 	api "github.com/OvyFlash/telegram-bot-api"
 	"github.com/iamwavecut/ngbot/internal/adapters/llm/gemini"
 	"github.com/iamwavecut/ngbot/internal/adapters/llm/openai"
+	"github.com/iamwavecut/ngbot/internal/adapters/llm/openrouter"
 	"github.com/iamwavecut/ngbot/internal/bot"
 	"github.com/iamwavecut/ngbot/internal/config"
 	"github.com/iamwavecut/ngbot/internal/db"
@@ -166,9 +167,10 @@ func TestMaskConfigurationRedactsCredentialsCompletely(t *testing.T) {
 	cfg := &config.Config{
 		TelegramAPIToken: "telegram-prefix-secret-suffix",
 		LLM: config.LLM{
-			APIKey:       "llm-prefix-secret-suffix",
-			GeminiAPIKey: "gemini-prefix-secret-suffix",
-			OpenAIAPIKey: "openai-prefix-secret-suffix",
+			APIKey:           "llm-prefix-secret-suffix",
+			GeminiAPIKey:     "gemini-prefix-secret-suffix",
+			OpenAIAPIKey:     "openai-prefix-secret-suffix",
+			OpenRouterAPIKey: "router-prefix-secret-suffix",
 		},
 	}
 
@@ -185,6 +187,9 @@ func TestMaskConfigurationRedactsCredentialsCompletely(t *testing.T) {
 	if masked.LLM.OpenAIAPIKey != redactedConfigurationValue {
 		t.Fatalf("OpenAI key = %q", masked.LLM.OpenAIAPIKey)
 	}
+	if masked.LLM.OpenRouterAPIKey != redactedConfigurationValue {
+		t.Fatal("OpenRouter key was not redacted")
+	}
 }
 
 func TestConfigureLLMUsesSelectedProviderCredential(t *testing.T) {
@@ -200,6 +205,11 @@ func TestConfigureLLMUsesSelectedProviderCredential(t *testing.T) {
 			llm:  config.LLM{Type: "gemini", GeminiAPIKey: "gemini-key"},
 			want: (*gemini.API)(nil),
 		},
+		{
+			name: "OpenRouter",
+			llm:  config.LLM{Type: "openrouter", OpenRouterAPIKey: "router-key"},
+			want: (*openrouter.API)(nil),
+		},
 		{
 			name: "OpenAI",
 			llm:  config.LLM{Type: "openai", OpenAIAPIKey: "openai-key", BaseURL: "https://api.openai.com/v1"},

compose.yaml

@@ -28,6 +28,7 @@ services:
       NG_GATEKEEPER_WEBAPP_REQUESTS_PER_MINUTE: ${NG_GATEKEEPER_WEBAPP_REQUESTS_PER_MINUTE:-120}
       NG_LLM_API_KEY: ${NG_LLM_API_KEY:-}
       NG_LLM_GEMINI_API_KEY: ${NG_LLM_GEMINI_API_KEY:-}
+      NG_LLM_OPENROUTER_API_KEY: ${NG_LLM_OPENROUTER_API_KEY:-}
       NG_LLM_OPENAI_API_KEY: ${NG_LLM_OPENAI_API_KEY:-}
       NG_LLM_API_URL: ${NG_LLM_API_URL:-https://api.openai.com/v1}
       NG_LLM_API_MODEL: ${NG_LLM_API_MODEL:-}

go.mod

@@ -3,17 +3,18 @@ module github.com/iamwavecut/ngbot
 go 1.26.8
 
 require (
-	github.com/OvyFlash/telegram-bot-api v0.0.0-20260715235732-aca8bf3898bb
+	github.com/OvyFlash/telegram-bot-api v0.0.0-20260826195316-83c4eada97a1
+	github.com/iamwavecut/gopenrouter v0.0.0-20260930193135-78d3d1821ee6
 	github.com/jmoiron/sqlx v1.4.0
 	github.com/mitchellh/go-homedir v1.1.0
 	github.com/pborman/uuid v1.2.1
 	github.com/pkg/errors v0.9.1
 	github.com/rubenv/sql-migrate v1.8.1
-	github.com/sirupsen/logrus v1.9.4
-	golang.org/x/sync v0.22.0
-	google.golang.org/genai v1.66.0
+	github.com/sirupsen/logrus v1.10.2
+	golang.org/x/sync v0.23.0
+	google.golang.org/genai v1.72.0
 	gopkg.in/yaml.v2 v2.4.0
-	modernc.org/sqlite v1.55.0
+	modernc.org/sqlite v1.60.1
 )
 
 require (
@@ -40,6 +41,7 @@ require (
 	github.com/Masterminds/semver/v3 v3.5.0 // indirect
 	github.com/MirrexOne/unqueryvet v1.5.4 // indirect
 	github.com/OpenPeeDeeP/depguard/v2 v2.2.1 // indirect
+	github.com/OpenRouterTeam/go-sdk v0.9.22 // indirect
 	github.com/alecthomas/chroma/v2 v2.24.1 // indirect
 	github.com/alecthomas/go-check-sumtype v0.3.1 // indirect
 	github.com/alexkohler/nakedret/v2 v2.0.6 // indirect
@@ -74,7 +76,6 @@ require (
 	github.com/curioswitch/go-reassign v0.3.0 // indirect
 	github.com/daixiang0/gci v0.13.7 // indirect
 	github.com/dave/dst v0.27.3 // indirect
-	github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
 	github.com/denis-tingaikin/go-header v0.5.0 // indirect
 	github.com/dlclark/regexp2 v1.12.0 // indirect
 	github.com/dustin/go-humanize v1.0.1 // indirect
@@ -170,7 +171,6 @@ require (
 	github.com/nunnatsa/ginkgolinter v0.23.0 // indirect
 	github.com/pelletier/go-toml v1.9.5 // indirect
 	github.com/pelletier/go-toml/v2 v2.3.1 // indirect
-	github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
 	github.com/prometheus/client_golang v1.12.1 // indirect
 	github.com/prometheus/client_model v0.2.0 // indirect
 	github.com/prometheus/common v0.32.1 // indirect
@@ -201,10 +201,11 @@ require (
 	github.com/spf13/jwalterweatherman v1.1.0 // indirect
 	github.com/spf13/pflag v1.0.10 // indirect
 	github.com/spf13/viper v1.12.0 // indirect
+	github.com/spyzhov/ajson v0.8.0 // indirect
 	github.com/ssgreg/nlreturn/v2 v2.2.1 // indirect
 	github.com/stbenjam/no-sprintf-host-port v0.3.1 // indirect
-	github.com/stretchr/objx v0.5.2 // indirect
-	github.com/stretchr/testify v1.11.1 // indirect
+	github.com/stretchr/objx v0.5.3 // indirect
+	github.com/stretchr/testify v1.12.1 // indirect
 	github.com/subosito/gotenv v1.4.1 // indirect
 	github.com/tetafro/godot v1.5.6 // indirect
 	github.com/timakin/bodyclose v0.0.0-20260129054331-73d1f95b84b4 // indirect
@@ -232,14 +233,14 @@ require (
 	go.opentelemetry.io/otel/trace v1.44.0 // indirect
 	go.uber.org/multierr v1.10.0 // indirect
 	go.uber.org/zap v1.27.0 // indirect
-	go.yaml.in/yaml/v3 v3.0.4 // indirect
-	golang.org/x/crypto v0.56.0 // indirect
+	go.yaml.in/yaml/v3 v3.0.5 // indirect
+	golang.org/x/crypto v0.57.0 // indirect
 	golang.org/x/exp/typeparams v0.0.0-20260209203927-2842357ff358 // indirect
-	golang.org/x/mod v0.40.0 // indirect
-	golang.org/x/net v0.58.0 // indirect
-	golang.org/x/telemetry v0.0.0-20260811182544-a038080d80e5 // indirect
-	golang.org/x/text v0.41.0 // indirect
-	golang.org/x/tools v0.49.0 // indirect
+	golang.org/x/mod v0.41.0 // indirect
+	golang.org/x/net v0.59.0 // indirect
+	golang.org/x/telemetry v0.0.0-20260908163034-4bcc4b2ee518 // indirect
+	golang.org/x/text v0.42.0 // indirect
+	golang.org/x/tools v0.50.0 // indirect
 	golang.org/x/vuln v1.6.0 // indirect
 	google.golang.org/api v0.291.0 // indirect
 	google.golang.org/genproto/googleapis/rpc v0.0.0-20260729162451-8efbd57d26e0 // indirect
@@ -248,20 +249,20 @@ require (
 	gopkg.in/ini.v1 v1.67.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 	honnef.co/go/tools v0.7.0 // indirect
-	modernc.org/libc v1.74.4 // indirect
+	modernc.org/libc v1.77.1 // indirect
 	modernc.org/mathutil v1.7.1 // indirect
-	modernc.org/memory v1.11.0 // indirect
+	modernc.org/memory v1.12.1 // indirect
 	mvdan.cc/gofumpt v0.10.0 // indirect
 	mvdan.cc/unparam v0.0.0-20251027182757-5beb8c8f8f15 // indirect
 )
 
 require (
 	github.com/go-gorp/gorp/v3 v3.1.0 // indirect
 	github.com/iamwavecut/tool v1.4.2
-	github.com/sashabaranov/go-openai v1.41.2
+	github.com/sashabaranov/go-openai v1.43.0
 	github.com/sethvargo/go-envconfig v1.4.3
 	golang.org/x/exp v0.0.0-20260727155853-b88d891fe743 // indirect
-	golang.org/x/sys v0.47.0 // indirect
+	golang.org/x/sys v0.48.0 // indirect
 )
 
 tool (