OPENNLP-1833: gRPC document analysis service with embeddings and chunking#493
Draft
krickert wants to merge 44 commits into
Draft
OPENNLP-1833: gRPC document analysis service with embeddings and chunking#493krickert wants to merge 44 commits into
krickert wants to merge 44 commits into
Conversation
- Move to org.apache.opennlp.grpc.* package layout - Document-centric OpenNlpAnalysisService (AnalyzeDocument, GetServiceInfo, ListModelBundles) replacing the per-tool POC services - Single-bundle ModelBundleCache with shared thread-safe *ME singletons; fail-fast UNIMPLEMENTED for steps/configs not yet supported - Tidy PipelineStepPolicy, dependency layering (Netty/reflection in service module), server version constant and exit code - Reconcile design RFC with the on-disk v1 protos
…he gRPC server Adds an EmbeddingProvider abstraction with ONNX Runtime CPU and CUDA implementations behind a strict factory (model.embedder.backend=onnx|cuda). Embedding models are declared per id in the server config, loaded eagerly, and the dimension is read from the ONNX session metadata. The embedder uses the standard single-segment BERT encoding, maps OOV tokens to the unknown id, truncates at 512 wordpieces, only sends inputs the model declares, and closes all native resources deterministically. Chunking adds sentence, token-window and semantic algorithms, wired through chunk_embed_configs and PIPELINE_STEP_CHUNK with per-chunk embeddings and group statistics. Semantic chunking places boundaries on consecutive-sentence cosine similarity with percentile or fixed thresholds and min/max size constraints. The GPU build (-Dgpu) swaps onnxruntime for onnxruntime_gpu so the CPU and CUDA runtimes never coexist on the classpath. Covered by unit tests for the providers, factory, chunkers and analyzer paths; README updated.
OpenNlpGrpcServerIT relied on DefaultClassPathModelProvider classpath scanning, which is unreliable on macOS and Windows in the failsafe JVM. Unpack English sentence and tokenizer models to target/test-classes/models and pass explicit model paths in a generated integration-test config.
Replaces the hardcoded backend switch with an EmbeddingBackendFactory service provider interface. The onnx and cuda backends register through META-INF/services and additional backends (e.g. OpenVINO) can be added by placing a jar on the classpath, selectable via model.embedder.backend without server changes. Unknown backends are rejected with the list of registered ids. Includes an SPI extension test via a test-only backend.
The implicit outer classname derived from opennlp_document.proto (OpennlpDocument) collides with the OpenNlpDocument message class on case-insensitive filesystems, breaking the build on macOS and Windows.
The config was written as a hand-formatted string but read back with Properties.load, which treats backslashes as escape characters and corrupts Windows model paths. Properties.store escapes them correctly.
Treat all C* categories (private use, unassigned, surrogates) as control characters, matching the reference BERT implementation.
The model id is backend-agnostic; the server resolves it against whatever embedding provider is configured. Removes the last backend-specific name from the request contract while it is still in draft. Field number is unchanged, so the binary wire format is unaffected.
Replace the InferenceBackend enum in the wire contract with a server-side backend SPI: providers now report a backend_id string published per model in the catalog, and the EmbeddingProvider interface is batch-first. New modules: - opennlp-grpc-backend-tei: delegates embedding inference to HuggingFace Text Embeddings Inference over its native gRPC API, with eager endpoint validation, dimension discovery and concurrent batch fan-out. - opennlp-grpc-backend-openvino: delegates to OpenVINO Model Server or any KServe v2 compatible server. Served models accept raw text (BYTES) and return FP32 embeddings; a helper script exports HuggingFace models as a single fused OpenVINO graph (tokenizer, encoder, mean pooling, L2 norm). - opennlp-grpc-integration-tests: black-box tests that spawn the shaded server jar and exercise it over the network against a stub TEI backend on every build, plus opt-in suites against real TEI and OVMS containers started by the bundled scripts (CPU and GPU). Verified live with all-MiniLM-L6-v2 (CPU and CUDA), Qwen3-Embedding-0.6B (CUDA) via TEI, and all-MiniLM-L6-v2 via OVMS on CPU, asserting unit-norm vectors and topical similarity ordering end to end.
…er into focused helpers New pipeline steps backed by the bundled 1.3.0 models: PIPELINE_STEP_POS_TAG, PIPELINE_STEP_LEMMATIZE and PIPELINE_STEP_LANGUAGE_DETECT. The language detector loads through model.properties scanning since the model is language-independent. BasicDocumentAnalyzer had grown to five responsibilities; it is now an orchestrator in processor.basic delegating to AnalysisRequestValidator, ClassicStepRunner, EmbedChunkStepRunner, DocumentOffsetEncoder and StepDiagnostics. The in-process embedding backends moved to embedding.onnx and embedding.cuda, mirroring the embedding.tei and embedding.openvino backend modules; only the SPI remains in the embedding package.
Add named entity recognition via classic OpenNLP NameFinderME models, configured per entity type with model.name_finder.<type>.path, exposed through the en-ner profile/bundle, and harden the implementation: - Match entity types case-insensitively on both sides: configuration keys and ner_entity_types request filters are normalized identically, so PERSON and person are equivalent. - Stop claiming a language for operator-supplied name finder models in the catalog: their language is unknown to the server, so the NER ModelDescriptors no longer set locale/languages (the bundle still reports en for its English sentence/tokenizer backbone). - Advertise the en-ner profile only when name finder models are loaded, matching the model catalog (which already hides the en-ner bundle in that case) so GetServiceInfo never promises an unusable profile. - Remove dead NameFinderRegistry API (entityTypeSet, modelPath and the backing modelPaths map). Replace the network-dependent NER tests, which downloaded a legacy SourceForge model and silently skipped when offline, with a hermetic fixture that trains a tiny person model in-memory (MAXENT). Add OpenNlpGrpcServerNerLiveIT exercising NER end-to-end through the spawned server process with real assertions: catalog language claim, entity detection (asserted on UTF-16 offsets), case-insensitive filter, profile-by-id resolution, and unknown-type rejection.
…lp-dl) Bump opennlp.version to 3.0.0-SNAPSHOT to consume the thread-safe opennlp-dl components. OnnxSentenceEmbedder extends AbstractDL, whose env/session/tokenizer/vocab fields became final and are now initialized through a protected constructor; delegate to super(...) instead of assigning the inherited fields directly. The shared ONNX session is consequently leak-protected by AbstractDL on failed construction.
…nt NER thread-safety Address review feedback on the NER work: - Loading a configured model path that does not exist is an operator error, not a server fault. NameFinderRegistry.loadNameFinder and ModelBundleCache.loadModel/loadLanguageDetectorModel now catch FileNotFoundException and throw AnalysisException.notFound with the offending path, instead of mapping it to INTERNAL. Other IOExceptions (unexpected deserialization failures) remain INTERNAL. - Document on NameFinderRegistry that the shared NameFinderME singletons rely on OpenNLP 3.0 NameFinderME being thread-safe, and why clearAdaptiveData() is still invoked per document (stateless RPC semantics, per-thread reset). Add a test asserting a configured-but-missing name finder path is rejected with NOT_FOUND.
…K, multi-emoji) Extend OffsetMapperTest beyond the single surrogate-pair case to cover the offset conversions Copilot flagged: a combining mark (U+0301, 2 UTF-8 bytes), a 3-byte CJK character, and back-to-back surrogate pairs, across UTF-8 byte and code-point encodings.
…ntity type) Refactor the classic NER path behind a NerModel abstraction, with no behavior change, to prepare for ONNX/transformer NER models that emit several entity types from one model. - NerModel: a recognizer keyed by model id that declares the entity types it emits and returns NamedEntity records with document-relative spans (each backend owns its coordinate mapping). - ClassicNerModel: wraps one NameFinderME (one entity type), moving the token-index -> document-offset mapping and entity-type resolution out of ClassicStepRunner. - NameFinderRegistry now stores models keyed by id with an entity-type index, and exposes modelsForTypes(...) to select the distinct models to run for a request. Removed get(); the orchestrator no longer reaches for a raw NameFinderME. - ClassicStepRunner.findNamedEntities runs each selected model once, filters emitted entities to the requested types and deduplicates identical spans (a no-op for single-type classic models, the seam for multi-type models). Existing NER unit tests and the live NER IT pass unchanged.
Add a second NER backend that serves ONNX transformer models through the
thread-safe opennlp-dl NameFinderDL, alongside the classic NameFinderME
models, behind the NerModel abstraction.
- DlNerModel wraps NameFinderDL. NameFinderDL returns character offsets
over the space-joined token text; DlNerModel.documentSpan maps those
back to document offsets by overlapping tokens (unit-tested directly
with synthetic spans, since the offset mapping is the real risk).
- NameFinderRegistry loads model.name_finder_dl.<id>.{path,vocab,labels,
entity_type,backend,gpu_device_id} entries alongside the classic
namespace (no key collision), validating attributes and mapping missing
files to NOT_FOUND. ONNX models need a sentence detector, threaded in
from ModelBundleCache; create(config) keeps the classic-only behaviour.
- The model catalog reports each name finder's real backend_id
(opennlp-me / onnx / cuda).
The current opennlp-dl NameFinderDL recognizes a single entity type, so
each ONNX entry emits one configured type; the NerModel seam already
supports multi-type models. Existing NER tests and the live NER IT pass
unchanged; new tests cover the offset mapping and DL config validation.
…el download Prove the ONNX name finder backend end-to-end against a real model without redistributing it. The dl-ner build profile downloads the ONNX export of dslim/bert-base-NER (MIT) from HuggingFace into target/ at build time, and BasicDocumentAnalyzerDlNerTest configures the DL name finder, runs NER over a sentence and asserts a person entity with correct document offsets. The label set is written by the test (public model metadata, not weights). The model is fetched at build time only; it is never bundled into a built artifact and is not redistributed. Without the profile the test skips, so the default build performs no download.
Now that NameFinderDL recognizes every BIO type and labels its spans, make the DlNerModel backend multi-type: - DlNerModel reports each entity under the model's own label (span.getType(), normalized) instead of a single configured type, and declares all the types it can emit. - NameFinderRegistry derives an ONNX model's entity types from its BIO labels (B-/I- prefixes stripped) and drops the now-redundant entity_type config key. A single ONNX model (e.g. dslim/bert-base-NER) therefore serves PER, ORG, LOC, MISC, ... and the orchestrator filters by the requested ner_entity_types. The end-to-end test asserts both a person and a location entity from one model.
Introduce a NerBackendFactory service provider interface, mirroring the
embedding backend SPI, so name finders are contributed by discoverable
backends instead of being hard-wired into the registry.
- NerBackendFactory: factoryId() + create(config, context) -> List<NerModel>
- NerBackendContext: shares the sentence detector ONNX finders require
- ClassicNerBackendFactory ("classic"): model.name_finder.<type>.path
- OnnxNerBackendFactory ("onnx"): model.name_finder_dl.<id>.*, entity types
derived from BIO labels (multi-type per model)
- NameFinderRegistry discovers factories via ServiceLoader and aggregates
their models (dedup by factoryId); parsing/loading moved into the factories
- Register built-in backends in META-INF/services
- Third-party backends register their own factory the same way - no server
change required
Tests: StubNerBackendFactory proves external discovery; full module suite
green.
Add document categorization to the gRPC pipeline, mirroring the NER backend SPI. A DocCategorizerModel interface is discovered via ServiceLoader through DocCategorizerBackendFactory, with built-in classic (DocumentCategorizerME) and ONNX (DocumentCategorizerDL) backends plus a "roll your own" extension point. - DocCategorizerModel / OpenNlpDocCategorizerModel wrapper with a TOKENS|RAW_TEXT input mode so maxent models score tokens and transformer models score whole text; AutoCloseable for ONNX sessions. - DocCategorizerRegistry aggregates factories, resolves the default model id (model.doccat.default_id; sole model is implicit default), and surfaces ambiguity as INVALID_ARGUMENT. - PipelineStepPolicy, ProfileRegistry (en-doccat profile/bundle), ModelBundleCache (catalog + close), ClassicStepRunner, and AnalysisRequestValidator wire the DOC_CATEGORIZE step end-to-end. - Tests: in-memory weather/finance topic fixture, registry unit tests, analyzer integration tests, and a ServiceLoader stub backend proving third-party extension. README documents classic, ONNX, and SPI config.
…an-SNAPSHOT The gRPC DL backends depend on opennlp-dl bug fixes that are not yet in the published 3.0.0-SNAPSHOT: OPENNLP-1844 (thread-safety), OPENNLP-1845 (doccat softmax), and OPENNLP-1846 (multi-type NER). A distinctly named, locally-built version pins those patches so a plain 3.0.0-SNAPSHOT refresh cannot clobber them. Scoped to opennlp-grpc so opennlp.models.version still tracks the apache-published models. Revert this override to the inherited 3.0.0-SNAPSHOT once the three PRs merge to apache/opennlp main.
…he doccat SPI Add sentence-level sentiment, populating AnnotatedSentence.sentiment_label and sentiment_confidence when a request runs PIPELINE_STEP_SENTIMENT. Sentiment is document categorization applied per sentence, so it reuses the DocCategorizerModel abstraction and DocCategorizerBackendFactory SPI rather than duplicating them. - DocCategorizerRegistry is now namespace-aware: createForNamespace canonicalizes model.<ns>.* / model.<ns>_dl.* keys onto the built-in model.doccat* keys the factories read and drops other namespaces, so the same classic/ONNX/third-party backends serve every namespace while the catalogs stay isolated. create() is the "doccat" namespace. - SentimentRegistry is a thin wrapper over the model.sentiment.* namespace with its own default_id selection and messages. A roll-your-own backend is written once and serves both capabilities. - PipelineStepPolicy, ProfileRegistry (en-sentiment profile/bundle), ModelBundleCache (registry + catalog + close), ClassicStepRunner (classify each sentence, write label + winning-category confidence), BasicDocumentAnalyzer, and AnalysisRequestValidator wire the step. Also make the token prerequisite conditional on the model's input mode via DocCategorizerModel.requiresTokens(): classic (token) categorizers still require TOKENIZE, but raw-text models (ONNX DocumentCategorizerDL) run under a DOC_CATEGORIZE-only profile, and raw-text sentiment models run with SENTENCE_DETECT but no TOKENIZE. This fixes a prerequisite that previously blocked the documented raw-text ONNX path. Tests: in-memory positive/negative sentiment fixture, registry unit tests (incl. both directions of namespace isolation and SPI reuse), per-sentence analyzer integration tests, and raw-text no-tokenize paths for both doccat and sentiment via a raw-text stub backend. README documents sentiment configuration (classic, ONNX, SPI).
…a format selector Add constituency parsing to the gRPC pipeline. PIPELINE_STEP_PARSE builds a phrase-structure parse per sentence and writes it to AnnotatedSentence.parse_tree. Proto (gRPC-native, not a 1:1 of OpenNLP's object graph): - ParseTree carries two independent views of one parse: a structured ParseNode tree (root) and the standard Penn-Treebank bracketed string (penn_treebank). - ParseNode gains an explicit kind (NONTERMINAL/TERMINAL) and token_index so terminals link back to the sentence's token list instead of repeating text; it keeps label, span, children, probability. - AnalysisOptions.parse_formats (repeated ParseFormat) lets a client select which view(s) to populate; empty defaults to STRUCTURED + BRACKETED. Backend + pipeline: - ModelBundleCache loads an optional shared parser from model.parser.path (the model is large and operator-supplied, not bundled); isParserAvailable/getParser and an en-parse catalog bundle. - PipelineStepPolicy, ProfileRegistry (en-parse profile/bundle), ClassicStepRunner (Parse.createFromTokens -> parser.parse -> ParseTreeConverter), BasicDocumentAnalyzer dispatch, and AnalysisRequestValidator (NOT_FOUND when no parser; parse-format resolution). DocumentOffsetEncoder now recurses parse-node spans. - ParseTreeConverter turns an OpenNLP Parse into the ParseTree views: preterminals become TERMINAL nodes with the POS label + token_index, nonterminal spans cover their descendants, and the bracketed view comes from Parse.show(). Tests: hermetic ParseTreeConverterTest (built from Parse.parseParse, no model) for typing, token linkage, span coverage and format selection; BasicDocumentAnalyzerParseTest for the NOT_FOUND path plus an opt-in real-model end-to-end gated on -Dparser.model.path. PARSE was the last unimplemented step, so the obsolete "unimplemented step" policy test becomes implementsEveryDefinedPipelineStep, asserting the full PipelineStep surface is implemented. README documents parser config, the two representations, and the selector.
Fixes, by severity, with regression tests for each: - High: NER named-entity spans were never offset-encoded, so they stayed in UTF-16 char indices while every other span was converted to the requested encoding. DocumentOffsetEncoder now remaps entity spans too. - High: OpenNLP's parser is not thread-safe (its beam search mutates per-instance state), but a single shared Parser was used across concurrent requests. Each request thread now gets its own Parser from the shared, immutable ParserModel via a ThreadLocal. (The classic POS/tokenizer/ sentence/lemmatizer ME components are thread-safe in 3.0 and unchanged.) - Service boundary: analyzeDocument only caught AnalysisException; any other RuntimeException escaped as an opaque gRPC UNKNOWN. Unexpected exceptions now map to INTERNAL, logged server-side, without leaking internals. - NER DL ONNX sessions leaked at shutdown: NameFinderRegistry/DlNerModel are now AutoCloseable and ModelBundleCache.close() releases them; the cache constructor also releases already-built models if a later load fails. - Parse node probability was OpenNLP's raw log-prob written into the [0,1] field, and was emitted regardless of include_probabilities. It is now exp()'d and clamped to [0,1], and gated by include_probabilities. - pos_tag_format was silently ignored; a non-UNSPECIFIED value now fails fast with UNIMPLEMENTED instead of returning the model's native tagset. - ONNX doc categorizer: a blank line in the categories file produced a sparse index map (NPE at load) and a blank model id was accepted; both now fail with a clear INVALID_ARGUMENT. Registry ids are normalized at registration so a backend returning a mixed-case id is still found. - Remote (TEI) backend failures collapsed to INTERNAL; a new UNAVAILABLE failure type maps them to gRPC UNAVAILABLE (retryable). - ChunkGroupStats.total_tokens double-counted tokens shared by overlapping token-window chunks; it now counts each token once. Adds DocumentOffsetEncoderTest and OpenNlpAnalysisServiceImplTest, plus a GrpcStatusMapper exhaustiveness test, an opt-in parser concurrency stress test, and targeted tests across the registries and chunking.
Apache-grade documentation pass on the gRPC modules. Javadoc: - Override the sandbox parent's doclint=none with doclint=all + failOnWarnings for the gRPC modules (generated protobuf excluded via sourcepath / excludePackageNames), so missing or malformed Javadoc now fails the build. - Fill every flagged public-API Javadoc gap across the service module (DocumentAnalyzer, AnalysisException, GrpcStatusMapper, ModelBundleCache, ProfileRegistry, the registries, OpenNlpGrpcServer, ...) and the two backend factory classes, with accurate @param/@return/@throws and null/normalization semantics. - Add Apache license headers to the two main SPI META-INF/services files (surfaced by apache-rat, which runs at package/install). Proto: - Comprehensively comment the v1 .proto API (service, RPCs, messages, fields, enums) grounded in the server implementation: gRPC error model, step preconditions, chunking units/constraints, offset-encoding semantics. Notes where the contract is not yet populated (DocumentAnalytics) or fields are currently no-ops (clean_text/preserve_urls). Ignore local-only buf proto tooling (buf.yaml/buf.gen.yaml/API.md) via a new opennlp-grpc/.gitignore so it is never committed. Comment/build-config changes only; no behavior changes. Full gRPC tree builds green (doclint + rat) and all tests pass.
Add classic shallow (syntactic) chunking, the last classic OpenNLP component with an unfilled proto field. PIPELINE_STEP_SYNTACTIC_CHUNK groups each sentence's tokens into base phrases (NP, VP, ...) and fills AnnotatedSentence.syntactic_chunks. Distinct from CHUNK, which is segmentation chunking for embedding. - New PIPELINE_STEP_SYNTACTIC_CHUNK enum value (dedicated step, clean separation from CHUNK) and the en-chunk profile/bundle. - ModelBundleCache loads an optional shared ChunkerME from model.chunker.path (operator-supplied, not bundled). ChunkerME is @threadsafe, so one instance is shared (unlike the parser). - ClassicStepRunner.chunkSyntactic runs chunkAsSpans over each sentence's tokens + POS tags; toChunkResult maps the token-index spans to document spans with the chunker's phrase tag. - AnalysisRequestValidator requires a configured chunker (NOT_FOUND) and POS_TAG in the profile (FAILED_PRECONDITION); BasicDocumentAnalyzer dispatch; DocumentOffsetEncoder now remaps syntactic-chunk spans. Tests: hermetic toChunkResult conversion (token-index -> document span), the NOT_FOUND path, the offset round-trip for chunk spans, and an opt-in real-model end-to-end gated on -Dchunker.model.path. README documents it. Every PipelineStep is now implemented. Comment/behavior changes build green (doclint + rat) and all tests pass.
…gistry Introduce the multi-backend pattern: one logical model id may be served by several engines at once, resolved by priority with fallback, and pinnable to a specific engine by a strongly-typed argument (never a parsed id string). - RankedBackends<T>: generic registry — logical id -> priority-sorted engine registrations; resolve(id) (ranked) / resolve(id, engine) (pinned); invoke(id, op) with fallback / invoke(id, engine, op) pinned; builder dedups (id, engine). This is the reusable core the other capabilities will adopt. - CompositeEmbeddingProvider: first consumer. EmbeddingProviderFactory now loads EVERY configured backend (ServiceLoader) and aggregates them, instead of selecting one via model.embedder.backend (removed). A model id served by multiple engines uses model.embedder.<id>.<engine>.priority; engines serving one logical id must report the same dimension (fallback safety, enforced at startup); model.embedder.default_id picks the default model. - ONNX vs CUDA now use per-engine path keys (.onnx.path / .cuda.path) so each engine claims only its own models and a model definition (vocab/pooling) is shared across engines; CUDA stays inert without a GPU when no .cuda.path models are configured. EmbeddingProvider gains backendId(modelId) so the catalog reports each model's resolved engine. Tests: RankedBackends (priority/fallback/typed pinning/dedup) and the composite (routing, fallback, typed engine pin, dimension-mismatch, default) with stub engines; factory + backend ServiceLoader tests updated for aggregation. Whole gRPC tree builds green (doclint + rat) and tests pass. Follow-ups (Layer 1): TEI/OpenVINO return-empty-when-no-models, ONNX padded-tensor batching + TEI single-batch request.
…ence Complete embedding Layer 1 for multi-engine deploys. - TEI and OpenVINO providers now construct inert (isAvailable()==false, no models) when no .tei.target / .openvino.target is configured, instead of throwing. The composite aggregates every discovered backend, so a deploy that only configures ONNX must not be aborted by an unconfigured remote engine. Genuinely misconfigured endpoints (unreachable, missing model_name, non-embedding model) still fail fast at startup. - OnnxSentenceEmbedder gains a true padded-tensor batch path: all sequences are right-padded to the batch max and run as one [batch, maxLength] session.run with attention_mask=0 on the pads; each row is pooled over its real tokens only, so masked-out padding cannot affect a vector. The ONNX provider overrides embedBatch to use it, replacing per-text dispatch. TEI batching is intentionally left as concurrent-unary fan-out: TEI's gRPC Embed RPC takes a single string (no batch RPC exists in the gRPC surface), and TEI does its own server-side dynamic batching across in-flight requests. Tests: OnnxEmbeddingBatchParityTest (opt-in, -Ddl.embedding.model.dir) asserts the batched vectors equal the per-text vectors within 1e-4 across mixed-length inputs — verified here against all-MiniLM-L6-v2. TEI/OpenVINO inert-when-empty tests replace the old reject-on-empty tests. Whole gRPC tree builds and tests green.
Make the server's concurrency model virtual-thread based and switch TEI batching to streaming, so blocking remote calls scale without tying up platform threads. - OpenNlpGrpcServer dispatches every RPC handler on a per-task virtual-thread executor (ServerBuilder.executor). Netty's event loops stay platform threads; only the application-callback executor is virtual, so a handler that blocks on a remote backend, a streaming latch, or native inference unmounts its carrier instead of pinning it. The executor is shut down with the server. - TEI embedBatch now sends the whole batch over the bidirectional EmbedStream RPC and collects the responses (which TEI returns in request order while batching server-side), replacing the concurrent-unary fan-out. This is one call instead of N and the streaming path a fully streaming v2 will build on. - TEI and OpenVINO channels get a provider-owned virtual-thread executor for their callbacks, lifecycle-managed across the active, inert, and failed-construction paths. No synchronized blocks exist on our request path, so there is no carrier pinning to worry about; remote calls are the I/O-bound work virtual threads are for. Tests: TEI/integration fake servers implement the bidi EmbedStream (echoing one response per request, in order); the existing order-preserving and live TEI batch tests now exercise the streaming path. Whole gRPC tree builds and tests green, doclint clean.
Generalize the multi-backend pattern to NER as its own concept (it produces spans over the whole text, not interchangeable vectors), so the same recognizer can be served by several engines and a request can combine them. Resolution is count-driven by the request's NerEnginePolicy.engines: - none -> each recognizer's highest-priority engine, with fallback on failure; - one -> pin that engine (no fallback); recognizers it doesn't serve are skipped; - many -> run all listed engines and UNION their entities. Engines are a strongly-typed list (never a parsed id@engine string); an unknown engine => NOT_FOUND, a blank one => INVALID_ARGUMENT. Output is made understandable for the union case: - NamedEntity now carries `text` (the exact matched surface text, always set) and `sources` (every recognizer/engine that produced it, each with its probability and — when a provider tokenized to different offsets — its own span). - MERGE_STRATEGY: CONSENSUS (default) collapses same-type overlapping hits into one entity whose canonical span comes from the highest-priority producer and whose `sources` list all contributors; RAW keeps each producer's hit separate. Internals: - NameFinderRegistry now groups NerModels into a RankedBackends<NerModel> keyed by recognizer id; registering the same id under different backends is the multi-engine case (same id + same backend is still an error). A type->recognizer ids index drives type selection. - NerModel gains priority(); the classic (model.name_finder.<id>.priority) and ONNX (model.name_finder_dl.<id>.priority) backends read their own priority key. - NerEntityResolver applies the policy + merge per sentence and attaches provenance. - DocumentOffsetEncoder also remaps per-source spans into the response encoding. Tests: NerEntityResolver (default/pin/union, RAW/CONSENSUS, divergent-offset provenance, priority fallback, all-fail rethrow, type filter); registry updated to the id-based API; end-to-end NER asserts text + sources and engine pin/unknown-engine. Whole gRPC tree builds and tests green; doclint clean.
Extend the multi-provider span pattern (built for NER) to syntactic chunking, and generalize the request-side policy so capabilities share one type. - EnginePolicy replaces the NER-specific NerEnginePolicy (same shape: engines + MergeStrategy). AnalysisProfile gains chunk_engine_policy and parse_engine_policy alongside ner_engine_policy; NER migrated to the shared type. - Chunker is now a provider/registry like NER: ChunkerBackendFactory SPI + ClassicChunkerBackendFactory (reads model.chunker.<id>.path + .priority, was the single model.chunker.path) + ChunkerRegistry grouping chunkers by id into a RankedBackends<ChunkerModel>. ModelBundleCache holds the registry; the validator takes it (validates chunk_engine_policy engines: unknown => NOT_FOUND). - ChunkResolver applies the count-driven policy per sentence (none = top priority with fallback; one = pinned; many = union) and attaches provenance + text: ChunkSpan gains `text` and `sources` (ChunkSource: chunker_id, engine, and its own span when it diverges). MERGE_STRATEGY CONSENSUS (default) collapses same-tag overlapping chunks; RAW keeps each separate. DocumentOffsetEncoder remaps per-source chunk spans too. Tests: ChunkResolver (default/pin/union, RAW/CONSENSUS, divergent-offset provenance, priority fallback, all-fail rethrow); the token->document span mapping moved to a hermetic ClassicChunkerModelTest; end-to-end chunk test asserts text + sources. Whole gRPC tree builds and tests green; doclint clean.
Extend the multi-provider pattern to constituency parsing. Unlike the span producers (NER, chunking), a parse is a tree, so a multi-engine union returns one tree per engine rather than a merged tree; MergeStrategy does not apply. - Parser is now a provider/registry like the others: ParserBackendFactory SPI + ClassicParserBackendFactory (reads model.parser.<id>.path + .priority, was the single model.parser.path) + ParserRegistry grouping parsers by id into a RankedBackends<ParserModel>. The SPI returns the engine-agnostic gRPC ParseTree (so a non-OpenNLP engine can implement it); ClassicParserModel keeps OpenNLP's not-thread-safe parser per-thread. ModelBundleCache holds the registry; the validator takes it (validates parse_engine_policy engines: unknown => NOT_FOUND). - ParseResolver applies the count-driven policy per sentence (none = top priority with fallback; one = pinned; many = one tree per engine). Each tree is stamped with its provenance: ParseTree gains parser_id and engine. AnnotatedSentence gains parse_trees: the primary tree stays on parse_tree (back-compatible), and parse_trees carries the full list only when a union produced more than one. DocumentOffsetEncoder remaps every union tree's spans too. - ParseTreeConverter moved from processor.basic to the model package so the SPI model can produce ParseTree; its test moved with it. Tests: ParseResolver (default/pin/union tree-per-engine, priority fallback, all-fail rethrow); ParseTreeConverter test relocated; end-to-end parse asserts parser_id/engine provenance and empty parse_trees for the single-engine case. Whole gRPC tree builds and tests green; doclint clean.
Add representative "centroid" vectors over existing embeddings — the cheap, CPU-only piece of the semantic-graph pattern (mean of member vectors, no extra inference). - ChunkEmbeddingGroup gains `centroids`: one EmbeddingResult per embedding model, the element-wise mean of that group's chunk vectors, spanning all its chunks (granularity EMBEDDING_GRANULARITY_GROUP_CENTROID, new). Computed in ChunkEmbedProcessor from the per-model vectors it already has. - OpenNlpDocument gains `document_centroids`: one per model, the mean of the document's sentence-level embeddings (granularity DOCUMENT), computed in the EMBED step. Set only when EMBED produced sentence embeddings. - Centroids helper does the element-wise average; DocumentOffsetEncoder remaps the new centroid source spans into the response encoding. These are hierarchical-retrieval primitives (group/document representative vectors) and apply to every grouping the server already emits. Tests: Centroids unit (element-wise mean, single-vector, empty=>null); end-to-end assertions that the group centroid equals the mean of its chunk vectors per model and the document centroid equals the mean of the sentence vectors. Whole gRPC tree builds and tests green; doclint clean.
Add a request-driven category-chunking output: group the document's sentences by their per-sentence category (the SENTIMENT label), concatenate each category's sentences into one chunk, embed it, and attach a centroid — the "categorize then cluster + embed + centroid" idea, emitted as ordinary ChunkEmbeddingGroups. - New repeated CategoryChunkConfigEntry on AnalyzeDocumentRequest (config_id, optional `categories` allowlist, embedding_model_ids, result_set_name). Mirrors chunk_embed_configs and reuses ChunkEmbeddingGroup as the output: one chunk per category (chunk_tag = the label, text_content = the concatenated sentences, contained_sentence_indices populated), plus the per-model group centroid. - ChunkEmbedProcessor.buildCategoryGroup groups by normalized sentiment label, preserving first-appearance order; an allowlist restricts/orders the categories (case-insensitive). Sentences without a label are ignored. - Runs when category_chunk_configs is present (after the strategy chunking). Validated up front: requires SENTIMENT in the profile (else FAILED_PRECONDITION), a non-blank config_id, and supported embedding models. Today the grouping key is the sentiment label (the only per-sentence category the server produces); the surface generalizes to other per-sentence categories later. Tests: buildCategoryGroup grouping/concat/centroid, allowlist restrict+order (case-insensitive), unlabelled sentences ignored; end-to-end rejection when SENTIMENT is absent. Whole gRPC tree builds and tests green; doclint clean.
Update from main
…he/opennlp-sandbox into OPENNLP-1833-grpc-expansion
mawiesne
changed the title
…ize, uax29, term layers) Built against 3.0.0-kristian-SNAPSHOT of the full OPENNLP-1850 stack plus the 1868 full-case-fold and 1869 emoji/emoticon rungs. - PIPELINE_STEP_NORMALIZE: offset-aware text normalization with the full rung set (invisible/whitespace/quotes/dashes/digits/ellipsis/ bullets/full case fold/emoji folds and the offset-opaque NFC/NFKC/ case/accent/confusable rungs). The response carries normalized_text plus AlignmentRun edit runs reconstructed from the library Alignment (AlignmentRuns), rescaled by the offset-encoding pass so run units always match AnnotationSpan units on both texts. require_alignment (default true) rejects offset-opaque rungs with INVALID_ARGUMENT; opting out yields text-only plus a diagnostic. - tokenizer_engine "uax29": rule-based Unicode UAX #29 word tokenizer as a model-free TOKENIZE engine, populating the new Token.word_type. - AnalysisProfile.term_dimensions: per-token character-level Term layers (Token.term_layers) computed with the library's cumulative ladder semantics; STEM/LEMMA and unknown names rejected. - DlNerModel maps ONNX spans through findInOriginal (OffsetMappingNameFinder), so entity offsets stay document-exact when the finder normalizes internally. - NamedEntity reserves field numbers 10-19 for entity resolution (geocoding) payloads. - Validator guards for the new surface; NORMALIZE registered in PipelineStepPolicy; ParityStepsTest covers typed uax29 tokens in document coordinates, aligned and opaque normalize paths, alignment run reconstruction and full-coverage invariants, term layers, and the UTF-8 byte rescale. Full service suite: 219 tests green.
- AnalysisProfile.term_profile selects the library's per-language NormalizationProfiles registry (e.g. "en", "de") and computes Token.term_layers with the profile's matching analyzer, including its language-appropriate Snowball stem layer. The analyzer is built per request because Snowball stemmers are stateful and must not be shared across threads. - Mutually exclusive with term_dimensions; requires TOKENIZE; a language with no registered profile => NOT_FOUND. Validator guards included. - Tests: profile ladder on English (STEM layer, run from Running), unknown-language fail-loud, and a rung-mapping invariant test that every declared NormalizationRung maps into the builder and that the offset-aware claims hold against the library's buildAligned gate. Full service suite: 222 tests green.
parity for the merged Unicode stack Wire parity with the library: the OPENNLP-205 sentence span mapping is pinned through AnalyzeDocument, every parity-batch request validator guard is covered by tests, and the README documents the model-free Unicode analysis surfaces. opennlp-grpc-backend-static serves static embedding tables through the opennlp-embeddings module: pure JVM, no native runtime, one immutable model instance shared by every request thread. Models are declared as a published model directory or as explicit safetensors and vocabulary paths; misconfiguration fails at startup. A default_id naming another engine's model no longer crashes the onnx and tei providers in mixed-engine configurations; the composite still validates the id against the union of engines. EmbedText is a new bidirectional streaming RPC for pre-segmented texts: the client streams texts, the server streams one vector per text back in request order, one model per stream. Response writes are gated on transport readiness with a bounded elastic window, and a client that stops reading gets RESOURCE_EXHAUSTED instead of growing the server heap. ModelBundleCache now hands each server thread its own ME decoder over the shared immutable models; the decoders keep per-call state and were shared across request threads before. benchmarks/embedding-throughput carries three methodology-matched harnesses (in-process, gRPC unary and streaming, Python baseline) and measured numbers for the static, onnx, and tei backends through the same wire.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See https://issues.apache.org/jira/browse/OPENNLP-1833
Draft for early feedback on the gRPC expansion. This branch restructures the
opennlp-grpc sandbox module around a v1 document-centric API and builds it out
into a working analysis server.
API / contract
models are rejected with meaningful gRPC status codes instead of being silently ignored
Embeddings
strict factory (model.embedder.backend=onnx|cuda)
ONNX session metadata
at 512 wordpieces, deterministic native resource management
never coexist on the classpath
Chunking (RAG-style segmentation)
thresholds and min/max chunk size constraints
Testing
green on both CPU and GPU build flavors
bit-identical to the corrected SentenceVectorsDL output (see OPENNLP-1836)
Known follow-ups
model jar ships a root-level model.properties; the shaded server currently needs explicit
model.sentence_detector.path / model.tokenizer.path config