Filter search by field values and references, with text optional for one type

CONTRIBUTING.md

@@ -150,7 +150,7 @@ A check is a versioned definition under a slug: a check type and its settings, a
 
 A submitted change set is checked as the check engine (`SystemInitiator.Checks`, ADR-0012), never as the agent: submit it through `IChangeSetChecks.SubmitAsync`, not by moving it to checks running yourself. While every active check is deterministic, a set of up to 50 operations is checked on the request that submits it; any other set is checked by a `checks.run` job, which reports its progress (PERF-02). Every entry and entity the set creates or updates is checked at the version it proposes, and an LLM-judged check charges its model call to the agent that opened the set (SEC-04). An item the engine cannot read, such as an entry deleted since, is a problem that fails the run. So is a job whose last attempt throws, and a set whose job ended without moving it is failed when it is submitted again, so it can be reopened. A reopened set is fixed in place: a set holds one operation per item, and a later draft of an entry or entity it already proposes, the same change from the same published version, moves that operation on to the new draft and appends `change_set.operation_updated`. Each run names its submission's job, and a run of an earlier submission changes nothing. A revert is never held by its checks: it goes to review with their results (CHG-06). The run, each item's results with their findings and model calls, and the move it decides (checks failed when a check fails, otherwise awaiting review) commit together, numbered per set, with the audit events `change_set.checked` and `check.judged` (CHK-03, CHK-05). Agents read the run of the latest submission and its job with the `get_change_set_checks` tool, which shows a job's state and progress but not the error a worker recorded.
 
-Search (`ISearchIndex` in `FluentCMS.Core.Search`, SRC-01, ADR-0010) indexes each entry's latest version in the same unit of work as the write, as the lowercase words of every string and number in its fields (`SearchText`), so a result appears the moment its write commits. PostgreSQL keeps them in a `tsvector` with a GIN index and SQLite in an FTS5 table, both outside the EF model and created by their migrations; an external engine implements the same interface, registered with `AddFluentCmsExtension<ISearchIndex, T>(ServiceLifetime.Scoped)` when it needs the request's scope. `ISearch` combines the matches with the entry rows, so deleting, restoring and publishing change results without touching the index, follows a match to the entries that reference it, and returns only types the caller may read. It reads the matches a page at a time, `Search__PageSize` (1,000 by default), and drops what the scope, deletions and filters leave out, until enough are left, the index has no more, or it has read `Search__MaxMatches` (10,000). A result that stopped there says it is not complete. Each hit has the state the entry tools show, in review included. A string that is a GUID, such as a reference, is not indexed.
+Search (`ISearchIndex` in `FluentCMS.Core.Search`, SRC-01, ADR-0010) indexes each entry's latest version in the same unit of work as the write, as the lowercase words of every string and number in its fields (`SearchText`), so a result appears the moment its write commits. PostgreSQL keeps them in a `tsvector` with a GIN index and SQLite in an FTS5 table, both outside the EF model and created by their migrations; an external engine implements the same interface, registered with `AddFluentCmsExtension<ISearchIndex, T>(ServiceLifetime.Scoped)` when it needs the request's scope. `ISearch` combines the matches with the entry rows, so deleting, restoring and publishing change results without touching the index, follows a match to the entries that reference it, and returns only types the caller may read. It reads the matches a page at a time, `Search__PageSize` (1,000 by default), and drops what the scope, deletions and filters leave out, until enough are left, the index has no more, or it has read `Search__MaxMatches` (10,000). A result that stopped there says it is not complete. Each hit has the state the entry tools show, in review included. A string that is a GUID, such as a reference, is not indexed. Field filters (`SearchQuery.Fields`) compare one type's latest field values, an entity's by its fact's value, and `SearchQuery.References` keeps entries whose latest version references an id; both are checked against the type before anything is read and applied to each page's candidates. With one type and either filter the text can be left out, and the candidates are then that type's entries.
 
 ## Code style
 

src/FluentCMS.Core/Builtin/Tools/SearchTools.cs

@@ -1,3 +1,4 @@
+using System.Text.Json;
 using System.Text.Json.Nodes;
 using FluentCMS.Core.Content;
 using FluentCMS.Core.Results;
@@ -7,25 +8,58 @@
 
 namespace FluentCMS.Core.Builtin.Tools;
 
-// Finding entries and entities by their words (SRC-01, MCP-01), within the types the agent may read.
+// Finding entries and entities by their words, field values and references (SRC-01, MCP-01), within the types the agent may read.
 public sealed class SearchTools : IToolProvider
 {
+    private const string FieldsForm = "[{\"field\": \"price\", \"operator\": \"less_than\", \"value\": 20}]";
+
     private static readonly string StateNames = string.Join(", ", Enum.GetValues<EntryState>().Select(state => ToolJson.Name(state)));
 
+    private static readonly string OperatorNames = string.Join(", ", Enum.GetValues<FieldOperator>().Select(FieldFilters.Name));
+
+    private static readonly JsonObject FieldsSchema = new()
+    {
+        ["type"] = "array",
+        ["minItems"] = 1,
+        ["items"] = new JsonObject
+        {
+            ["type"] = "object",
+            ["properties"] = new JsonObject
+            {
+                ["field"] = new JsonObject { ["type"] = "string", ["minLength"] = 1 },
+                ["operator"] = new JsonObject { ["enum"] = new JsonArray([.. Enum.GetValues<FieldOperator>().Select(op => (JsonNode)FieldFilters.Name(op))]) },
+                ["value"] = new JsonObject(),
+            },
+            ["required"] = new JsonArray("field", "operator", "value"),
+            ["additionalProperties"] = false,
+        },
+    };
+
     public IReadOnlyList<AgentTool> Tools { get; } =
     [
         new AgentTool(
             "search_entries",
             "Search entries and entities",
             "Finds entries and entities whose latest version has every word of the query, each matched as a prefix, best first: "
             + "'4411' finds a SKU and '441' does too. An entry that references a match is found as well, with the match in 'via', "
-            + "such as the page that shows a matching entity. Narrow the search with types, states or published_only; read a hit "
-            + "with get_entry, or an entity's facts with get_entity_facts.",
+            + "such as the page that shows a matching entity. Narrow the search with types, states or published_only, with fields "
+            + "on one type's field values, such as price less_than 20, or with references to keep entries that reference an id. "
+            + "With one type and fields or references, the query can be left out. Read a hit with get_entry, or an entity's facts "
+            + "with get_entity_facts.",
             ToolSchemas.Object(
-                ToolSchemas.String("query", "The words to find, such as a name, a SKU or part of a sentence.", "4411"),
+                ToolSchemas.String("query", "The words to find, such as a name, a SKU or part of a sentence; optional with one type and fields or references.", "4411", required: false),
                 ToolSchemas.Strings("types", "Content type slugs to search in; leave out for every type you may read.", "product", required: false),
                 ToolSchemas.Strings("states", $"Entry states to keep: {StateNames}.", "published", required: false),
                 ToolSchemas.Flag("published_only", "Only entries that have a published version."),
+                ToolSchemas.Json(
+                    "fields",
+                    $"Filters on the latest version's field values, all of which must hold; they need exactly one type in types. Operators: "
+                    + $"{OperatorNames}. Text, rich text and enum fields take equals and contains, numbers and date-times the comparisons, "
+                    + "booleans and references equals. An entity's field is compared by its fact's value.",
+                    FieldsSchema,
+                    JsonNode.Parse(FieldsForm)!,
+                    required: false),
+                ToolSchemas.Uuid("references", "An entry or entity id; only entries whose latest version references it are kept.", required: false),
                 ToolSchemas.PositiveInteger("limit", $"At most this many hits, up to {SearchQuery.MaxLimit}; {SearchQuery.DefaultLimit} if left out.", 20, required: false)),
             readOnly: true,
             SearchAsync),
@@ -34,12 +68,15 @@ public sealed class SearchTools : IToolProvider
     private static async Task<ToolResult> SearchAsync(ToolInvocation call, CancellationToken cancellationToken)
     {
         var arguments = new ToolArgumentsReader(call.Arguments);
-        var text = arguments.RequiredString("query", "4411");
+        var text = arguments.OptionalString("query", "4411");
         var types = arguments.OptionalStrings("types", "product");
         var states = arguments.OptionalStrings("states", "published");
         var publishedOnly = arguments.Flag("published_only");
         var limit = arguments.OptionalPositiveInteger("limit") ?? SearchQuery.DefaultLimit;
+        var fields = arguments.OptionalArray("fields", FieldsForm);
+        var references = arguments.OptionalGuid("references");
         var errors = new List<Error>(arguments.Errors());
+        var filters = fields is { } items ? FieldFiltersOf(items, errors) : null;
 
         var parsedStates = new List<EntryState>();
         foreach (var name in states ?? [])
@@ -61,8 +98,10 @@ private static async Task<ToolResult> SearchAsync(ToolInvocation call, Cancellat
         }
 
         var results = await call.Services.GetRequiredService<ISearch>().SearchAsync(
-            new SearchQuery(text!)
+            new SearchQuery(text ?? string.Empty)
             {
+                Fields = filters,
+                References = references,
                 ContentTypes = types,
                 States = states is null ? null : parsedStates,
                 PublishedOnly = publishedOnly,
@@ -89,4 +128,36 @@ private static async Task<ToolResult> SearchAsync(ToolInvocation call, Cancellat
                     : "Read a hit with get_entry, or an entity's facts with get_entity_facts. A hit with 'via' references that match.",
         });
     }
+
+    // Each item names a field, an operator by its tool name and a value. Whether the field exists and the value fits its type is
+    // checked against the type by the search.
+    private static List<FieldFilter> FieldFiltersOf(JsonElement items, List<Error> errors)
+    {
+        var filters = new List<FieldFilter>();
+        var position = 0;
+        foreach (var item in items.EnumerateArray())
+        {
+            var at = $"fields[{position++}]";
+            if (item.ValueKind != JsonValueKind.Object
+                || item.EnumerateObject().Any(property => property.Name is not ("field" or "operator" or "value"))
+                || !item.TryGetProperty("field", out var field) || field.ValueKind != JsonValueKind.String || string.IsNullOrWhiteSpace(field.GetString())
+                || !item.TryGetProperty("operator", out var name) || name.ValueKind != JsonValueKind.String
+                || !item.TryGetProperty("value", out var value) || value.ValueKind == JsonValueKind.Null)
+            {
+                errors.Add(new Error(ToolErrorCodes.ArgumentInvalid, $"{at} must be an object with a field, an operator and a value, and nothing else, such as {FieldsForm[1..^1]}."));
+                continue;
+            }
+
+            var operators = Enum.GetValues<FieldOperator>().Where(candidate => FieldFilters.Name(candidate) == name.GetString()).ToList();
+            if (operators.Count != 1)
+            {
+                errors.Add(new Error(ToolErrorCodes.ArgumentInvalid, $"{at}: '{name.GetString()}' is not an operator; use {OperatorNames}."));
+                continue;
+            }
+
+            filters.Add(new FieldFilter(field.GetString()!, operators[0], value.Clone()));
+        }
+
+        return filters;
+    }
 }