Rate limit the delivery API and check the read performance target

CONTRIBUTING.md

@@ -54,7 +54,7 @@ Tests run against SQLite in-process. To run the PostgreSQL tests as well, set `F
 
 The browser tests drive the admin in headless Chromium through Playwright: keyboard-only review on the desk and automated WCAG 2.1 AA checks with axe-core. They run when `FLUENTCMS_TEST_BROWSER` is set. Install the browser once after a build with `pwsh tests/FluentCMS.IntegrationTests/bin/Debug/net10.0/playwright.ps1 install chromium`; CI installs it and sets the variable.
 
-To run the host, `dotnet run --project src/FluentCMS.Host` starts the web host on `http://localhost:5000` with a SQLite database in `fluentcms.db`, and `dotnet run --project src/FluentCMS.Host -- mcp` starts the stdio mode. Configuration comes from environment variables such as `Database__ConnectionString`, `Database__Provider` (`sqlite` or `postgresql`, inferred from the connection string when omitted), `JobQueue__WorkerCount`, `Content__DeletedRetention` (how long a deleted entry can be restored, `30.00:00:00` by default), `Entities__VerifyWithin` (how long a fact stays verified before it is listed as stale, `180.00:00:00` by default), `Delivery__PreviewTokenLifetime` (how long a preview token works unless it is revoked or its change set ends, `7.00:00:00` by default), `Media__Storage` (the storage new uploads go to, `local` by default), `Media__LocalDisk__Root` (where the local storage keeps bytes, `media` under the content root by default) and `Media__MaxBytes` (the largest upload, 50 MB by default; over MCP on HTTP, the server's 30 MB request limit holds an upload to about 22 MB, since base64 grows a file by a third), and from an optional `appsettings.json` next to the binary.
+To run the host, `dotnet run --project src/FluentCMS.Host` starts the web host on `http://localhost:5000` with a SQLite database in `fluentcms.db`, and `dotnet run --project src/FluentCMS.Host -- mcp` starts the stdio mode. Configuration comes from environment variables such as `Database__ConnectionString`, `Database__Provider` (`sqlite` or `postgresql`, inferred from the connection string when omitted), `JobQueue__WorkerCount`, `Content__DeletedRetention` (how long a deleted entry can be restored, `30.00:00:00` by default), `Entities__VerifyWithin` (how long a fact stays verified before it is listed as stale, `180.00:00:00` by default), `Delivery__PreviewTokenLifetime` (how long a preview token works unless it is revoked or its change set ends, `7.00:00:00` by default), `Delivery__RateLimit__PermitLimit` and `Delivery__RateLimit__Window` (how many delivery API requests one client address may make per window, 600 a minute by default), `ReverseProxy__KnownProxies__0` and `ReverseProxy__KnownNetworks__0` (the reverse proxies, by address such as `10.0.0.5` or network such as `10.0.0.0/8`, whose `X-Forwarded-For` names the client that rate limits count, read one hop deep; with none, the header is ignored), `Media__Storage` (the storage new uploads go to, `local` by default), `Media__LocalDisk__Root` (where the local storage keeps bytes, `media` under the content root by default) and `Media__MaxBytes` (the largest upload, 50 MB by default; over MCP on HTTP, the server's 30 MB request limit holds an upload to about 22 MB, since base64 grows a file by a third), and from an optional `appsettings.json` next to the binary.
 
 Agents reach the MCP server (ADR-0007) through either mode. The web host serves streamable HTTP at `/mcp` in stateless mode: each request stands alone, no session id is issued, and a GET answers 405 because the server never opens a stream to the client. `fluentcms mcp` serves the same server over standard input and output to an agent that starts it as a local process, for example with `dotnet run --project src/FluentCMS.Host -- mcp` as its command. It reads the same settings as the web host, so pass it the web host's `Database__ConnectionString` and `DataProtection__KeysDirectory` to work on the same database and key ring. Relative paths resolve against the process's working directory, so give an agent absolute ones. Standard output carries only protocol messages, log lines go to standard error, and the process stops when its standard input closes. The stdio mode runs with the local user's database access, so it is meant for development and single-user setups. Clients on protocol revision 2026-07-28 connect with `server/discover`, earlier ones with `initialize`. Agents read content types, the schema document and change sets with the read-only tools, and propose changes with `open_change_set`, `create_content_type`, `revise_content_type` and `submit_change_set`. A write stores an unpublished version and its operation in the agent's change set together, and the version is not published until a person approves and publishes the set; no tool approves, rejects or publishes. Agents follow review with the read-only desk tools: `get_desk_counts`, `list_desk_queue` and `get_change_set_review`, which see only the change sets the agent opened.
 

docs/api/delivery.md

@@ -86,6 +86,12 @@ References to entity fields resolve to the values the set proposes, so a page pr
 
 A token stops working when it is revoked or expires (`Delivery__PreviewTokenLifetime`, 7 days by default), and when its set is published, rejected or reverted. Every one of these is the same 401 with `preview_token_invalid`. Preview responses carry `Cache-Control: no-store`.
 
+## Limits
+
+Each client address may make 600 requests a minute to the delivery API, `/llms.txt` included (`Delivery__RateLimit__PermitLimit` and `Delivery__RateLimit__Window`). A request beyond that gets a 429 problem with `Retry-After`.
+
+An IPv6 client is counted by its /64 prefix. Behind a reverse proxy, list the proxy in `ReverseProxy__KnownProxies__0` or its network in `ReverseProxy__KnownNetworks__0`, so each client is counted by the address the proxy forwards. Without that, every client shares the proxy's address. `X-Forwarded-For` from anyone else is ignored. Only the address the nearest listed proxy forwards is read, so with a chain such as a CDN in front of nginx, have nginx pass the real client address in its own `X-Forwarded-For`.
+
 ## Errors
 
 Errors are [RFC 9457](https://www.rfc-editor.org/rfc/rfc9457) problem details (`application/problem+json`). `errors` lists each error with a stable `code`:
@@ -95,3 +101,4 @@ Errors are [RFC 9457](https://www.rfc-editor.org/rfc/rfc9457) problem details (`
 | 400 | `limit_invalid`, `parameter_invalid` |
 | 401 | `preview_token_invalid` |
 | 404 | `entry_not_found`, `content_type_not_found`, `endpoint_not_found` |
+| 429 | `rate_limited` |

src/FluentCMS.Api/ApiServiceCollectionExtensions.cs

@@ -1,12 +1,20 @@
+using Microsoft.AspNetCore.Builder;
+using Microsoft.AspNetCore.RateLimiting;
+using Microsoft.Extensions.Configuration;
 using Microsoft.Extensions.DependencyInjection;
 
 namespace FluentCMS.Api;
 
 public static class ApiServiceCollectionExtensions
 {
-    // The delivery API reads through Core services that Infrastructure registers, so it adds none of its own yet.
-    public static IServiceCollection AddFluentCmsApi(this IServiceCollection services)
+    // The delivery API reads through Core services that Infrastructure registers; it adds its rate limit policy (SEC-07).
+    public static IServiceCollection AddFluentCmsApi(this IServiceCollection services, IConfiguration configuration)
     {
+        services.AddOptions<DeliveryRateLimitOptions>()
+            .Bind(configuration.GetSection(DeliveryRateLimitOptions.SectionName))
+            .Validate(options => options.PermitLimit >= 1 && options.Window > TimeSpan.Zero, "Delivery__RateLimit__PermitLimit must be at least 1 and Delivery__RateLimit__Window longer than zero.")
+            .ValidateOnStart();
+        services.AddRateLimiter(options => options.AddPolicy<string, DeliveryRateLimit>(DeliveryRateLimit.PolicyName));
         return services;
     }
 }

src/FluentCMS.Api/DeliveryEndpoints.cs

@@ -21,7 +21,7 @@ public static class DeliveryEndpoints
 
     public static RouteGroupBuilder MapFluentCmsApi(this IEndpointRouteBuilder endpoints)
     {
-        var api = endpoints.MapGroup(Prefix).AllowAnonymous();
+        var api = endpoints.MapGroup(Prefix).AllowAnonymous().RequireRateLimiting(DeliveryRateLimit.PolicyName);
         api.MapGet("/content/{type}", ListAsync);
         api.MapGet("/entries/{id}", GetAsync);
         api.MapGet("/entries/{id}.md", GetMarkdownAsync);
@@ -36,7 +36,7 @@ public static RouteGroupBuilder MapFluentCmsApi(this IEndpointRouteBuilder endpo
             [new Error(DeliveryErrorCodes.EndpointNotFound, $"There is no delivery endpoint at {Prefix}/{path}; see docs/api/delivery.md.")]));
 
         // Where agents look for a site's index first, by the llms.txt convention.
-        endpoints.MapGet("/llms.txt", IndexMarkdownAsync).AllowAnonymous();
+        endpoints.MapGet("/llms.txt", IndexMarkdownAsync).AllowAnonymous().RequireRateLimiting(DeliveryRateLimit.PolicyName);
         return api;
     }
 
@@ -58,6 +58,12 @@ private static async Task<IResult> ListAsync(
         IPublishedContent content,
         CancellationToken cancellationToken)
     {
+        // Not a slug, so no content type has it, and the database is not asked.
+        if (!SchemaNames.IsSlug(type))
+        {
+            return Problem([new Error(SchemaErrorCodes.ContentTypeNotFound, $"There is no content type '{type}' in this site.")]);
+        }
+
         var errors = new List<Error>();
         var size = PublishedQuery.DefaultLimit;
         if (limit is not null && !int.TryParse(limit, NumberStyles.None, CultureInfo.InvariantCulture, out size))

src/FluentCMS.Api/DeliveryRateLimit.cs

@@ -0,0 +1,81 @@
+using System.Globalization;
+using System.Net;
+using System.Net.Sockets;
+using System.Threading.RateLimiting;
+using Microsoft.AspNetCore.Http;
+using Microsoft.AspNetCore.RateLimiting;
+using Microsoft.Extensions.DependencyInjection;
+using Microsoft.Extensions.Options;
+
+namespace FluentCMS.Api;
+
+public sealed class DeliveryRateLimitOptions
+{
+    public const string SectionName = "Delivery:RateLimit";
+
+    public int PermitLimit { get; set; } = 600;
+
+    public TimeSpan Window { get; set; } = TimeSpan.FromMinutes(1);
+}
+
+// Limits the delivery API per client address (SEC-07). Its own rejection answers with a problem, so a throttled client reads the
+// same shape as every other API error, not the plain text the sign-in forms get.
+public sealed class DeliveryRateLimit : IRateLimiterPolicy<string>
+{
+    public const string PolicyName = "delivery";
+
+    public const string RateLimited = "rate_limited";
+
+    public Func<OnRejectedContext, CancellationToken, ValueTask>? OnRejected { get; } = RejectAsync;
+
+    public RateLimitPartition<string> GetPartition(HttpContext httpContext)
+    {
+        var options = httpContext.RequestServices.GetRequiredService<IOptions<DeliveryRateLimitOptions>>().Value;
+        return RateLimitPartition.GetFixedWindowLimiter(
+            ClientKey(httpContext.Connection.RemoteIpAddress),
+            _ => new FixedWindowRateLimiterOptions { PermitLimit = options.PermitLimit, Window = options.Window, QueueLimit = 0 });
+    }
+
+    // An IPv6 client usually holds a whole /64, so it is counted by that prefix; by its full address it could take a fresh one per
+    // request. Behind a trusted proxy the address is the client's, as the host's forwarded headers set it.
+    public static string ClientKey(IPAddress? address)
+    {
+        if (address is null)
+        {
+            return "unknown";
+        }
+
+        if (address.IsIPv4MappedToIPv6)
+        {
+            address = address.MapToIPv4();
+        }
+
+        if (address.AddressFamily != AddressFamily.InterNetworkV6)
+        {
+            return address.ToString();
+        }
+
+        var prefix = address.GetAddressBytes();
+        Array.Clear(prefix, 8, 8);
+        return $"{new IPAddress(prefix)}/64";
+    }
+
+    private static async ValueTask RejectAsync(OnRejectedContext context, CancellationToken cancellationToken)
+    {
+        var http = context.HttpContext;
+        http.Response.StatusCode = StatusCodes.Status429TooManyRequests;
+        var retry = context.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter) ? (int)Math.Ceiling(retryAfter.TotalSeconds) : (int?)null;
+        if (retry is { } seconds)
+        {
+            http.Response.Headers.RetryAfter = seconds.ToString(CultureInfo.InvariantCulture);
+        }
+
+        await Results.Problem(
+            statusCode: StatusCodes.Status429TooManyRequests,
+            detail: retry is { } wait ? $"Too many requests from this address; try again in {wait} seconds." : "Too many requests from this address; try again shortly.",
+            extensions: new Dictionary<string, object?>
+            {
+                ["errors"] = new[] { new { Code = RateLimited, Message = "Too many requests from this address." } },
+            }).ExecuteAsync(http);
+    }
+}

src/FluentCMS.Host/FluentCmsHost.cs

@@ -53,12 +53,14 @@ public static WebApplication BuildWebApplication(string[] args, Action<WebApplic
         builder.Services.AddFluentCmsMcpHttp();
         var provider = AddCookieSignIn(builder.Services, builder.Configuration, builder.Environment);
         builder.Services.AddSignInRateLimit(builder.Configuration);
+        builder.Services.AddReverseProxy(builder.Configuration);
         builder.Services.AddHealthChecks();
         builder.Services.AddLocalization();
         builder.Services.AddRazorComponents().AddInteractiveServerComponents();
         RunJobsAsTheirInitiator(builder.Services);
 
         var app = builder.Build();
+        app.UseReverseProxy();
         app.UseSecurityHeaders(provider.IsConfigured ? [provider.AuthorityOrigin] : []);
         app.UseStatusCodePagesWithReExecute("/not-found", createScopeForStatusCodePages: true);
         KeepStatusCodePagesToPageLoads(app);
@@ -196,7 +198,7 @@ private static void AddFluentCms(IServiceCollection services, IConfiguration con
     {
         services.AddFluentCmsCore();
         services.AddFluentCmsInfrastructure(configuration);
-        services.AddFluentCmsApi();
+        services.AddFluentCmsApi(configuration);
         services.AddFluentCmsAdmin();
 
         // Both modes share one key ring, so whatever the web host protects the stdio host can read.

src/FluentCMS.Host/Security/ReverseProxy.cs

@@ -0,0 +1,71 @@
+using System.Net;
+using Microsoft.AspNetCore.HttpOverrides;
+using Microsoft.Extensions.Options;
+
+namespace FluentCMS.Host.Security;
+
+// The proxies whose X-Forwarded-For the host believes. Without any, the header is ignored, so a client cannot choose the address that
+// rate limits count it by; behind a proxy, every client would otherwise share the proxy's address (SEC-07).
+public sealed class ReverseProxyOptions
+{
+    public const string SectionName = "ReverseProxy";
+
+    // Addresses such as 10.0.0.5.
+    public List<string> KnownProxies { get; set; } = [];
+
+    // Networks in CIDR notation such as 10.0.0.0/8.
+    public List<string> KnownNetworks { get; set; } = [];
+
+    public bool IsConfigured => KnownProxies.Count > 0 || KnownNetworks.Count > 0;
+}
+
+internal sealed class ReverseProxyOptionsValidation : IValidateOptions<ReverseProxyOptions>
+{
+    public ValidateOptionsResult Validate(string? name, ReverseProxyOptions options)
+    {
+        var failures = options.KnownProxies
+            .Where(proxy => !IPAddress.TryParse(proxy, out _))
+            .Select(proxy => $"ReverseProxy__KnownProxies has '{proxy}', which is not an IP address such as 10.0.0.5.")
+            .Concat(options.KnownNetworks
+                .Where(network => !System.Net.IPNetwork.TryParse(network, out _))
+                .Select(network => $"ReverseProxy__KnownNetworks has '{network}', which is not a network such as 10.0.0.0/8."))
+            .ToList();
+
+        return failures.Count == 0 ? ValidateOptionsResult.Success : ValidateOptionsResult.Fail(failures);
+    }
+}
+
+internal static class ReverseProxy
+{
+    public static IServiceCollection AddReverseProxy(this IServiceCollection services, IConfiguration configuration)
+    {
+        services.AddOptions<ReverseProxyOptions>().Bind(configuration.GetSection(ReverseProxyOptions.SectionName)).ValidateOnStart();
+        services.AddSingleton<IValidateOptions<ReverseProxyOptions>, ReverseProxyOptionsValidation>();
+        return services;
+    }
+
+    // Before the rate limiters, so they count the client the trusted proxy names. Only the nearest proxy's entry is taken.
+    public static IApplicationBuilder UseReverseProxy(this IApplicationBuilder app)
+    {
+        var proxy = app.ApplicationServices.GetRequiredService<IOptions<ReverseProxyOptions>>().Value;
+        if (!proxy.IsConfigured)
+        {
+            return app;
+        }
+
+        var forwarded = new ForwardedHeadersOptions { ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto };
+        forwarded.KnownProxies.Clear();
+        forwarded.KnownIPNetworks.Clear();
+        foreach (var address in proxy.KnownProxies)
+        {
+            forwarded.KnownProxies.Add(IPAddress.Parse(address));
+        }
+
+        foreach (var network in proxy.KnownNetworks)
+        {
+            forwarded.KnownIPNetworks.Add(System.Net.IPNetwork.Parse(network));
+        }
+
+        return app.UseForwardedHeaders(forwarded);
+    }
+}

tests/FluentCMS.IntegrationTests/Delivery/DeliveryEndpointsTests.cs

@@ -83,6 +83,20 @@ public async Task An_entry_reads_by_id_and_a_missing_one_is_a_problem_with_404()
         await AssertProblemAsync(unknownType, HttpStatusCode.NotFound, SchemaErrorCodes.ContentTypeNotFound);
     }
 
+    [Theory]
+    [InlineData("Product-Page")]
+    [InlineData("product_page")]
+    public async Task A_type_that_cannot_be_a_slug_is_not_found_without_asking_the_reader(string type)
+    {
+        var content = new StubContent();
+        await using var api = await StubApi.StartAsync(content);
+
+        var response = await api.Client.GetAsync($"/api/v1/content/{type}");
+
+        await AssertProblemAsync(response, HttpStatusCode.NotFound, SchemaErrorCodes.ContentTypeNotFound);
+        Assert.Null(content.Query);
+    }
+
     [Theory]
     [InlineData("limit=ten", DeliveryErrorCodes.LimitInvalid, "'limit'")]
     [InlineData("limit=-1", DeliveryErrorCodes.LimitInvalid, "'limit'")]