Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
137 changes: 137 additions & 0 deletions docs/waf-login-rate-limit-todo.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# TODO: Vercel WAF rate limit on `/v1/auth/login` (deferred)

**Status:** Not yet applied — deferred until we're comfortable making firewall
changes on the `api.cropwatch.io` Vercel project. This is config, not code; it
does not ship with the codebase and must be run against Vercel directly.

**Goal:** Stop DoS / brute-force floods against the login endpoint at Vercel's
edge, *before* they reach the function (and before they trigger the expensive
Supabase password check).

## Why the edge, not the app

The app-level throttler (`src/app.module.ts` `ThrottlerModule` +
`src/v1/common/guards/user-throttler.guard.ts`, plus the `@Throttle(2/60)` on
`POST /v1/auth/login` in `src/v1/auth/auth.controller.ts`) is a **backstop, not
a DoS shield**:

- It runs *inside* the function, so a flood still costs an invocation and still
runs guard code before it can say "no".
- Its counter is in-memory and **per-instance**; Vercel spins up many instances,
each with its own count, so the real limit is much looser than configured and
blocks don't propagate.
- It can't stop distributed attacks.

The WAF sits at Vercel's edge: it blocks **before** the function runs, counts
**globally**, and **Vercel does not bill for blocked traffic**. Platform DDoS
mitigation (L3/L4/L7) is already on for free underneath this.

## The nuance for our architecture (read before setting the number)

The WAF rate-limits **by IP**. That's ideal for an attacker (they hit
`api.cropwatch.io/v1/auth/login` directly from their own IP/botnet), but our
**legit** logins arrive from **Vercel's shared egress IPs** — the web app's
SvelteKit server action (`CropWatch/src/lib/server/auth/login-action.ts`) calls
the API server-side. So a too-tight per-IP limit could clip real users clustered
on Vercel IPs. → **Always log first, read real traffic, then enforce.**

Two more, because login is a JSON API (not a browser page):

- Use **`deny` (403)** or **`rate_limit` (429)** when the limit trips — **not
`challenge`** (an HTML challenge page would break the web app's `fetch` and the
Android widget, which expect JSON).
- WAF counters are **per-region**, so the effective global limit is ~N× the
number. Fine for login, just expected.

## Step 0 — prerequisites (in this repo)

```bash
vercel login
vercel link # link to the api.cropwatch.io project
```

## Step 1 — stage the rule in LOG mode (blocks nothing)

```bash
vercel firewall rules add "Rate limit login" \
--condition '{"type":"path","op":"eq","value":"/v1/auth/login"}' \
--condition '{"type":"method","op":"eq","value":"POST"}' \
--action rate_limit \
--rate-limit-window 60 \
--rate-limit-requests 30 \
--rate-limit-keys ip \
--rate-limit-action log \
--yes

vercel firewall diff # review the staged draft
vercel firewall publish --yes # make it live (log-only, safe)
```

## Step 2 — watch real traffic for ~a day

Get the rule ID (`rule_…`) from `vercel firewall rules list --json`, then open:

```
https://vercel.com/<team>/<project>/firewall/traffic?filter=<ruleId>
```

Check one thing: **is anything legit exceeding 30/min?**

- Only obvious attackers trip it → proceed to Step 3.
- Real users (clustered Vercel IPs) trip it → raise `--rate-limit-requests`.
- **Zero hits when you actually log in** → the WAF is matching the pre-rewrite
path. Change the first condition's `"type":"path"` to `"type":"raw_path"` and
re-publish. (Our `vercel.json` rewrites everything to `/src/main.ts`.)

## Step 3 — enforce

```bash
vercel firewall rules edit "Rate limit login" \
--rate-limit-action deny \
--rate-limit-requests 20 \
--yes
vercel firewall diff && vercel firewall publish --yes
```

If the CLI rejects editing the rate-limit sub-flags, remove and re-add:

```bash
vercel firewall rules remove "Rate limit login" --yes
# then re-run the Step 1 `add` with --rate-limit-action deny
```

Optional: add `--duration 15m` (Pro/Enterprise) so a tripped IP stays blocked
for 15 min instead of resetting each window. Keep the dashboard URL handy for the
first 24h in case a rollback is needed (`--rate-limit-action log` or
`rules disable "Rate limit login"`).

## Optional refinement — zero collateral on legit traffic

To fully separate legit app traffic from attackers: have the web app attach a
**secret header** (server-side only, never in the browser bundle) on its API
calls, and add a higher-priority WAF rule that **`bypass`es** the login rate
limit when that header is present. Then legit app logins are never limited, and
only direct hits (attackers) face the per-IP cap.

- Tradeoff: the Android widget and any direct API callers *would* be subject to
the limit (fine — low volume, distinct IPs), and there's a shared secret to
manage. Keep the bypass narrow (secret header **plus** it never appears client
side).

## Related follow-ups we chose NOT to do in this pass (app-level, code)

If the WAF alone isn't enough later, these are the code-side layers:

- **Per-email login throttle** — key the login throttle on the submitted email
instead of IP, so legit users on shared Vercel IPs never collide and one
account can't be brute-forced fast. (Backstop to the WAF.)
- **Per-account failed-attempt lockout / backoff** — defends against distributed
credential stuffing (one password vs many accounts from many IPs) that IP and
email limits miss.

## References

- Firewall CLI / WAF: https://vercel.com/docs/cli/firewall ,
https://vercel.com/docs/vercel-firewall/vercel-waf/custom-rules
- Rate Limiting SDK (for custom counting/buckets):
https://vercel.com/docs/vercel-firewall/vercel-waf/rate-limiting-sdk
34 changes: 24 additions & 10 deletions src/app.module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,9 @@ import { WaterModule } from './v1/water/water.module';
import { TrafficModule } from './v1/traffic/traffic.module';
import { ServeStaticModule } from '@nestjs/serve-static';
import { join } from 'path';
import { ThrottlerGuard, ThrottlerModule } from '@nestjs/throttler';
import { ThrottlerModule } from '@nestjs/throttler';
import { APP_GUARD } from '@nestjs/core';
import { UserThrottlerGuard } from './v1/common/guards/user-throttler.guard';
import { DevicesModule } from './v1/devices/devices.module';
import { RulesModule } from './v1/rules/rules.module';
import { ReportsModule } from './v1/reports/reports.module';
Expand All @@ -20,6 +21,7 @@ import { GatewayModule } from './v1/gateway/gateway.module';
import { DashboardModule } from './v1/dashboard/dashboard.module';
import { PaymentsModule } from './v1/payments/payments.module';
import { LineModule } from './v1/line/line.module';
import { PushModule } from './v1/push/push.module';
import { CropwatchMcpModule } from './v1/mcp/mcp.module';

@Module({
Expand All @@ -33,20 +35,31 @@ import { CropwatchMcpModule } from './v1/mcp/mcp.module';
ServeStaticModule.forRoot({
rootPath: join(process.cwd(), 'static'),
}),
// Limits are keyed per user (bearer token) by UserThrottlerGuard, not per
// IP — the web app's SSR fans out many requests from a shared pool of Vercel
// egress IPs, so an IP-keyed limit would throttle everyone at once.
// NOTE: the default store is in-memory and per-instance; on Vercel the
// effective limit is (limit x concurrent instances) and blocks don't
// propagate. For hard, distributed enforcement use the Vercel WAF / a shared
// store — tracked separately.
ThrottlerModule.forRoot([
{
// app wide, if you send more than 10 requests in 1 minute, you get a 2-minute ban.
// Burst window: 120 requests / 10s per user. Covers the heaviest
// legitimate client burst — a ~100-device dashboard foreground-resume
// fans out ~100 requests within 15s. Offenders blocked for 30s.
name: 'default',
ttl: 2000,
limit: 2000,
blockDuration: 6000,
ttl: 10_000,
limit: 120,
blockDuration: 30_000,
},
{
// If you send more than 100 requests in 1 minute, you get a 24-hour ban.
// Sustained window: 600 requests / minute per user, with headroom for
// steady polling (relay 30s, per-device refresh). Blocked for 60s on
// breach — no longer a 24h ban.
name: 'long',
ttl: 60000,
limit: 2000,
blockDuration: 86400000, // 24 hours
ttl: 60_000,
limit: 600,
blockDuration: 60_000,
},
]),
DevicesModule,
Expand All @@ -58,9 +71,10 @@ import { CropwatchMcpModule } from './v1/mcp/mcp.module';
DashboardModule,
PaymentsModule,
LineModule,
PushModule,
CropwatchMcpModule,
],
controllers: [AppController],
providers: [AppService, { provide: APP_GUARD, useClass: ThrottlerGuard }],
providers: [AppService, { provide: APP_GUARD, useClass: UserThrottlerGuard }],
})
export class AppModule {}
60 changes: 31 additions & 29 deletions src/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,17 +13,10 @@ import { AllExceptionsFilter } from './v1/common/filters/all-exceptions.filter';
import { STATUS_CODES } from 'http';
import type { Express, NextFunction, Request, Response } from 'express';

// With `trust proxy` pinned to a single hop (Vercel), Express resolves req.ip to
// the authentic client address (the right-most X-Forwarded-For entry Vercel
// appends), so we no longer hand-parse the spoofable left-most XFF entry.
function getRequesterIp(req: Request): string {
const forwardedFor = req.headers['x-forwarded-for'];

if (typeof forwardedFor === 'string' && forwardedFor.trim().length > 0) {
return forwardedFor.split(',')[0].trim();
}

if (Array.isArray(forwardedFor) && forwardedFor.length > 0) {
return forwardedFor[0].split(',')[0].trim();
}

return req.ip || req.socket.remoteAddress || 'unknown';
}

Expand All @@ -35,8 +28,35 @@ async function bootstrap() {
// The Express adapter's getInstance() is typed `any`; pin it once here.
const expressApp = app.getHttpAdapter().getInstance() as Express;

expressApp.set('trust proxy', true);
// Vercel puts exactly one proxy hop in front of the function, so trust only
// that single hop. Express then resolves req.ip to the address Vercel appended
// (the right-most XFF entry) rather than a client-spoofable left-most one — the
// rate-limit tracker and request logs both depend on this being authentic.
expressApp.set('trust proxy', 1);
app.enableCors();

// Register Helmet BEFORE the routes and Swagger below, so every response —
// including the Swagger UI and the /docs-json-* handlers — carries the security
// headers. (It was previously added after Swagger, leaving those routes bare.)
app.use(
helmet({
contentSecurityPolicy: {
directives: {
defaultSrc: ["'self'"],
connectSrc: ["'self'", 'https://cdn.jsdelivr.net'],
styleSrc: [
"'self'",
"'unsafe-inline'",
'https://fonts.googleapis.com',
],
fontSrc: ["'self'", 'https://fonts.gstatic.com'],
imgSrc: ["'self'", 'data:'],
scriptSrc: ["'self'", "'unsafe-inline'", 'https://cdn.jsdelivr.net'],
},
},
}),
);

app.use((req: Request, res: Response, next: NextFunction) => {
const endpoint = req.originalUrl || req.url || 'unknown';
const method = req.method || 'UNKNOWN';
Expand Down Expand Up @@ -161,24 +181,6 @@ Developer notes:
'urls.primaryName': 'v1',
},
});
app.use(
helmet({
contentSecurityPolicy: {
directives: {
defaultSrc: ["'self'"],
connectSrc: ["'self'", 'https://cdn.jsdelivr.net'],
styleSrc: [
"'self'",
"'unsafe-inline'",
'https://fonts.googleapis.com',
],
fontSrc: ["'self'", 'https://fonts.gstatic.com'],
imgSrc: ["'self'", 'data:'],
scriptSrc: ["'self'", "'unsafe-inline'", 'https://cdn.jsdelivr.net'],
},
},
}),
);
await app.listen(process.env.PORT ?? 3000);
}
void bootstrap();
21 changes: 3 additions & 18 deletions src/v1/auth/auth.module.ts
Original file line number Diff line number Diff line change
@@ -1,31 +1,16 @@
import { Module } from '@nestjs/common';
import { ConfigModule, ConfigService } from '@nestjs/config';
import { ConfigModule } from '@nestjs/config';
import { PassportModule } from '@nestjs/passport';
import { JwtModule } from '@nestjs/jwt';
import { JwtAuthGuard } from './guards/jwt.auth.guard';
import { SupabaseStrategy } from './strategies/supabase.strategy';
import { AuthController } from './auth.controller';
import { AuthService } from './auth.service';
import { SupabaseModule } from '../../supabase/supabase.module';

@Module({
imports: [
PassportModule,
ConfigModule,
SupabaseModule,
JwtModule.registerAsync({
useFactory: (configService: ConfigService) => {
return {
global: true,
secret: configService.get<string>('PRIVATE_SUPABASE_JWT_SECRET'),
signOptions: { expiresIn: 40000 },
};
},
inject: [ConfigService],
}),
],
imports: [PassportModule, ConfigModule, SupabaseModule],
providers: [JwtAuthGuard, SupabaseStrategy, AuthService],
exports: [JwtAuthGuard, JwtModule],
exports: [JwtAuthGuard],
controllers: [AuthController],
})
export class AuthModule {}
Loading
Loading