From e89bdbfbd2a335816e894c8dc1d02f369b11e161 Mon Sep 17 00:00:00 2001 From: singchia Date: Fri, 1 May 2026 19:42:58 +0800 Subject: [PATCH 01/12] chore(repo): adopt gospec project spec in AGENTS.md Add a priority="0" project_spec block referencing the gospec Go backend SDLC standard (architecture, coding, API, observability, delivery, ops red lines). Existing openskills index preserved at priority="1". Also list gospec in the available skills table. Co-Authored-By: Claude Opus 4.7 (1M context) --- AGENTS.md | 134 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 134 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 00d9bad..a3366d2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,5 +1,133 @@ + + +# 项目规范（gospec） + +> 本项目遵循 [gospec](https://github.com/singchia/gospec) — Go 后端项目 SDLC 全流程规范。 + +## Agent 必读 + +任何编码 / 设计 / API / 数据 / 测试 / CI / 部署 / 监控 / 安全 / 文档 任务，**先按 gospec 规范走**。 + +### 第一步：找到 gospec 任务路由表 + +按以下顺序查找 spec 入口： + +1. `~/.claude/skills/gospec/spec/spec.md`（个人安装，推荐） +2. `.claude/skills/gospec/spec/spec.md`（项目级安装） +3. 上面都不存在 → 重新安装： + ```bash + git clone https://github.com/singchia/gospec ~/.claude/skills/gospec + ``` + +### 第二步：路由 → 加载 + +读 `spec/spec.md` 顶部的"任务路由表"，找到当前任务对应的 1-3 个子文件，**只读必要文件**，不要顺序读完整个 spec。 + +### 第三步：实施 + 自查 + +按子文件指引实施，结束前对照文件末尾的"自查清单"逐项核对。 + +### 第四步：PR 前对照 review 清单 + +提交 PR 前对照 `spec/07-code-review.md` 自查清单。 + +--- + +## 核心约束（无需读 spec 也要遵守） + +> 这些是任何任务都要守的红线。不论 agent 是否加载了完整 spec，都不能违反。 + +### 架构 +- **单服务**：`cmd → server → service → biz → data → model`，禁止跨层调用（`service` 不能直连 `data`） +- **monorepo**：`cmd/` 按 service 切、`internal/` 按 **Bounded Context** 切；跨 BC 禁止直接 import，必须通过 API / 事件 / `internal/pkg/` +- 接口在消费方定义，禁止循环依赖 +- `internal/pkg/`、`model/` 不依赖任何业务层 +- 依赖通过构造函数注入，不使用全局变量 +- 每个目录都被 CODEOWNERS 覆盖 + +### 编码 +- 禁止 `_ = fn()` 忽略错误（确实想丢弃必须注释说明） +- 共享状态必须加锁，测试必须带 `-race` +- 错误用 `%w` 包装；不重复记录（要么处理要么传播） +- 所有涉及 IO 的函数第一个参数为 `context.Context` +- `init()` 仅允许做注册（pprof / metrics collector / driver），禁止做 IO 或可能 panic +- 禁止全局可变变量（只读单例 / collector 除外） +- 避免 `any` / `interface{}` 出现在公共 API 边界（解码 / SDK 适配等不可避免时就近注释） + +### API +- 所有 API 变更先更新 `.proto`，禁止改生成代码 +- Handler 必须有 Swagger 注释：`@Summary`、`@Router`、`@Success` 缺一不可 +- 响应格式统一：`{code, message, data}` +- 破坏性变更走新版本，原版本只允许加非破坏性内容 + +### 测试 +- 新功能必须有单元测试 +- CI 强制启用 `-race` +- E2E 测试必须清理数据 + +### Git +- 提交格式：`(): `（Conventional Commits） +- 禁止提交敏感信息（密码、密钥、token） +- 禁止 force push main/master + +### 构建 / 交付 +- **所有构建 / 产物 / 部署目标必须由根 Makefile 作为唯一入口** +- CI / README / Dockerfile 外层 / 本地开发统一调 `make `，禁止直接 `go build` / `docker build` / `kubectl apply` +- 版本号 / 镜像 tag 用变量注入，禁止硬编码 +- 生产部署 target 必须有审批保护 + +### 可观测性 +- 所有对外服务必须暴露 `/healthz`、`/readyz`、`/metrics` +- 日志结构化（slog / zap）+ `trace_id`，ERROR 包含完整 error chain +- 高基数字段（user_id、email、url）禁止作为 Prometheus label +- 敏感字段禁止明文入日志 + +### 安全 +- 密码必须用 bcrypt / argon2id，禁止 MD5 / SHA1 +- SQL 全部参数化，禁止字符串拼接 +- 密钥禁止进代码仓库 / 镜像 / 日志 +- 容器以非 root 用户运行 +- 多租户接口强制 `tenant_id` 过滤 +- CI 必须包含 `govulncheck` + 依赖 / 镜像漏洞扫描 + +### 运维 +- 任何变更必须有回滚方案 +- 告警规则必须配 Runbook 链接 +- 高风险变更走金丝雀或 feature flag +- P0 / P1 事故必须产出 blameless postmortem + +### 数据存储 +- **MySQL**：生产 schema 变更走 migration 文件；大表用在线 DDL 工具；变更兼容滚动发布（expand-contract） +- **Redis**：所有 key 必须设 TTL；禁止大 key（value > 10KB / 集合 > 5000）；分布式锁必须有 owner 校验 +- **ClickHouse**：必须 Replicated engine；写入必须批量；ORDER BY 从低基数到高基数 +- **InfluxDB**：tag 必须低基数（user_id / url 等禁止做 tag）；bucket 必须有 retention +- PII 字段加密存储，测试环境禁止生产数据明文 + +--- + +## 需求载体选择 + +不是所有变更都要写 PRD。按变更类型选载体（详见 `spec/01-requirement/`）： + +| 变更类型 | 载体 | +|---------|------| +| Bug / 小改 / 配置 / 文档修复 | Issue（issue tracker） | +| 重构 / 升级依赖 / 性能优化（用户不感知） | RFC（`docs/rfc/RFC-XXX-*.md`） | +| 用户可感知的功能 / 业务变更 | PRD（`docs/requirements/PRD-XXX-*.md`） | +| 跨多个 PRD 的战略 | Epic（`docs/requirements/EPIC-XXX-*.md`） | + +--- + +## 输出语言 + +默认中文（代码注释、文档、commit message）。 + +完整规范、所有子主题的具体细节、模板和自查清单见 `spec/spec.md` 的任务路由表。 + + + ## Available Skills @@ -58,6 +186,12 @@ Usage notes: global + +gospec +Go 后端 SDLC 全流程中文规范。覆盖 Bounded Context 切分、Kratos 风格分层（cmd/server/service/biz/data/model）、API 设计、数据模型（MySQL / Redis / ClickHouse / InfluxDB）、测试、CI/CD、日志指标追踪 SLO、认证密钥安全、部署与事故、数据库 migration、PRD/RFC/ADR/HLD 文档。框架中性——Web 框架可选 Kratos / gin / Hertz / chi / echo，规范只约束分层和依赖方向。写或审查 Go 代码时按需加载。Go backend SDLC spec — framework-neutral, load on demand for coding/design/testing/ops/docs. +global + + internal-comms A set of resources to help me write all kinds of internal communications, using the formats that my company likes to use. Claude should use this skill whenever asked to write some sort of internal communications (status reports, leadership updates, 3P updates, company newsletters, FAQs, incident reports, project updates, etc.). From 87c1951ba24e7c1450a16bbe22150c6ee672a78a Mon Sep 17 00:00:00 2001 From: singchia Date: Fri, 1 May 2026 19:43:05 +0800 Subject: [PATCH 02/12] docs(rfc): land RFC-001 cloud-native optimization plan Four-milestone proposal scoping the Frontier / Frontlas cloud-native hardening: M1 P0 bug fixes, M2 production baseline (CRD v1alpha2 + graceful shutdown + non-root), M3 observability (/healthz /readyz /metrics + Prometheus + slog), M4 delivery (Makefile single entry + frontlas Helm + image signing + Conditions status). Status: draft. Each milestone ships as its own PR. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/rfc/RFC-001-cloud-native-optimization.md | 251 ++++++++++++++++++ 1 file changed, 251 insertions(+) create mode 100644 docs/rfc/RFC-001-cloud-native-optimization.md diff --git a/docs/rfc/RFC-001-cloud-native-optimization.md b/docs/rfc/RFC-001-cloud-native-optimization.md new file mode 100644 index 0000000..75fd31a --- /dev/null +++ b/docs/rfc/RFC-001-cloud-native-optimization.md @@ -0,0 +1,251 @@ +# RFC-001: 云原生部署优化专项 + +> 完整规范见 `~/.claude/skills/gospec/spec/01-requirement/technical-rfc.md`。 +> +> 本 RFC 聚焦 Frontier / Frontlas 在 Kubernetes 云原生场景下的部署形态、CRD/Operator 健壮性、可观测性与交付链路。属纯技术重构，用户侧使用方式不变。 + +## 元信息 + +- 状态：草稿 +- 作者：singchia +- 日期：2026-05-01 +- 关联 ADR：待 M2 启动时拆分（CRD v1alpha2 引入策略、TLS 证书生命周期、可观测性指标命名） +- 关联 issue：— + +## 背景 + +Frontier 现已交付 Helm chart、Dockerfile 和基于 kubebuilder 的 Operator（FrontierCluster CRD），但在云原生生产场景下存在以下痛点： + +### 1. 数据正确性 bug（P0） + +代码审计发现 4 处真 bug： + +- `pkg/operator/internal/controller/frontiercluster_deployment.go:114` — TLS Cert/Key volume 错挂成 CA Secret，证书目录里塞的是 CA。 +- `pkg/operator/internal/controller/frontiercluster_tls.go:99` — `getEBCAFromSecret` 把 Secret 读取错误吞成空字符串 + nil err，CA 不存在的报错变成"成功但 CA 为空"。 +- `pkg/operator/api/v1alpha1/frontiercluster_fields.go:117,124` — Operator 自管 Secret 名拼接 `fc.Name + "edgebound-..."`，缺连字符。 +- `pkg/operator/api/v1alpha1/frontiercluster_fields.go:76-78` — Frontlas 的 `fpport`（frontier-plane 端口）硬编码 40012，CRD 里改不动。 + +### 2. 生产可用底座缺失 + +- Frontier deployment **没 liveness、没 readiness、没 preStop、没 terminationGracePeriodSeconds**。长连接型网关在滚动更新时秒断 edge。 +- CRD 不暴露 `Resources / Tolerations / TopologySpread / ImagePullPolicy / ImagePullSecrets / ServiceAccountName / Annotations / PriorityClassName`，生产用户无法做基本的资源治理与调度配置。 +- `Redis.Password` 是 spec 里的明文字段，违反 gospec 安全红线"密钥禁止进代码仓库"，且 `kubectl describe pod` 会打出环境变量明文。 +- 容器以 root 运行，违反 gospec 安全红线。 +- `ImagePullPolicy: PullAlways` 硬编码，启动慢且强依赖 registry。 +- PodAntiAffinity 用了 `RequiredDuringSchedulingIgnoredDuringExecution` 硬反亲和，replicas 超 node 数即不可调度。 + +### 3. 可观测性不达标 + +gospec 红线："所有对外服务必须暴露 `/healthz`、`/readyz`、`/metrics`"。当前： + +- Frontier 完全没有 health 端点；Frontlas 只有 readiness（`/cluster/v1/health`）。 +- 配置结构体里有 `metrics` 字段（`pkg/frontier/config/config.go:247`）但**未实现** Prometheus exporter。 +- MQ 投递、Frontlas RPC 延迟、cmux 路由分布、edge 在线数等关键运行指标全部缺失。 +- 日志未结构化（用 klog），未贯通 trace_id。 + +### 4. 交付链路不规范 + +gospec 红线："所有构建 / 产物 / 部署目标必须由根 Makefile 作为唯一入口"。当前： + +- Helm chart 只有 frontier 模板，**没有 frontlas 模板**，生产用户得自己拼。 +- 镜像无签名、CI 无 govulncheck、无依赖漏洞扫描。 +- Operator 不发 Kubernetes Event；Status 用 Phase（Running/Failed/Pending）而非现代的 `[]metav1.Condition`。 + +### 为什么现在做 + +1. ROADMAP.md 已把 "Helm / Operator / Atlas cluster" 全部勾选为完成，但实际不完整，文档与代码状态不一致，会误导生产用户。 +2. 1.2.4 已发布、当前主分支处于功能稳定期，正适合做横切的非功能性优化，不与新功能 PR 抢 review 资源。 +3. M1 的 4 个 bug 影响 mTLS 用户上线，**已经是阻塞问题**。 + +## 方案 + +分四个里程碑，每个里程碑独立 PR、独立可回滚。 + +### M1 — 数据正确性兜底（1-2 天） + +纯 bug 修复，不改 API、不改 CRD schema： + +| 位置 | 修复 | +|---|---| +| `frontiercluster_deployment.go:114` | Cert/Key volume 改用 `EBTLSOperatorCertKeyNamespacedName()` | +| `frontiercluster_tls.go:99` | `return "", err` 替代 `return "", nil` | +| `frontiercluster_fields.go:117,124` | 拼接修正为 `fc.Name + "-edgebound-..."`；保留旧名查找做软迁移 | +| `frontiercluster_fields.go:64-95` + `types.go:73-76` | `ControlPlane.FrontierPlanePort` 字段补到 CRD spec，与 ControlPlane.Port 解耦 | + +### M2 — 生产可用底座（CRD v1alpha2，1-2 周） + +引入 `v1alpha2`，老的 `v1alpha1` 保留 served=true 直到 M4 完成后下线。CRD 暴露： + +```go +type PodOverrides struct { + Resources *corev1.ResourceRequirements + Tolerations []corev1.Toleration + TopologySpread []corev1.TopologySpreadConstraint + NodeSelector map[string]string + Affinity *corev1.Affinity // 替代单独的 NodeAffinity + PriorityClassName string + ServiceAccountName string + ImagePullSecrets []corev1.LocalObjectReference + ImagePullPolicy corev1.PullPolicy // 默认 IfNotPresent + Annotations map[string]string + Labels map[string]string + SecurityContext *corev1.PodSecurityContext // 强制 nonRoot +} + +type Redis struct { + // ... 旧字段保留 + PasswordSecret *corev1.SecretKeySelector // 新增；与 Password 互斥，优先用此 +} +``` + +代码侧： + +- Frontier 增加 `/healthz`、`/readyz` HTTP endpoint（监听独立调试端口或 cmux 复用 30010）。 +- Frontier signal 处理：SIGTERM → 从 Frontlas 注销自己 → close edge listener → 等存量连接自然结束（`min(graceWindow, terminationGracePeriod-5s)`）→ exit。 +- Operator 给 frontier deployment 加 `livenessProbe`（TCP 30010 或 gRPC health）+ `readinessProbe`（注册到 Frontlas 成功后 ready）+ `preStop`（sleep 让 endpoint 摘除）+ `terminationGracePeriodSeconds: 60` 默认值，可被 PodOverrides 覆盖。 +- 容器 `securityContext.runAsNonRoot: true`，Dockerfile 改 distroless/nonroot UID。 +- Default ImagePullPolicy 改 IfNotPresent。 +- PodAntiAffinity 默认改 `PreferredDuringSchedulingIgnoredDuringExecution`，可被 Affinity 覆盖。 + +### M3 — 可观测性达标（1-2 周） + +按 gospec `10-observability/` 规范： + +- 实现 `pkg/frontier/config/config.go` 里 metrics 字段对应的 Prometheus exporter，导出指标： + - `frontier_edge_connections_total{state="online|disconnected"}` + - `frontier_servicebound_rpc_duration_seconds`（histogram） + - `frontier_frontlas_rpc_duration_seconds` + - `frontier_mq_publish_total{backend, topic, result}` + - `frontier_cmux_route_total{route="grpc|tcp"}` + - 严守 gospec 红线：高基数字段（edge_id、user_id、url）禁止做 label。 +- 替换 klog → `log/slog`，输出 JSON，注入 `trace_id`/`edge_id`/`pod_name`。 +- `/healthz`（liveness：进程存活）、`/readyz`（readiness：Frontlas 已注册 + 关键 listener up）、`/metrics` 三个端点统一暴露。 +- Helm 加 `ServiceMonitor` 模板，开关在 values.yaml。 +- 默认 PrometheusRule：edge 连接异常下跌、Frontlas RPC P99 延迟、MQ 投递失败率。每条告警带 Runbook 链接（M3 末把 Runbook 写到 `docs/runbooks/`）。 + +### M4 — 交付与发布规范（1 周） + +按 gospec `08-delivery/` + `12-operations/` 规范： + +- **Makefile 唯一入口**：`make build`、`make image`、`make helm-package`、`make e2e`、`make release`，CI 一律 `make `，README 同步。 +- Helm chart 补 `templates/frontlas/{deployment,service,configmap,servicemonitor}.yaml` + 可选 redis 子 chart 依赖（`bitnami/redis`）。 +- CI 加 `govulncheck`、`trivy image`、`cosign sign`（Keyless OIDC）。 +- Operator Status 从 `Phase` 改为 `[]metav1.Condition`（Available / Progressing / Degraded），保留 Phase 字段做软迁移直到 v1alpha1 下线。 +- Operator 增加 `record.EventRecorder`，重要状态变更（TLS Secret 缺失、Frontlas not ready、副本不齐）发 K8s Event。 +- 写 `docs/runbooks/frontier-rollback.md` + `docs/runbooks/redis-loss-recovery.md`。 + +## 备选方案 + +| 方案 | 优点 | 缺点 | 是否采用 | +|---|---|---|---| +| **A. 分四个里程碑、各自 PR、CRD 走 v1alpha2** | 风险隔离、可分阶段回滚、bug 修复可立即放出 | 总跨度长，需要 v1alpha1/v1alpha2 共存期 | ✅ | +| B. 一个大 PR 全量重构 | 一次到位 | 评审困难、回滚成本高、阻塞用户上线 | ❌ | +| C. 仅修 M1 bug，其它写 ROADMAP 留坑 | 改动最小 | gospec 红线长期违反、生产用户继续踩坑 | ❌ | +| D. 直接 v1，不走 v1alpha2 | 一步到位 | 字段还在演进、过早承诺兼容会形成包袱 | ❌ | +| E. v1alpha1 加 deprecated 字段就地扩展，不发新版本 | 不引入版本管理负担 | 字段语义混乱、后续退不出去 | ❌ | + +## 影响范围 + +- **代码范围**： + - `pkg/operator/api/v1alpha1/`（M1 修复 + M2 起共存） + - `pkg/operator/api/v1alpha2/`（M2 新增） + - `pkg/operator/internal/controller/`（M1 + M2 + M4） + - `pkg/frontier/`（M2 优雅停机、M3 metrics/log） + - `pkg/frontlas/`（M3 metrics/log 对齐） + - `dist/helm/`（M3 ServiceMonitor、M4 frontlas 模板） + - `images/Dockerfile.*`（M2 distroless / nonroot） + - `Makefile`（M4 重构入口） + - `.github/workflows/`（M4 govulncheck / trivy / cosign） + +- **运行时影响**： + - M1 修复后 mTLS 用户证书目录内容会变（之前是错的），需要在 CHANGELOG 提示用户重启 frontier pod。 + - M2 新增 preStop + terminationGracePeriod 会让滚动更新慢约 60 秒，但避免长连接秒断。 + - M2 切非 root 用户，使用 hostPath 或要求 root 的部署需要在 values.yaml 主动覆盖 SecurityContext。 + - M3 暴露 `/metrics` 默认开启会增加少量 CPU/内存（可关）。 + +- **迁移成本**： + - 已部署 v1alpha1 用户：M2 发布后无强制迁移，v1alpha1 与 v1alpha2 共存至少一个 minor 版本。 + - Helm 用户：M4 后建议切换到 chart 内置的 frontlas 模板。 + +- **回滚预案**： + - M1：单个 PR revert 即可，无 schema 变更。 + - M2：Operator 镜像降级；CRD 不需要回滚，v1alpha2 字段全部 optional 且向后兼容。 + - M3：metrics/healthz endpoint 通过 config 关闭即可不影响业务。 + - M4：Status conditions 与 Phase 同时写，回滚 controller 镜像即恢复 Phase-only 行为。 + +## 风险 + +| 风险 | 影响 | 概率 | 缓解措施 | +|---|---|---|---| +| M1 修复 TLS volume 后老用户重启即生效，mTLS 证书目录变化可能导致连接中断 | 中 | 中 | CHANGELOG 显式说明；建议用户低峰期滚动重启 | +| v1alpha2 conversion webhook 实施复杂 | 中 | 中 | M2 不引入 webhook，依赖字段全部 optional 走默认值；conversion 推迟到 v1beta1 | +| 优雅停机 + 从 Frontlas 注销若 Frontlas 不可达可能阻塞 SIGTERM | 中 | 低 | 注销加 5s 超时，超时后强制关闭 listener | +| Prometheus 指标基数过高拖垮 prom server | 高 | 低 | gospec 红线已禁止高基数 label；M3 加单元测试断言 label set | +| Distroless 镜像缺 shell 排查困难 | 低 | 中 | 提供 `:debug` tag 方案 + 文档化 ephemeral container 排查 | +| Makefile 重构破坏现有 CI | 中 | 低 | M4 先双轨运行（旧 target 加 deprecation 提示）一个版本周期再下线 | + +## 排期 + +- 预计开始：2026-05-01 +- 预计完成：2026-06-19（7 周） + +里程碑节点： + +| 里程碑 | 起止 | 产出 | +|---|---|---| +| M1 | 2026-05-01 → 2026-05-04 | 1 PR，4 个 bug 修复 + 单元测试 | +| M2 | 2026-05-05 → 2026-05-22 | 1 PR，CRD v1alpha2 + 优雅停机 + 探针 + 非 root | +| M3 | 2026-05-25 → 2026-06-12 | 1 PR，metrics + slog + ServiceMonitor + Runbook 初稿 | +| M4 | 2026-06-15 → 2026-06-19 | 1 PR，Makefile + frontlas chart + 镜像签名 + Conditions | + +## 验收标准 + +### M1 +- [ ] 4 个 bug 全部有对应单元测试 +- [ ] `make test` 全绿 +- [ ] mTLS e2e 场景验证证书目录内容正确 + +### M2 +- [ ] `kubectl apply` v1alpha2 sample 可成功 reconcile +- [ ] frontier pod 滚动更新期间 edge 长连接不被秒断（手工验证） +- [ ] 容器内 `id` 命令显示非 root +- [ ] Redis 密码可走 SecretKeySelector，`kubectl describe pod` 不打印明文 + +### M3 +- [ ] `curl :30010/metrics` 返回 Prometheus 格式 +- [ ] `curl :30010/healthz` 进程存活时返回 200 +- [ ] `curl :30010/readyz` 在未注册 Frontlas 时返回 503 +- [ ] 所有 metrics label 经单元测试断言无高基数 +- [ ] 日志为 JSON，含 `trace_id`、`pod_name`、`level` + +### M4 +- [ ] `make build` / `make image` / `make helm-package` / `make e2e` 全部可用 +- [ ] CI 包含 `govulncheck`、`trivy`、`cosign sign` +- [ ] Helm chart `helm install frontier dist/helm` 可一键拉起 frontier + frontlas + redis +- [ ] `kubectl describe fc` 显示 Conditions +- [ ] `kubectl get events` 能看到 Operator 发的事件 + +## 实施任务 + +颗粒度按 1-3 天拆，每个里程碑落到具体 PR： + +### M1（任务 #3） +1. 修 deployment.go:114 TLS volume 错挂 + 单元测试 +2. 修 tls.go:99 吞错误 + 单元测试 +3. 修 fields.go:117/124 命名 + 兼容旧名查找 +4. CRD types 加 `Frontlas.ControlPlane.FrontierPlanePort`，fields.go 用之 + +### M2（任务 #4） +拆为：v1alpha2 类型定义 / Reconciler 切版本路由 / Frontier 优雅停机 / 容器非 root / Redis SecretRef / preStop+probes + +### M3（任务 #5） +拆为：metrics 包搭建 / frontier 接入指标 / frontlas 接入指标 / log 切 slog / health endpoints / ServiceMonitor + 默认 PrometheusRule + Runbook + +### M4（任务 #6） +拆为：Makefile 改造 / Helm frontlas 模板 / CI govulncheck+trivy / cosign 签名 / Status Conditions / EventRecorder + +## 变更记录 + +| 日期 | 变更人 | 变更内容 | 原因 | +|---|---|---|---| +| 2026-05-01 | singchia | 初稿 | 立项 | From 1a6dd31976e1e4bbb8ae9b5022a0006bd3e90a1b Mon Sep 17 00:00:00 2001 From: singchia Date: Fri, 1 May 2026 19:43:22 +0800 Subject: [PATCH 03/12] =?UTF-8?q?fix(operator):=20RFC-001=20M1=20=E2=80=94?= =?UTF-8?q?=20four=20P0=20correctness=20bugs=20in=20CRD/Operator?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1. TLS cert/key volume mounted the CA secret instead of the cert/key secret, so mTLS pods received an empty certificate path. (frontiercluster_deployment.go:114) 2. getEBCAFromSecret silently swallowed read errors as ("", nil), masking apiserver / RBAC failures behind a misleading "CA not found" path. Now wraps with %w and propagates. (frontiercluster_tls.go) 3. Operator-managed TLS secret names lacked a separator ("edgebound-..." → "-edgebound-..."), producing names that clashed with project naming style. (frontiercluster_fields.go) 4. Frontlas frontier-plane port was hard-coded to 40012 with no way to override it via CRD. Add ControlPlane.FrontierPlanePort (default 40012) and wire it through a new FRONTLAS_FRONTIERPLANE_PORT env var into pkg/frontlas/config so the listen addr follows. (types.go, fields.go, deployment.go, frontlas/config/config.go) Add 7 unit tests covering all four fixes (api/v1alpha1 + internal/controller). go.mod toolchain version directive realigned by `go mod tidy` to satisfy local Go 1.24; no dependency changes. Refs: docs/rfc/RFC-001-cloud-native-optimization.md (M1) Co-Authored-By: Claude Opus 4.7 (1M context) --- pkg/frontlas/config/config.go | 8 ++ .../api/v1alpha1/frontiercluster_fields.go | 13 ++-- .../v1alpha1/frontiercluster_fields_test.go | 63 +++++++++++++++ .../api/v1alpha1/frontiercluster_types.go | 7 +- ...frontier.singchia.io_frontierclusters.yaml | 2 + pkg/operator/dist/install.yaml | 2 + pkg/operator/go.mod | 5 +- .../controller/frontiercluster_deployment.go | 14 ++-- .../controller/frontiercluster_tls.go | 2 +- .../controller/frontiercluster_tls_test.go | 77 +++++++++++++++++++ 10 files changed, 177 insertions(+), 16 deletions(-) create mode 100644 pkg/operator/api/v1alpha1/frontiercluster_fields_test.go create mode 100644 pkg/operator/internal/controller/frontiercluster_tls_test.go diff --git a/pkg/frontlas/config/config.go b/pkg/frontlas/config/config.go index 6e29389..0e82061 100644 --- a/pkg/frontlas/config/config.go +++ b/pkg/frontlas/config/config.go @@ -209,6 +209,14 @@ func Parse() (*Configuration, error) { } conf.ControlPlane.Listen.Addr = net.JoinHostPort(host, cpPort) } + fpPort := os.Getenv("FRONTLAS_FRONTIERPLANE_PORT") + if fpPort != "" { + host, _, err := net.SplitHostPort(conf.FrontierManager.Listen.Addr) + if err != nil { + return nil, err + } + conf.FrontierManager.Listen.Addr = net.JoinHostPort(host, fpPort) + } redisType := os.Getenv("REDIS_TYPE") redisAddrs := os.Getenv("REDIS_ADDRS") redisUser := os.Getenv("REDIS_USER") diff --git a/pkg/operator/api/v1alpha1/frontiercluster_fields.go b/pkg/operator/api/v1alpha1/frontiercluster_fields.go index 352d0cd..439ddef 100644 --- a/pkg/operator/api/v1alpha1/frontiercluster_fields.go +++ b/pkg/operator/api/v1alpha1/frontiercluster_fields.go @@ -73,10 +73,13 @@ func (fc *FrontierCluster) FrontlasServicePort() (string, corev1.ServiceType, co cpport.TargetPort = intstr.FromInt32(cpport.Port) fpport := corev1.ServicePort{ - Port: 40012, - TargetPort: intstr.FromInt32(40012), - Name: fc.Name + "-frontierplane", + Port: 40012, + Name: fc.Name + "-frontierplane", } + if fc.Spec.Frontlas.ControlPlane.FrontierPlanePort != 0 { + fpport.Port = int32(fc.Spec.Frontlas.ControlPlane.FrontierPlanePort) + } + fpport.TargetPort = intstr.FromInt32(fpport.Port) // service type serviceType := corev1.ServiceTypeClusterIP if fc.Spec.Frontlas.ControlPlane.ServiceType != "" { @@ -114,14 +117,14 @@ func (fc *FrontierCluster) EBTLSCertKeySecretNamespacedName() types.NamespacedNa // operator ca and secret func (fc *FrontierCluster) EBTLSOperatorCASecretNamespacedName() types.NamespacedName { return types.NamespacedName{ - Name: fc.Name + "edgebound-ca-certificate", + Name: fc.Name + "-edgebound-ca-certificate", Namespace: fc.Namespace, } } func (fc *FrontierCluster) EBTLSOperatorCertKeyNamespacedName() types.NamespacedName { return types.NamespacedName{ - Name: fc.Name + "edgebound-certkey-certificate", + Name: fc.Name + "-edgebound-certkey-certificate", Namespace: fc.Namespace, } } diff --git a/pkg/operator/api/v1alpha1/frontiercluster_fields_test.go b/pkg/operator/api/v1alpha1/frontiercluster_fields_test.go new file mode 100644 index 0000000..bbc1db7 --- /dev/null +++ b/pkg/operator/api/v1alpha1/frontiercluster_fields_test.go @@ -0,0 +1,63 @@ +package v1alpha1 + +import ( + "testing" + + "github.com/stretchr/testify/assert" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +func newCluster(name, ns string) *FrontierCluster { + return &FrontierCluster{ + ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: ns}, + } +} + +// 回归 M1 fix #3：Operator 自管 Secret 名应包含连字符，避免与其它资源命名风格冲突。 +func TestEBTLSOperatorSecretNames_HaveDashSeparator(t *testing.T) { + fc := newCluster("foo", "default") + + caName := fc.EBTLSOperatorCASecretNamespacedName().Name + ckName := fc.EBTLSOperatorCertKeyNamespacedName().Name + + assert.Equal(t, "foo-edgebound-ca-certificate", caName) + assert.Equal(t, "foo-edgebound-certkey-certificate", ckName) + assert.NotEqual(t, caName, ckName, "CA secret 名不应等于 CertKey secret 名") +} + +// 回归 M1 fix #4：fpport 不再硬编码 40012，应可由 ControlPlane.FrontierPlanePort 配置。 +func TestFrontlasServicePort_FrontierPlanePort(t *testing.T) { + t.Run("默认 40012", func(t *testing.T) { + fc := newCluster("foo", "default") + _, _, _, fp := fc.FrontlasServicePort() + assert.Equal(t, int32(40012), fp.Port) + assert.Equal(t, int32(40012), fp.TargetPort.IntVal) + }) + + t.Run("可由 spec 覆盖", func(t *testing.T) { + fc := newCluster("foo", "default") + fc.Spec.Frontlas.ControlPlane.FrontierPlanePort = 50012 + _, _, _, fp := fc.FrontlasServicePort() + assert.Equal(t, int32(50012), fp.Port) + assert.Equal(t, int32(50012), fp.TargetPort.IntVal) + }) + + t.Run("NodePort 模式下 fpport 跟随自定义端口", func(t *testing.T) { + fc := newCluster("foo", "default") + fc.Spec.Frontlas.ControlPlane.FrontierPlanePort = 30412 + fc.Spec.Frontlas.ControlPlane.ServiceType = corev1.ServiceTypeNodePort + _, _, _, fp := fc.FrontlasServicePort() + assert.Equal(t, int32(30412), fp.Port) + assert.Equal(t, int32(30412), fp.NodePort) + }) + + t.Run("cpport 与 fpport 解耦", func(t *testing.T) { + fc := newCluster("foo", "default") + fc.Spec.Frontlas.ControlPlane.Port = 40021 + fc.Spec.Frontlas.ControlPlane.FrontierPlanePort = 40022 + _, _, cp, fp := fc.FrontlasServicePort() + assert.Equal(t, int32(40021), cp.Port) + assert.Equal(t, int32(40022), fp.Port) + }) +} diff --git a/pkg/operator/api/v1alpha1/frontiercluster_types.go b/pkg/operator/api/v1alpha1/frontiercluster_types.go index 315a437..0ccda01 100644 --- a/pkg/operator/api/v1alpha1/frontiercluster_types.go +++ b/pkg/operator/api/v1alpha1/frontiercluster_types.go @@ -70,9 +70,10 @@ type Frontier struct { } type ControlPlane struct { - Port int `json:"port,omitempty"` // control plane for service - ServiceName string `json:"service,omitempty"` - ServiceType corev1.ServiceType `json:"serviceType,omitempty"` // typically edgebound should and default be ClusterIP + Port int `json:"port,omitempty"` // control plane port exposed to service-side callers, default 40011 + FrontierPlanePort int `json:"frontierPlanePort,omitempty"` // frontier-plane port exposed to frontier nodes, default 40012 + ServiceName string `json:"service,omitempty"` + ServiceType corev1.ServiceType `json:"serviceType,omitempty"` // typically should default to ClusterIP } type RedisType string diff --git a/pkg/operator/config/crd/bases/frontier.singchia.io_frontierclusters.yaml b/pkg/operator/config/crd/bases/frontier.singchia.io_frontierclusters.yaml index d1e23bf..ab9125a 100644 --- a/pkg/operator/config/crd/bases/frontier.singchia.io_frontierclusters.yaml +++ b/pkg/operator/config/crd/bases/frontier.singchia.io_frontierclusters.yaml @@ -309,6 +309,8 @@ spec: properties: controlplane: properties: + frontierPlanePort: + type: integer port: type: integer service: diff --git a/pkg/operator/dist/install.yaml b/pkg/operator/dist/install.yaml index e743b20..1bde2f0 100644 --- a/pkg/operator/dist/install.yaml +++ b/pkg/operator/dist/install.yaml @@ -321,6 +321,8 @@ spec: properties: controlplane: properties: + frontierPlanePort: + type: integer port: type: integer service: diff --git a/pkg/operator/go.mod b/pkg/operator/go.mod index dec4c6c..715a2d2 100644 --- a/pkg/operator/go.mod +++ b/pkg/operator/go.mod @@ -1,7 +1,8 @@ module github.com/singchia/frontier/operator -go 1.21 -toolchain go1.24.1 +go 1.23.0 + +toolchain go1.24.13 require ( github.com/onsi/ginkgo/v2 v2.14.0 diff --git a/pkg/operator/internal/controller/frontiercluster_deployment.go b/pkg/operator/internal/controller/frontiercluster_deployment.go index 1590808..402a001 100644 --- a/pkg/operator/internal/controller/frontiercluster_deployment.go +++ b/pkg/operator/internal/controller/frontiercluster_deployment.go @@ -25,9 +25,10 @@ const ( NodeNameEnv = "NODE_NAME" // port for frontier and frontlas - FrontierServiceboundPortEnv = "FRONTIER_SERVICEBOUND_PORT" - FrontierEdgeboundPortEnv = "FRONTIER_EDGEBOUND_PORT" - FrontlasControlPlanePortEnv = "FRONTLAS_CONTROLPLANE_PORT" + FrontierServiceboundPortEnv = "FRONTIER_SERVICEBOUND_PORT" + FrontierEdgeboundPortEnv = "FRONTIER_EDGEBOUND_PORT" + FrontlasControlPlanePortEnv = "FRONTLAS_CONTROLPLANE_PORT" + FrontlasFrontierPlanePortEnv = "FRONTLAS_FRONTIERPLANE_PORT" // tls for frontier FrontierEdgeboundTLSCAMountPath = "/app/conf/edgebound/tls/ca" @@ -111,7 +112,7 @@ func (r *FrontierClusterReconciler) ensureFrontierDeployment(ctx context.Context Name: "tls-secret", VolumeSource: corev1.VolumeSource{ Secret: &corev1.SecretVolumeSource{ - SecretName: fc.EBTLSOperatorCASecretNamespacedName().Name, + SecretName: fc.EBTLSOperatorCertKeyNamespacedName().Name, DefaultMode: &permission, }, }, @@ -230,7 +231,7 @@ func (r *FrontierClusterReconciler) ensureFrontlasDeployment(ctx context.Context image = "singchia/frontlas:1.1.0" } - service, _, cpport, _ := fc.FrontlasServicePort() + service, _, cpport, fpport := fc.FrontlasServicePort() // container container := container.Builder(). @@ -240,6 +241,9 @@ func (r *FrontierClusterReconciler) ensureFrontlasDeployment(ctx context.Context SetEnvs([]corev1.EnvVar{{ Name: FrontlasControlPlanePortEnv, Value: strconv.Itoa(int(cpport.Port)), + }, { + Name: FrontlasFrontierPlanePortEnv, + Value: strconv.Itoa(int(fpport.Port)), }, { Name: FrontlasRedisAddrsEnv, Value: strings.Join(fc.Spec.Frontlas.Redis.Addrs, ","), diff --git a/pkg/operator/internal/controller/frontiercluster_tls.go b/pkg/operator/internal/controller/frontiercluster_tls.go index 9680d06..d23138d 100644 --- a/pkg/operator/internal/controller/frontiercluster_tls.go +++ b/pkg/operator/internal/controller/frontiercluster_tls.go @@ -96,7 +96,7 @@ func getEBCertAndKeyFromSecret(ctx context.Context, getter secret.Getter, secret func getEBCAFromSecret(ctx context.Context, getter secret.Getter, secretName types.NamespacedName) (string, error) { data, err := secret.ReadStringData(ctx, getter, secretName) if err != nil { - return "", nil + return "", fmt.Errorf("read CA secret %s: %w", secretName, err) } if ca, ok := data[tlsCACertName]; !ok || ca == "" { return "", ErrCANotFoundInSecret diff --git a/pkg/operator/internal/controller/frontiercluster_tls_test.go b/pkg/operator/internal/controller/frontiercluster_tls_test.go new file mode 100644 index 0000000..1a3afa4 --- /dev/null +++ b/pkg/operator/internal/controller/frontiercluster_tls_test.go @@ -0,0 +1,77 @@ +package controller + +import ( + "context" + "errors" + "testing" + + "github.com/stretchr/testify/assert" + corev1 "k8s.io/api/core/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +// errSecretGetter 模拟 Get 调用失败（非 NotFound），用于回归 fix #2： +// getEBCAFromSecret 必须把读 Secret 的错误向上传播，禁止吞成 (空字符串, nil)。 +type errSecretGetter struct { + err error +} + +func (e errSecretGetter) GetSecret(ctx context.Context, key client.ObjectKey) (corev1.Secret, error) { + return corev1.Secret{}, e.err +} + +type fakeSecretGetter struct { + secret corev1.Secret +} + +func (f fakeSecretGetter) GetSecret(ctx context.Context, key client.ObjectKey) (corev1.Secret, error) { + if f.secret.Name == key.Name && f.secret.Namespace == key.Namespace { + return f.secret, nil + } + return corev1.Secret{}, &apierrors.StatusError{ErrStatus: metav1.Status{Reason: metav1.StatusReasonNotFound}} +} + +func TestGetEBCAFromSecret_PropagatesReadError(t *testing.T) { + ctx := context.Background() + wantErr := errors.New("boom: api server unreachable") + getter := errSecretGetter{err: wantErr} + + ca, err := getEBCAFromSecret(ctx, getter, types.NamespacedName{Namespace: "ns", Name: "ca"}) + + assert.Empty(t, ca) + assert.Error(t, err, "读 Secret 失败必须向上传播，不能被静默成 nil") + assert.ErrorIs(t, err, wantErr) +} + +func TestGetEBCAFromSecret_MissingCAKey(t *testing.T) { + ctx := context.Background() + getter := fakeSecretGetter{ + secret: corev1.Secret{ + ObjectMeta: metav1.ObjectMeta{Name: "ca", Namespace: "ns"}, + Data: map[string][]byte{"other.crt": []byte("x")}, + }, + } + + ca, err := getEBCAFromSecret(ctx, getter, types.NamespacedName{Namespace: "ns", Name: "ca"}) + + assert.Empty(t, ca) + assert.ErrorIs(t, err, ErrCANotFoundInSecret) +} + +func TestGetEBCAFromSecret_Happy(t *testing.T) { + ctx := context.Background() + getter := fakeSecretGetter{ + secret: corev1.Secret{ + ObjectMeta: metav1.ObjectMeta{Name: "ca", Namespace: "ns"}, + Data: map[string][]byte{"ca.crt": []byte("PEM")}, + }, + } + + ca, err := getEBCAFromSecret(ctx, getter, types.NamespacedName{Namespace: "ns", Name: "ca"}) + + assert.NoError(t, err) + assert.Equal(t, "PEM", ca) +} From 122ebba1cf920a05900fee17052aae643b2d5563 Mon Sep 17 00:00:00 2001 From: singchia Date: Sat, 2 May 2026 00:09:05 +0800 Subject: [PATCH 04/12] feat(operator): extend v1alpha1 schema with PodOverrides + Redis.PasswordSecret MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add a generic PodOverrides struct that lets users tune the most common production-grade Pod-level fields without breaking v1alpha1 — Resources, NodeSelector, Tolerations, TopologySpreadConstraints, Affinity, PriorityClassName, ServiceAccountName, ImagePullSecrets, ImagePullPolicy, Annotations, Labels, PodSecurityContext, ContainerSecurityContext, TerminationGracePeriodSeconds, LivenessProbe, ReadinessProbe, Lifecycle. Embed PodOverrides as Frontier.Pod and Frontlas.Pod. The legacy NodeAffinity field stays for backwards compat but Pod.Affinity, when set, fully replaces it. Add Redis.PasswordSecret (corev1.SecretKeySelector). Takes precedence over the existing Redis.Password plaintext when both are set; the plaintext field is kept for backwards compat but marked deprecated. Resolves the gospec security red line "密钥禁止进代码仓库 / 镜像 / 日志" — operator no longer needs to materialize the password into spec env. zz_generated.deepcopy.go and CRD schema regenerated by controller-gen. v1alpha1 → v1alpha1: existing CRs keep working (all new fields optional). Version bump to v1beta1 deferred to M4 to bundle with Status conditions. Refs: docs/rfc/RFC-001-cloud-native-optimization.md (M2) Co-Authored-By: Claude Opus 4.7 (1M context) --- .../api/v1alpha1/frontiercluster_types.go | 49 +- .../api/v1alpha1/zz_generated.deepcopy.go | 102 + ...frontier.singchia.io_frontierclusters.yaml | 4584 ++++++++++++++++- 3 files changed, 4543 insertions(+), 192 deletions(-) diff --git a/pkg/operator/api/v1alpha1/frontiercluster_types.go b/pkg/operator/api/v1alpha1/frontiercluster_types.go index 0ccda01..865ea6d 100644 --- a/pkg/operator/api/v1alpha1/frontiercluster_types.go +++ b/pkg/operator/api/v1alpha1/frontiercluster_types.go @@ -24,6 +24,30 @@ import ( // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! // NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. +// PodOverrides 集中暴露 Frontier / Frontlas Pod 的可配置项， +// 让用户在不破坏 v1alpha1 schema 的前提下，按生产需要覆盖资源、调度、 +// 安全上下文、探针、生命周期等关键字段。所有字段都是 optional—— +// 不填走 operator 内置默认值（见 internal/controller/podoverrides.go）。 +type PodOverrides struct { + Resources *corev1.ResourceRequirements `json:"resources,omitempty"` + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + Tolerations []corev1.Toleration `json:"tolerations,omitempty"` + TopologySpreadConstraints []corev1.TopologySpreadConstraint `json:"topologySpreadConstraints,omitempty"` + Affinity *corev1.Affinity `json:"affinity,omitempty"` + PriorityClassName string `json:"priorityClassName,omitempty"` + ServiceAccountName string `json:"serviceAccountName,omitempty"` + ImagePullSecrets []corev1.LocalObjectReference `json:"imagePullSecrets,omitempty"` + ImagePullPolicy corev1.PullPolicy `json:"imagePullPolicy,omitempty"` + Annotations map[string]string `json:"annotations,omitempty"` + Labels map[string]string `json:"labels,omitempty"` + PodSecurityContext *corev1.PodSecurityContext `json:"podSecurityContext,omitempty"` + ContainerSecurityContext *corev1.SecurityContext `json:"containerSecurityContext,omitempty"` + TerminationGracePeriodSeconds *int64 `json:"terminationGracePeriodSeconds,omitempty"` + LivenessProbe *corev1.Probe `json:"livenessProbe,omitempty"` + ReadinessProbe *corev1.Probe `json:"readinessProbe,omitempty"` + Lifecycle *corev1.Lifecycle `json:"lifecycle,omitempty"` +} + // TLS is the configuration used to set up TLS encryption type TLS struct { Enabled bool `json:"enabled"` @@ -67,6 +91,11 @@ type Frontier struct { Edgebound Edgebound `json:"edgebound"` Image string `json:"image,omitempty"` // default singchia/frontier:1.1.0 NodeAffinity corev1.NodeAffinity `json:"nodeAffinity,omitempty"` + // Pod is the optional set of generic Pod-level overrides applied to the + // frontier Deployment. Fields here win over operator defaults; fields not + // provided fall back to defaults documented in PodOverrides. + // When Pod.Affinity is set it fully replaces the legacy NodeAffinity above. + Pod PodOverrides `json:"pod,omitempty"` } type ControlPlane struct { @@ -85,12 +114,17 @@ const ( ) type Redis struct { - Addrs []string `json:"addrs"` - DB int `json:"db,omitempty"` - User string `json:"user,omitempty"` - Password string `json:"password,omitempty"` - RedisType RedisType `json:"redisType"` - MasterName string `json:"masterName,omitempty"` + Addrs []string `json:"addrs"` + DB int `json:"db,omitempty"` + User string `json:"user,omitempty"` + // Password 是密码明文，会被原样写进 Pod 环境变量。 + // Deprecated: 生产场景请改用 PasswordSecret，避免密钥进 spec / event / describe 输出。 + Password string `json:"password,omitempty"` + // PasswordSecret 引用一个 Secret 中的字段作为 Redis 密码来源。 + // 设置后优先级高于 Password，env 通过 valueFrom.secretKeyRef 注入。 + PasswordSecret *corev1.SecretKeySelector `json:"passwordSecret,omitempty"` + RedisType RedisType `json:"redisType"` + MasterName string `json:"masterName,omitempty"` } type Frontlas struct { @@ -99,6 +133,9 @@ type Frontlas struct { NodeAffinity corev1.NodeAffinity `json:"nodeAffinity,omitempty"` Image string `json:"image,omitempty"` Redis Redis `json:"redis"` + // Pod is the optional set of generic Pod-level overrides applied to the + // frontlas Deployment. See Frontier.Pod for semantics. + Pod PodOverrides `json:"pod,omitempty"` } // FrontierClusterSpec defines the desired state of FrontierCluster diff --git a/pkg/operator/api/v1alpha1/zz_generated.deepcopy.go b/pkg/operator/api/v1alpha1/zz_generated.deepcopy.go index b9c3990..bef374a 100644 --- a/pkg/operator/api/v1alpha1/zz_generated.deepcopy.go +++ b/pkg/operator/api/v1alpha1/zz_generated.deepcopy.go @@ -62,6 +62,7 @@ func (in *Frontier) DeepCopyInto(out *Frontier) { out.Servicebound = in.Servicebound in.Edgebound.DeepCopyInto(&out.Edgebound) in.NodeAffinity.DeepCopyInto(&out.NodeAffinity) + in.Pod.DeepCopyInto(&out.Pod) } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Frontier. @@ -171,6 +172,7 @@ func (in *Frontlas) DeepCopyInto(out *Frontlas) { out.ControlPlane = in.ControlPlane in.NodeAffinity.DeepCopyInto(&out.NodeAffinity) in.Redis.DeepCopyInto(&out.Redis) + in.Pod.DeepCopyInto(&out.Pod) } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Frontlas. @@ -183,6 +185,101 @@ func (in *Frontlas) DeepCopy() *Frontlas { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PodOverrides) DeepCopyInto(out *PodOverrides) { + *out = *in + if in.Resources != nil { + in, out := &in.Resources, &out.Resources + *out = new(v1.ResourceRequirements) + (*in).DeepCopyInto(*out) + } + if in.NodeSelector != nil { + in, out := &in.NodeSelector, &out.NodeSelector + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.Tolerations != nil { + in, out := &in.Tolerations, &out.Tolerations + *out = make([]v1.Toleration, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.TopologySpreadConstraints != nil { + in, out := &in.TopologySpreadConstraints, &out.TopologySpreadConstraints + *out = make([]v1.TopologySpreadConstraint, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.Affinity != nil { + in, out := &in.Affinity, &out.Affinity + *out = new(v1.Affinity) + (*in).DeepCopyInto(*out) + } + if in.ImagePullSecrets != nil { + in, out := &in.ImagePullSecrets, &out.ImagePullSecrets + *out = make([]v1.LocalObjectReference, len(*in)) + copy(*out, *in) + } + if in.Annotations != nil { + in, out := &in.Annotations, &out.Annotations + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.Labels != nil { + in, out := &in.Labels, &out.Labels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.PodSecurityContext != nil { + in, out := &in.PodSecurityContext, &out.PodSecurityContext + *out = new(v1.PodSecurityContext) + (*in).DeepCopyInto(*out) + } + if in.ContainerSecurityContext != nil { + in, out := &in.ContainerSecurityContext, &out.ContainerSecurityContext + *out = new(v1.SecurityContext) + (*in).DeepCopyInto(*out) + } + if in.TerminationGracePeriodSeconds != nil { + in, out := &in.TerminationGracePeriodSeconds, &out.TerminationGracePeriodSeconds + *out = new(int64) + **out = **in + } + if in.LivenessProbe != nil { + in, out := &in.LivenessProbe, &out.LivenessProbe + *out = new(v1.Probe) + (*in).DeepCopyInto(*out) + } + if in.ReadinessProbe != nil { + in, out := &in.ReadinessProbe, &out.ReadinessProbe + *out = new(v1.Probe) + (*in).DeepCopyInto(*out) + } + if in.Lifecycle != nil { + in, out := &in.Lifecycle, &out.Lifecycle + *out = new(v1.Lifecycle) + (*in).DeepCopyInto(*out) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PodOverrides. +func (in *PodOverrides) DeepCopy() *PodOverrides { + if in == nil { + return nil + } + out := new(PodOverrides) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *Redis) DeepCopyInto(out *Redis) { *out = *in @@ -191,6 +288,11 @@ func (in *Redis) DeepCopyInto(out *Redis) { *out = make([]string, len(*in)) copy(*out, *in) } + if in.PasswordSecret != nil { + in, out := &in.PasswordSecret, &out.PasswordSecret + *out = new(v1.SecretKeySelector) + (*in).DeepCopyInto(*out) + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Redis. diff --git a/pkg/operator/config/crd/bases/frontier.singchia.io_frontierclusters.yaml b/pkg/operator/config/crd/bases/frontier.singchia.io_frontierclusters.yaml index ab9125a..41103db 100644 --- a/pkg/operator/config/crd/bases/frontier.singchia.io_frontierclusters.yaml +++ b/pkg/operator/config/crd/bases/frontier.singchia.io_frontierclusters.yaml @@ -288,120 +288,4268 @@ spec: type: object x-kubernetes-map-type: atomic type: object - replicas: - type: integer - servicebound: + pod: + description: |- + Pod is the optional set of generic Pod-level overrides applied to the + frontier Deployment. Fields here win over operator defaults; fields not + provided fall back to defaults documented in PodOverrides. + When Pod.Affinity is set it fully replaces the legacy NodeAffinity above. properties: - port: + affinity: + description: Affinity is a group of affinity scheduling rules. + properties: + nodeAffinity: + description: Describes node affinity scheduling rules + for the pod. + properties: + preferredDuringSchedulingIgnoredDuringExecution: + description: |- + The scheduler will prefer to schedule pods to nodes that satisfy + the affinity expressions specified by this field, but it may choose + a node that violates one or more of the expressions. The node that is + most preferred is the one with the greatest sum of weights, i.e. + for each node that meets all of the scheduling requirements (resource + request, requiredDuringScheduling affinity expressions, etc.), + compute a sum by iterating through the elements of this field and adding + "weight" to the sum if the node matches the corresponding matchExpressions; the + node(s) with the highest sum are the most preferred. + items: + description: |- + An empty preferred scheduling term matches all objects with implicit weight 0 + (i.e. it's a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op). + properties: + preference: + description: A node selector term, associated + with the corresponding weight. + properties: + matchExpressions: + description: A list of node selector requirements + by node's labels. + items: + description: |- + A node selector requirement is a selector that contains values, a key, and an operator + that relates the key and values. + properties: + key: + description: The label key that the + selector applies to. + type: string + operator: + description: |- + Represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt. + type: string + values: + description: |- + An array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. If the operator is Gt or Lt, the values + array must have a single element, which will be interpreted as an integer. + This array is replaced during a strategic merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchFields: + description: A list of node selector requirements + by node's fields. + items: + description: |- + A node selector requirement is a selector that contains values, a key, and an operator + that relates the key and values. + properties: + key: + description: The label key that the + selector applies to. + type: string + operator: + description: |- + Represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt. + type: string + values: + description: |- + An array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. If the operator is Gt or Lt, the values + array must have a single element, which will be interpreted as an integer. + This array is replaced during a strategic merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + type: object + x-kubernetes-map-type: atomic + weight: + description: Weight associated with matching + the corresponding nodeSelectorTerm, in the + range 1-100. + format: int32 + type: integer + required: + - preference + - weight + type: object + type: array + requiredDuringSchedulingIgnoredDuringExecution: + description: |- + If the affinity requirements specified by this field are not met at + scheduling time, the pod will not be scheduled onto the node. + If the affinity requirements specified by this field cease to be met + at some point during pod execution (e.g. due to an update), the system + may or may not try to eventually evict the pod from its node. + properties: + nodeSelectorTerms: + description: Required. A list of node selector + terms. The terms are ORed. + items: + description: |- + A null or empty node selector term matches no objects. The requirements of + them are ANDed. + The TopologySelectorTerm type implements a subset of the NodeSelectorTerm. + properties: + matchExpressions: + description: A list of node selector requirements + by node's labels. + items: + description: |- + A node selector requirement is a selector that contains values, a key, and an operator + that relates the key and values. + properties: + key: + description: The label key that the + selector applies to. + type: string + operator: + description: |- + Represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt. + type: string + values: + description: |- + An array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. If the operator is Gt or Lt, the values + array must have a single element, which will be interpreted as an integer. + This array is replaced during a strategic merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchFields: + description: A list of node selector requirements + by node's fields. + items: + description: |- + A node selector requirement is a selector that contains values, a key, and an operator + that relates the key and values. + properties: + key: + description: The label key that the + selector applies to. + type: string + operator: + description: |- + Represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt. + type: string + values: + description: |- + An array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. If the operator is Gt or Lt, the values + array must have a single element, which will be interpreted as an integer. + This array is replaced during a strategic merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + type: object + x-kubernetes-map-type: atomic + type: array + required: + - nodeSelectorTerms + type: object + x-kubernetes-map-type: atomic + type: object + podAffinity: + description: Describes pod affinity scheduling rules (e.g. + co-locate this pod in the same node, zone, etc. as some + other pod(s)). + properties: + preferredDuringSchedulingIgnoredDuringExecution: + description: |- + The scheduler will prefer to schedule pods to nodes that satisfy + the affinity expressions specified by this field, but it may choose + a node that violates one or more of the expressions. The node that is + most preferred is the one with the greatest sum of weights, i.e. + for each node that meets all of the scheduling requirements (resource + request, requiredDuringScheduling affinity expressions, etc.), + compute a sum by iterating through the elements of this field and adding + "weight" to the sum if the node has pods which matches the corresponding podAffinityTerm; the + node(s) with the highest sum are the most preferred. + items: + description: The weights of all of the matched WeightedPodAffinityTerm + fields are added per-node to find the most preferred + node(s) + properties: + podAffinityTerm: + description: Required. A pod affinity term, + associated with the corresponding weight. + properties: + labelSelector: + description: |- + A label query over a set of resources, in this case pods. + If it's null, this PodAffinityTerm matches with no Pods. + properties: + matchExpressions: + description: matchExpressions is a list + of label selector requirements. The + requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label + key that the selector applies + to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + matchLabelKeys: + description: |- + MatchLabelKeys is a set of pod label keys to select which pods will + be taken into consideration. The keys are used to lookup values from the + incoming pod labels, those key-value labels are merged with `LabelSelector` as `key in (value)` + to select the group of existing pods which pods will be taken into consideration + for the incoming pod's pod (anti) affinity. Keys that don't exist in the incoming + pod labels will be ignored. The default value is empty. + The same key is forbidden to exist in both MatchLabelKeys and LabelSelector. + Also, MatchLabelKeys cannot be set when LabelSelector isn't set. + This is an alpha field and requires enabling MatchLabelKeysInPodAffinity feature gate. + items: + type: string + type: array + x-kubernetes-list-type: atomic + mismatchLabelKeys: + description: |- + MismatchLabelKeys is a set of pod label keys to select which pods will + be taken into consideration. The keys are used to lookup values from the + incoming pod labels, those key-value labels are merged with `LabelSelector` as `key notin (value)` + to select the group of existing pods which pods will be taken into consideration + for the incoming pod's pod (anti) affinity. Keys that don't exist in the incoming + pod labels will be ignored. The default value is empty. + The same key is forbidden to exist in both MismatchLabelKeys and LabelSelector. + Also, MismatchLabelKeys cannot be set when LabelSelector isn't set. + This is an alpha field and requires enabling MatchLabelKeysInPodAffinity feature gate. + items: + type: string + type: array + x-kubernetes-list-type: atomic + namespaceSelector: + description: |- + A label query over the set of namespaces that the term applies to. + The term is applied to the union of the namespaces selected by this field + and the ones listed in the namespaces field. + null selector and null or empty namespaces list means "this pod's namespace". + An empty selector ({}) matches all namespaces. + properties: + matchExpressions: + description: matchExpressions is a list + of label selector requirements. The + requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label + key that the selector applies + to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + namespaces: + description: |- + namespaces specifies a static list of namespace names that the term applies to. + The term is applied to the union of the namespaces listed in this field + and the ones selected by namespaceSelector. + null or empty namespaces list and null namespaceSelector means "this pod's namespace". + items: + type: string + type: array + topologyKey: + description: |- + This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching + the labelSelector in the specified namespaces, where co-located is defined as running on a node + whose value of the label with key topologyKey matches that of any node on which any of the + selected pods is running. + Empty topologyKey is not allowed. + type: string + required: + - topologyKey + type: object + weight: + description: |- + weight associated with matching the corresponding podAffinityTerm, + in the range 1-100. + format: int32 + type: integer + required: + - podAffinityTerm + - weight + type: object + type: array + requiredDuringSchedulingIgnoredDuringExecution: + description: |- + If the affinity requirements specified by this field are not met at + scheduling time, the pod will not be scheduled onto the node. + If the affinity requirements specified by this field cease to be met + at some point during pod execution (e.g. due to a pod label update), the + system may or may not try to eventually evict the pod from its node. + When there are multiple elements, the lists of nodes corresponding to each + podAffinityTerm are intersected, i.e. all terms must be satisfied. + items: + description: |- + Defines a set of pods (namely those matching the labelSelector + relative to the given namespace(s)) that this pod should be + co-located (affinity) or not co-located (anti-affinity) with, + where co-located is defined as running on a node whose value of + the label with key matches that of any node on which + a pod of the set of pods is running + properties: + labelSelector: + description: |- + A label query over a set of resources, in this case pods. + If it's null, this PodAffinityTerm matches with no Pods. + properties: + matchExpressions: + description: matchExpressions is a list + of label selector requirements. The requirements + are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key + that the selector applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + matchLabelKeys: + description: |- + MatchLabelKeys is a set of pod label keys to select which pods will + be taken into consideration. The keys are used to lookup values from the + incoming pod labels, those key-value labels are merged with `LabelSelector` as `key in (value)` + to select the group of existing pods which pods will be taken into consideration + for the incoming pod's pod (anti) affinity. Keys that don't exist in the incoming + pod labels will be ignored. The default value is empty. + The same key is forbidden to exist in both MatchLabelKeys and LabelSelector. + Also, MatchLabelKeys cannot be set when LabelSelector isn't set. + This is an alpha field and requires enabling MatchLabelKeysInPodAffinity feature gate. + items: + type: string + type: array + x-kubernetes-list-type: atomic + mismatchLabelKeys: + description: |- + MismatchLabelKeys is a set of pod label keys to select which pods will + be taken into consideration. The keys are used to lookup values from the + incoming pod labels, those key-value labels are merged with `LabelSelector` as `key notin (value)` + to select the group of existing pods which pods will be taken into consideration + for the incoming pod's pod (anti) affinity. Keys that don't exist in the incoming + pod labels will be ignored. The default value is empty. + The same key is forbidden to exist in both MismatchLabelKeys and LabelSelector. + Also, MismatchLabelKeys cannot be set when LabelSelector isn't set. + This is an alpha field and requires enabling MatchLabelKeysInPodAffinity feature gate. + items: + type: string + type: array + x-kubernetes-list-type: atomic + namespaceSelector: + description: |- + A label query over the set of namespaces that the term applies to. + The term is applied to the union of the namespaces selected by this field + and the ones listed in the namespaces field. + null selector and null or empty namespaces list means "this pod's namespace". + An empty selector ({}) matches all namespaces. + properties: + matchExpressions: + description: matchExpressions is a list + of label selector requirements. The requirements + are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key + that the selector applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + namespaces: + description: |- + namespaces specifies a static list of namespace names that the term applies to. + The term is applied to the union of the namespaces listed in this field + and the ones selected by namespaceSelector. + null or empty namespaces list and null namespaceSelector means "this pod's namespace". + items: + type: string + type: array + topologyKey: + description: |- + This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching + the labelSelector in the specified namespaces, where co-located is defined as running on a node + whose value of the label with key topologyKey matches that of any node on which any of the + selected pods is running. + Empty topologyKey is not allowed. + type: string + required: + - topologyKey + type: object + type: array + type: object + podAntiAffinity: + description: Describes pod anti-affinity scheduling rules + (e.g. avoid putting this pod in the same node, zone, + etc. as some other pod(s)). + properties: + preferredDuringSchedulingIgnoredDuringExecution: + description: |- + The scheduler will prefer to schedule pods to nodes that satisfy + the anti-affinity expressions specified by this field, but it may choose + a node that violates one or more of the expressions. The node that is + most preferred is the one with the greatest sum of weights, i.e. + for each node that meets all of the scheduling requirements (resource + request, requiredDuringScheduling anti-affinity expressions, etc.), + compute a sum by iterating through the elements of this field and adding + "weight" to the sum if the node has pods which matches the corresponding podAffinityTerm; the + node(s) with the highest sum are the most preferred. + items: + description: The weights of all of the matched WeightedPodAffinityTerm + fields are added per-node to find the most preferred + node(s) + properties: + podAffinityTerm: + description: Required. A pod affinity term, + associated with the corresponding weight. + properties: + labelSelector: + description: |- + A label query over a set of resources, in this case pods. + If it's null, this PodAffinityTerm matches with no Pods. + properties: + matchExpressions: + description: matchExpressions is a list + of label selector requirements. The + requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label + key that the selector applies + to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + matchLabelKeys: + description: |- + MatchLabelKeys is a set of pod label keys to select which pods will + be taken into consideration. The keys are used to lookup values from the + incoming pod labels, those key-value labels are merged with `LabelSelector` as `key in (value)` + to select the group of existing pods which pods will be taken into consideration + for the incoming pod's pod (anti) affinity. Keys that don't exist in the incoming + pod labels will be ignored. The default value is empty. + The same key is forbidden to exist in both MatchLabelKeys and LabelSelector. + Also, MatchLabelKeys cannot be set when LabelSelector isn't set. + This is an alpha field and requires enabling MatchLabelKeysInPodAffinity feature gate. + items: + type: string + type: array + x-kubernetes-list-type: atomic + mismatchLabelKeys: + description: |- + MismatchLabelKeys is a set of pod label keys to select which pods will + be taken into consideration. The keys are used to lookup values from the + incoming pod labels, those key-value labels are merged with `LabelSelector` as `key notin (value)` + to select the group of existing pods which pods will be taken into consideration + for the incoming pod's pod (anti) affinity. Keys that don't exist in the incoming + pod labels will be ignored. The default value is empty. + The same key is forbidden to exist in both MismatchLabelKeys and LabelSelector. + Also, MismatchLabelKeys cannot be set when LabelSelector isn't set. + This is an alpha field and requires enabling MatchLabelKeysInPodAffinity feature gate. + items: + type: string + type: array + x-kubernetes-list-type: atomic + namespaceSelector: + description: |- + A label query over the set of namespaces that the term applies to. + The term is applied to the union of the namespaces selected by this field + and the ones listed in the namespaces field. + null selector and null or empty namespaces list means "this pod's namespace". + An empty selector ({}) matches all namespaces. + properties: + matchExpressions: + description: matchExpressions is a list + of label selector requirements. The + requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label + key that the selector applies + to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + namespaces: + description: |- + namespaces specifies a static list of namespace names that the term applies to. + The term is applied to the union of the namespaces listed in this field + and the ones selected by namespaceSelector. + null or empty namespaces list and null namespaceSelector means "this pod's namespace". + items: + type: string + type: array + topologyKey: + description: |- + This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching + the labelSelector in the specified namespaces, where co-located is defined as running on a node + whose value of the label with key topologyKey matches that of any node on which any of the + selected pods is running. + Empty topologyKey is not allowed. + type: string + required: + - topologyKey + type: object + weight: + description: |- + weight associated with matching the corresponding podAffinityTerm, + in the range 1-100. + format: int32 + type: integer + required: + - podAffinityTerm + - weight + type: object + type: array + requiredDuringSchedulingIgnoredDuringExecution: + description: |- + If the anti-affinity requirements specified by this field are not met at + scheduling time, the pod will not be scheduled onto the node. + If the anti-affinity requirements specified by this field cease to be met + at some point during pod execution (e.g. due to a pod label update), the + system may or may not try to eventually evict the pod from its node. + When there are multiple elements, the lists of nodes corresponding to each + podAffinityTerm are intersected, i.e. all terms must be satisfied. + items: + description: |- + Defines a set of pods (namely those matching the labelSelector + relative to the given namespace(s)) that this pod should be + co-located (affinity) or not co-located (anti-affinity) with, + where co-located is defined as running on a node whose value of + the label with key matches that of any node on which + a pod of the set of pods is running + properties: + labelSelector: + description: |- + A label query over a set of resources, in this case pods. + If it's null, this PodAffinityTerm matches with no Pods. + properties: + matchExpressions: + description: matchExpressions is a list + of label selector requirements. The requirements + are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key + that the selector applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + matchLabelKeys: + description: |- + MatchLabelKeys is a set of pod label keys to select which pods will + be taken into consideration. The keys are used to lookup values from the + incoming pod labels, those key-value labels are merged with `LabelSelector` as `key in (value)` + to select the group of existing pods which pods will be taken into consideration + for the incoming pod's pod (anti) affinity. Keys that don't exist in the incoming + pod labels will be ignored. The default value is empty. + The same key is forbidden to exist in both MatchLabelKeys and LabelSelector. + Also, MatchLabelKeys cannot be set when LabelSelector isn't set. + This is an alpha field and requires enabling MatchLabelKeysInPodAffinity feature gate. + items: + type: string + type: array + x-kubernetes-list-type: atomic + mismatchLabelKeys: + description: |- + MismatchLabelKeys is a set of pod label keys to select which pods will + be taken into consideration. The keys are used to lookup values from the + incoming pod labels, those key-value labels are merged with `LabelSelector` as `key notin (value)` + to select the group of existing pods which pods will be taken into consideration + for the incoming pod's pod (anti) affinity. Keys that don't exist in the incoming + pod labels will be ignored. The default value is empty. + The same key is forbidden to exist in both MismatchLabelKeys and LabelSelector. + Also, MismatchLabelKeys cannot be set when LabelSelector isn't set. + This is an alpha field and requires enabling MatchLabelKeysInPodAffinity feature gate. + items: + type: string + type: array + x-kubernetes-list-type: atomic + namespaceSelector: + description: |- + A label query over the set of namespaces that the term applies to. + The term is applied to the union of the namespaces selected by this field + and the ones listed in the namespaces field. + null selector and null or empty namespaces list means "this pod's namespace". + An empty selector ({}) matches all namespaces. + properties: + matchExpressions: + description: matchExpressions is a list + of label selector requirements. The requirements + are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key + that the selector applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + namespaces: + description: |- + namespaces specifies a static list of namespace names that the term applies to. + The term is applied to the union of the namespaces listed in this field + and the ones selected by namespaceSelector. + null or empty namespaces list and null namespaceSelector means "this pod's namespace". + items: + type: string + type: array + topologyKey: + description: |- + This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching + the labelSelector in the specified namespaces, where co-located is defined as running on a node + whose value of the label with key topologyKey matches that of any node on which any of the + selected pods is running. + Empty topologyKey is not allowed. + type: string + required: + - topologyKey + type: object + type: array + type: object + type: object + annotations: + additionalProperties: + type: string + type: object + containerSecurityContext: + description: |- + SecurityContext holds security configuration that will be applied to a container. + Some fields are present in both SecurityContext and PodSecurityContext. When both + are set, the values in SecurityContext take precedence. + properties: + allowPrivilegeEscalation: + description: |- + AllowPrivilegeEscalation controls whether a process can gain more + privileges than its parent process. This bool directly controls if + the no_new_privs flag will be set on the container process. + AllowPrivilegeEscalation is true always when the container is: + 1) run as Privileged + 2) has CAP_SYS_ADMIN + Note that this field cannot be set when spec.os.name is windows. + type: boolean + capabilities: + description: |- + The capabilities to add/drop when running containers. + Defaults to the default set of capabilities granted by the container runtime. + Note that this field cannot be set when spec.os.name is windows. + properties: + add: + description: Added capabilities + items: + description: Capability represent POSIX capabilities + type + type: string + type: array + drop: + description: Removed capabilities + items: + description: Capability represent POSIX capabilities + type + type: string + type: array + type: object + privileged: + description: |- + Run container in privileged mode. + Processes in privileged containers are essentially equivalent to root on the host. + Defaults to false. + Note that this field cannot be set when spec.os.name is windows. + type: boolean + procMount: + description: |- + procMount denotes the type of proc mount to use for the containers. + The default is DefaultProcMount which uses the container runtime defaults for + readonly paths and masked paths. + This requires the ProcMountType feature flag to be enabled. + Note that this field cannot be set when spec.os.name is windows. + type: string + readOnlyRootFilesystem: + description: |- + Whether this container has a read-only root filesystem. + Default is false. + Note that this field cannot be set when spec.os.name is windows. + type: boolean + runAsGroup: + description: |- + The GID to run the entrypoint of the container process. + Uses runtime default if unset. + May also be set in PodSecurityContext. If set in both SecurityContext and + PodSecurityContext, the value specified in SecurityContext takes precedence. + Note that this field cannot be set when spec.os.name is windows. + format: int64 + type: integer + runAsNonRoot: + description: |- + Indicates that the container must run as a non-root user. + If true, the Kubelet will validate the image at runtime to ensure that it + does not run as UID 0 (root) and fail to start the container if it does. + If unset or false, no such validation will be performed. + May also be set in PodSecurityContext. If set in both SecurityContext and + PodSecurityContext, the value specified in SecurityContext takes precedence. + type: boolean + runAsUser: + description: |- + The UID to run the entrypoint of the container process. + Defaults to user specified in image metadata if unspecified. + May also be set in PodSecurityContext. If set in both SecurityContext and + PodSecurityContext, the value specified in SecurityContext takes precedence. + Note that this field cannot be set when spec.os.name is windows. + format: int64 + type: integer + seLinuxOptions: + description: |- + The SELinux context to be applied to the container. + If unspecified, the container runtime will allocate a random SELinux context for each + container. May also be set in PodSecurityContext. If set in both SecurityContext and + PodSecurityContext, the value specified in SecurityContext takes precedence. + Note that this field cannot be set when spec.os.name is windows. + properties: + level: + description: Level is SELinux level label that applies + to the container. + type: string + role: + description: Role is a SELinux role label that applies + to the container. + type: string + type: + description: Type is a SELinux type label that applies + to the container. + type: string + user: + description: User is a SELinux user label that applies + to the container. + type: string + type: object + seccompProfile: + description: |- + The seccomp options to use by this container. If seccomp options are + provided at both the pod & container level, the container options + override the pod options. + Note that this field cannot be set when spec.os.name is windows. + properties: + localhostProfile: + description: |- + localhostProfile indicates a profile defined in a file on the node should be used. + The profile must be preconfigured on the node to work. + Must be a descending path, relative to the kubelet's configured seccomp profile location. + Must be set if type is "Localhost". Must NOT be set for any other type. + type: string + type: + description: |- + type indicates which kind of seccomp profile will be applied. + Valid options are: + + + Localhost - a profile defined in a file on the node should be used. + RuntimeDefault - the container runtime default profile should be used. + Unconfined - no profile should be applied. + type: string + required: + - type + type: object + windowsOptions: + description: |- + The Windows specific settings applied to all containers. + If unspecified, the options from the PodSecurityContext will be used. + If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence. + Note that this field cannot be set when spec.os.name is linux. + properties: + gmsaCredentialSpec: + description: |- + GMSACredentialSpec is where the GMSA admission webhook + (https://github.com/kubernetes-sigs/windows-gmsa) inlines the contents of the + GMSA credential spec named by the GMSACredentialSpecName field. + type: string + gmsaCredentialSpecName: + description: GMSACredentialSpecName is the name of + the GMSA credential spec to use. + type: string + hostProcess: + description: |- + HostProcess determines if a container should be run as a 'Host Process' container. + All of a Pod's containers must have the same effective HostProcess value + (it is not allowed to have a mix of HostProcess containers and non-HostProcess containers). + In addition, if HostProcess is true then HostNetwork must also be set to true. + type: boolean + runAsUserName: + description: |- + The UserName in Windows to run the entrypoint of the container process. + Defaults to the user specified in image metadata if unspecified. + May also be set in PodSecurityContext. If set in both SecurityContext and + PodSecurityContext, the value specified in SecurityContext takes precedence. + type: string + type: object + type: object + imagePullPolicy: + description: PullPolicy describes a policy for if/when to + pull a container image + type: string + imagePullSecrets: + items: + description: |- + LocalObjectReference contains enough information to let you locate the + referenced object inside the same namespace. + properties: + name: + description: |- + Name of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid? + type: string + type: object + x-kubernetes-map-type: atomic + type: array + labels: + additionalProperties: + type: string + type: object + lifecycle: + description: |- + Lifecycle describes actions that the management system should take in response to container lifecycle + events. For the PostStart and PreStop lifecycle handlers, management of the container blocks + until the action is complete, unless the container process fails, in which case the handler is aborted. + properties: + postStart: + description: |- + PostStart is called immediately after a container is created. If the handler fails, + the container is terminated and restarted according to its restart policy. + Other management of the container blocks until the hook completes. + More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: |- + Command is the command line to execute inside the container, the working directory for the + command is root ('/') in the container's filesystem. The command is simply exec'd, it is + not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use + a shell, you need to explicitly call out to that shell. + Exit status of 0 is treated as live/healthy and non-zero is unhealthy. + items: + type: string + type: array + type: object + httpGet: + description: HTTPGet specifies the http request to + perform. + properties: + host: + description: |- + Host name to connect to, defaults to the pod IP. You probably want to set + "Host" in httpHeaders instead. + type: string + httpHeaders: + description: Custom headers to set in the request. + HTTP allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: |- + The header field name. + This will be canonicalized upon output, so case-variant names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: |- + Name or number of the port to access on the container. + Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: |- + Scheme to use for connecting to the host. + Defaults to HTTP. + type: string + required: + - port + type: object + sleep: + description: Sleep represents the duration that the + container should sleep before being terminated. + properties: + seconds: + description: Seconds is the number of seconds + to sleep. + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + description: |- + Deprecated. TCPSocket is NOT supported as a LifecycleHandler and kept + for the backward compatibility. There are no validation of this field and + lifecycle hooks will fail in runtime when tcp handler is specified. + properties: + host: + description: 'Optional: Host name to connect to, + defaults to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: |- + Number or name of the port to access on the container. + Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + preStop: + description: |- + PreStop is called immediately before a container is terminated due to an + API request or management event such as liveness/startup probe failure, + preemption, resource contention, etc. The handler is not called if the + container crashes or exits. The Pod's termination grace period countdown begins before the + PreStop hook is executed. Regardless of the outcome of the handler, the + container will eventually terminate within the Pod's termination grace + period (unless delayed by finalizers). Other management of the container blocks until the hook completes + or until the termination grace period is reached. + More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: |- + Command is the command line to execute inside the container, the working directory for the + command is root ('/') in the container's filesystem. The command is simply exec'd, it is + not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use + a shell, you need to explicitly call out to that shell. + Exit status of 0 is treated as live/healthy and non-zero is unhealthy. + items: + type: string + type: array + type: object + httpGet: + description: HTTPGet specifies the http request to + perform. + properties: + host: + description: |- + Host name to connect to, defaults to the pod IP. You probably want to set + "Host" in httpHeaders instead. + type: string + httpHeaders: + description: Custom headers to set in the request. + HTTP allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: |- + The header field name. + This will be canonicalized upon output, so case-variant names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: |- + Name or number of the port to access on the container. + Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: |- + Scheme to use for connecting to the host. + Defaults to HTTP. + type: string + required: + - port + type: object + sleep: + description: Sleep represents the duration that the + container should sleep before being terminated. + properties: + seconds: + description: Seconds is the number of seconds + to sleep. + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + description: |- + Deprecated. TCPSocket is NOT supported as a LifecycleHandler and kept + for the backward compatibility. There are no validation of this field and + lifecycle hooks will fail in runtime when tcp handler is specified. + properties: + host: + description: 'Optional: Host name to connect to, + defaults to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: |- + Number or name of the port to access on the container. + Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + type: object + livenessProbe: + description: |- + Probe describes a health check to be performed against a container to determine whether it is + alive or ready to receive traffic. + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: |- + Command is the command line to execute inside the container, the working directory for the + command is root ('/') in the container's filesystem. The command is simply exec'd, it is + not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use + a shell, you need to explicitly call out to that shell. + Exit status of 0 is treated as live/healthy and non-zero is unhealthy. + items: + type: string + type: array + type: object + failureThreshold: + description: |- + Minimum consecutive failures for the probe to be considered failed after having succeeded. + Defaults to 3. Minimum value is 1. + format: int32 + type: integer + grpc: + description: GRPC specifies an action involving a GRPC + port. + properties: + port: + description: Port number of the gRPC service. Number + must be in the range 1 to 65535. + format: int32 + type: integer + service: + description: |- + Service is the name of the service to place in the gRPC HealthCheckRequest + (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md). + + + If this is not specified, the default behavior is defined by gRPC. + type: string + required: + - port + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: |- + Host name to connect to, defaults to the pod IP. You probably want to set + "Host" in httpHeaders instead. + type: string + httpHeaders: + description: Custom headers to set in the request. + HTTP allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: |- + The header field name. + This will be canonicalized upon output, so case-variant names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: |- + Name or number of the port to access on the container. + Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: |- + Scheme to use for connecting to the host. + Defaults to HTTP. + type: string + required: + - port + type: object + initialDelaySeconds: + description: |- + Number of seconds after the container has started before liveness probes are initiated. + More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes + format: int32 + type: integer + periodSeconds: + description: |- + How often (in seconds) to perform the probe. + Default to 10 seconds. Minimum value is 1. + format: int32 + type: integer + successThreshold: + description: |- + Minimum consecutive successes for the probe to be considered successful after having failed. + Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1. + format: int32 + type: integer + tcpSocket: + description: TCPSocket specifies an action involving a + TCP port. + properties: + host: + description: 'Optional: Host name to connect to, defaults + to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: |- + Number or name of the port to access on the container. + Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + description: |- + Optional duration in seconds the pod needs to terminate gracefully upon probe failure. + The grace period is the duration in seconds after the processes running in the pod are sent + a termination signal and the time when the processes are forcibly halted with a kill signal. + Set this value longer than the expected cleanup time for your process. + If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this + value overrides the value provided by the pod spec. + Value must be non-negative integer. The value zero indicates stop immediately via + the kill signal (no opportunity to shut down). + This is a beta field and requires enabling ProbeTerminationGracePeriod feature gate. + Minimum value is 1. spec.terminationGracePeriodSeconds is used if unset. + format: int64 + type: integer + timeoutSeconds: + description: |- + Number of seconds after which the probe times out. + Defaults to 1 second. Minimum value is 1. + More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes + format: int32 + type: integer + type: object + nodeSelector: + additionalProperties: + type: string + type: object + podSecurityContext: + description: |- + PodSecurityContext holds pod-level security attributes and common container settings. + Some fields are also present in container.securityContext. Field values of + container.securityContext take precedence over field values of PodSecurityContext. + properties: + fsGroup: + description: |- + A special supplemental group that applies to all containers in a pod. + Some volume types allow the Kubelet to change the ownership of that volume + to be owned by the pod: + + + 1. The owning GID will be the FSGroup + 2. The setgid bit is set (new files created in the volume will be owned by FSGroup) + 3. The permission bits are OR'd with rw-rw---- + + + If unset, the Kubelet will not modify the ownership and permissions of any volume. + Note that this field cannot be set when spec.os.name is windows. + format: int64 + type: integer + fsGroupChangePolicy: + description: |- + fsGroupChangePolicy defines behavior of changing ownership and permission of the volume + before being exposed inside Pod. This field will only apply to + volume types which support fsGroup based ownership(and permissions). + It will have no effect on ephemeral volume types such as: secret, configmaps + and emptydir. + Valid values are "OnRootMismatch" and "Always". If not specified, "Always" is used. + Note that this field cannot be set when spec.os.name is windows. + type: string + runAsGroup: + description: |- + The GID to run the entrypoint of the container process. + Uses runtime default if unset. + May also be set in SecurityContext. If set in both SecurityContext and + PodSecurityContext, the value specified in SecurityContext takes precedence + for that container. + Note that this field cannot be set when spec.os.name is windows. + format: int64 + type: integer + runAsNonRoot: + description: |- + Indicates that the container must run as a non-root user. + If true, the Kubelet will validate the image at runtime to ensure that it + does not run as UID 0 (root) and fail to start the container if it does. + If unset or false, no such validation will be performed. + May also be set in SecurityContext. If set in both SecurityContext and + PodSecurityContext, the value specified in SecurityContext takes precedence. + type: boolean + runAsUser: + description: |- + The UID to run the entrypoint of the container process. + Defaults to user specified in image metadata if unspecified. + May also be set in SecurityContext. If set in both SecurityContext and + PodSecurityContext, the value specified in SecurityContext takes precedence + for that container. + Note that this field cannot be set when spec.os.name is windows. + format: int64 + type: integer + seLinuxOptions: + description: |- + The SELinux context to be applied to all containers. + If unspecified, the container runtime will allocate a random SELinux context for each + container. May also be set in SecurityContext. If set in + both SecurityContext and PodSecurityContext, the value specified in SecurityContext + takes precedence for that container. + Note that this field cannot be set when spec.os.name is windows. + properties: + level: + description: Level is SELinux level label that applies + to the container. + type: string + role: + description: Role is a SELinux role label that applies + to the container. + type: string + type: + description: Type is a SELinux type label that applies + to the container. + type: string + user: + description: User is a SELinux user label that applies + to the container. + type: string + type: object + seccompProfile: + description: |- + The seccomp options to use by the containers in this pod. + Note that this field cannot be set when spec.os.name is windows. + properties: + localhostProfile: + description: |- + localhostProfile indicates a profile defined in a file on the node should be used. + The profile must be preconfigured on the node to work. + Must be a descending path, relative to the kubelet's configured seccomp profile location. + Must be set if type is "Localhost". Must NOT be set for any other type. + type: string + type: + description: |- + type indicates which kind of seccomp profile will be applied. + Valid options are: + + + Localhost - a profile defined in a file on the node should be used. + RuntimeDefault - the container runtime default profile should be used. + Unconfined - no profile should be applied. + type: string + required: + - type + type: object + supplementalGroups: + description: |- + A list of groups applied to the first process run in each container, in addition + to the container's primary GID, the fsGroup (if specified), and group memberships + defined in the container image for the uid of the container process. If unspecified, + no additional groups are added to any container. Note that group memberships + defined in the container image for the uid of the container process are still effective, + even if they are not included in this list. + Note that this field cannot be set when spec.os.name is windows. + items: + format: int64 + type: integer + type: array + sysctls: + description: |- + Sysctls hold a list of namespaced sysctls used for the pod. Pods with unsupported + sysctls (by the container runtime) might fail to launch. + Note that this field cannot be set when spec.os.name is windows. + items: + description: Sysctl defines a kernel parameter to be + set + properties: + name: + description: Name of a property to set + type: string + value: + description: Value of a property to set + type: string + required: + - name + - value + type: object + type: array + windowsOptions: + description: |- + The Windows specific settings applied to all containers. + If unspecified, the options within a container's SecurityContext will be used. + If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence. + Note that this field cannot be set when spec.os.name is linux. + properties: + gmsaCredentialSpec: + description: |- + GMSACredentialSpec is where the GMSA admission webhook + (https://github.com/kubernetes-sigs/windows-gmsa) inlines the contents of the + GMSA credential spec named by the GMSACredentialSpecName field. + type: string + gmsaCredentialSpecName: + description: GMSACredentialSpecName is the name of + the GMSA credential spec to use. + type: string + hostProcess: + description: |- + HostProcess determines if a container should be run as a 'Host Process' container. + All of a Pod's containers must have the same effective HostProcess value + (it is not allowed to have a mix of HostProcess containers and non-HostProcess containers). + In addition, if HostProcess is true then HostNetwork must also be set to true. + type: boolean + runAsUserName: + description: |- + The UserName in Windows to run the entrypoint of the container process. + Defaults to the user specified in image metadata if unspecified. + May also be set in PodSecurityContext. If set in both SecurityContext and + PodSecurityContext, the value specified in SecurityContext takes precedence. + type: string + type: object + type: object + priorityClassName: + type: string + readinessProbe: + description: |- + Probe describes a health check to be performed against a container to determine whether it is + alive or ready to receive traffic. + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: |- + Command is the command line to execute inside the container, the working directory for the + command is root ('/') in the container's filesystem. The command is simply exec'd, it is + not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use + a shell, you need to explicitly call out to that shell. + Exit status of 0 is treated as live/healthy and non-zero is unhealthy. + items: + type: string + type: array + type: object + failureThreshold: + description: |- + Minimum consecutive failures for the probe to be considered failed after having succeeded. + Defaults to 3. Minimum value is 1. + format: int32 + type: integer + grpc: + description: GRPC specifies an action involving a GRPC + port. + properties: + port: + description: Port number of the gRPC service. Number + must be in the range 1 to 65535. + format: int32 + type: integer + service: + description: |- + Service is the name of the service to place in the gRPC HealthCheckRequest + (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md). + + + If this is not specified, the default behavior is defined by gRPC. + type: string + required: + - port + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: |- + Host name to connect to, defaults to the pod IP. You probably want to set + "Host" in httpHeaders instead. + type: string + httpHeaders: + description: Custom headers to set in the request. + HTTP allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: |- + The header field name. + This will be canonicalized upon output, so case-variant names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: |- + Name or number of the port to access on the container. + Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: |- + Scheme to use for connecting to the host. + Defaults to HTTP. + type: string + required: + - port + type: object + initialDelaySeconds: + description: |- + Number of seconds after the container has started before liveness probes are initiated. + More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes + format: int32 + type: integer + periodSeconds: + description: |- + How often (in seconds) to perform the probe. + Default to 10 seconds. Minimum value is 1. + format: int32 + type: integer + successThreshold: + description: |- + Minimum consecutive successes for the probe to be considered successful after having failed. + Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1. + format: int32 + type: integer + tcpSocket: + description: TCPSocket specifies an action involving a + TCP port. + properties: + host: + description: 'Optional: Host name to connect to, defaults + to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: |- + Number or name of the port to access on the container. + Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + description: |- + Optional duration in seconds the pod needs to terminate gracefully upon probe failure. + The grace period is the duration in seconds after the processes running in the pod are sent + a termination signal and the time when the processes are forcibly halted with a kill signal. + Set this value longer than the expected cleanup time for your process. + If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this + value overrides the value provided by the pod spec. + Value must be non-negative integer. The value zero indicates stop immediately via + the kill signal (no opportunity to shut down). + This is a beta field and requires enabling ProbeTerminationGracePeriod feature gate. + Minimum value is 1. spec.terminationGracePeriodSeconds is used if unset. + format: int64 + type: integer + timeoutSeconds: + description: |- + Number of seconds after which the probe times out. + Defaults to 1 second. Minimum value is 1. + More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes + format: int32 + type: integer + type: object + resources: + description: ResourceRequirements describes the compute resource + requirements. + properties: + claims: + description: |- + Claims lists the names of resources, defined in spec.resourceClaims, + that are used by this container. + + + This is an alpha field and requires enabling the + DynamicResourceAllocation feature gate. + + + This field is immutable. It can only be set for containers. + items: + description: ResourceClaim references one entry in PodSpec.ResourceClaims. + properties: + name: + description: |- + Name must match the name of one entry in pod.spec.resourceClaims of + the Pod where this field is used. It makes that resource available + inside a container. + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: |- + Limits describes the maximum amount of compute resources allowed. + More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/ + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: |- + Requests describes the minimum amount of compute resources required. + If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, + otherwise to an implementation-defined value. Requests cannot exceed Limits. + More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/ + type: object + type: object + serviceAccountName: + type: string + terminationGracePeriodSeconds: + format: int64 type: integer - service: + tolerations: + items: + description: |- + The pod this Toleration is attached to tolerates any taint that matches + the triple using the matching operator . + properties: + effect: + description: |- + Effect indicates the taint effect to match. Empty means match all taint effects. + When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute. + type: string + key: + description: |- + Key is the taint key that the toleration applies to. Empty means match all taint keys. + If the key is empty, operator must be Exists; this combination means to match all values and all keys. + type: string + operator: + description: |- + Operator represents a key's relationship to the value. + Valid operators are Exists and Equal. Defaults to Equal. + Exists is equivalent to wildcard for value, so that a pod can + tolerate all taints of a particular category. + type: string + tolerationSeconds: + description: |- + TolerationSeconds represents the period of time the toleration (which must be + of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, + it is not set, which means tolerate the taint forever (do not evict). Zero and + negative values will be treated as 0 (evict immediately) by the system. + format: int64 + type: integer + value: + description: |- + Value is the taint value the toleration matches to. + If the operator is Exists, the value should be empty, otherwise just a regular string. + type: string + type: object + type: array + topologySpreadConstraints: + items: + description: TopologySpreadConstraint specifies how to spread + matching pods among the given topology. + properties: + labelSelector: + description: |- + LabelSelector is used to find matching pods. + Pods that match this label selector are counted to determine the number of pods + in their corresponding topology domain. + properties: + matchExpressions: + description: matchExpressions is a list of label + selector requirements. The requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key that the + selector applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + matchLabelKeys: + description: |- + MatchLabelKeys is a set of pod label keys to select the pods over which + spreading will be calculated. The keys are used to lookup values from the + incoming pod labels, those key-value labels are ANDed with labelSelector + to select the group of existing pods over which spreading will be calculated + for the incoming pod. The same key is forbidden to exist in both MatchLabelKeys and LabelSelector. + MatchLabelKeys cannot be set when LabelSelector isn't set. + Keys that don't exist in the incoming pod labels will + be ignored. A null or empty list means only match against labelSelector. + + + This is a beta field and requires the MatchLabelKeysInPodTopologySpread feature gate to be enabled (enabled by default). + items: + type: string + type: array + x-kubernetes-list-type: atomic + maxSkew: + description: |- + MaxSkew describes the degree to which pods may be unevenly distributed. + When `whenUnsatisfiable=DoNotSchedule`, it is the maximum permitted difference + between the number of matching pods in the target topology and the global minimum. + The global minimum is the minimum number of matching pods in an eligible domain + or zero if the number of eligible domains is less than MinDomains. + For example, in a 3-zone cluster, MaxSkew is set to 1, and pods with the same + labelSelector spread as 2/2/1: + In this case, the global minimum is 1. + | zone1 | zone2 | zone3 | + | P P | P P | P | + - if MaxSkew is 1, incoming pod can only be scheduled to zone3 to become 2/2/2; + scheduling it onto zone1(zone2) would make the ActualSkew(3-1) on zone1(zone2) + violate MaxSkew(1). + - if MaxSkew is 2, incoming pod can be scheduled onto any zone. + When `whenUnsatisfiable=ScheduleAnyway`, it is used to give higher precedence + to topologies that satisfy it. + It's a required field. Default value is 1 and 0 is not allowed. + format: int32 + type: integer + minDomains: + description: |- + MinDomains indicates a minimum number of eligible domains. + When the number of eligible domains with matching topology keys is less than minDomains, + Pod Topology Spread treats "global minimum" as 0, and then the calculation of Skew is performed. + And when the number of eligible domains with matching topology keys equals or greater than minDomains, + this value has no effect on scheduling. + As a result, when the number of eligible domains is less than minDomains, + scheduler won't schedule more than maxSkew Pods to those domains. + If value is nil, the constraint behaves as if MinDomains is equal to 1. + Valid values are integers greater than 0. + When value is not nil, WhenUnsatisfiable must be DoNotSchedule. + + + For example, in a 3-zone cluster, MaxSkew is set to 2, MinDomains is set to 5 and pods with the same + labelSelector spread as 2/2/2: + | zone1 | zone2 | zone3 | + | P P | P P | P P | + The number of domains is less than 5(MinDomains), so "global minimum" is treated as 0. + In this situation, new pod with the same labelSelector cannot be scheduled, + because computed skew will be 3(3 - 0) if new Pod is scheduled to any of the three zones, + it will violate MaxSkew. + + + This is a beta field and requires the MinDomainsInPodTopologySpread feature gate to be enabled (enabled by default). + format: int32 + type: integer + nodeAffinityPolicy: + description: |- + NodeAffinityPolicy indicates how we will treat Pod's nodeAffinity/nodeSelector + when calculating pod topology spread skew. Options are: + - Honor: only nodes matching nodeAffinity/nodeSelector are included in the calculations. + - Ignore: nodeAffinity/nodeSelector are ignored. All nodes are included in the calculations. + + + If this value is nil, the behavior is equivalent to the Honor policy. + This is a beta-level feature default enabled by the NodeInclusionPolicyInPodTopologySpread feature flag. + type: string + nodeTaintsPolicy: + description: |- + NodeTaintsPolicy indicates how we will treat node taints when calculating + pod topology spread skew. Options are: + - Honor: nodes without taints, along with tainted nodes for which the incoming pod + has a toleration, are included. + - Ignore: node taints are ignored. All nodes are included. + + + If this value is nil, the behavior is equivalent to the Ignore policy. + This is a beta-level feature default enabled by the NodeInclusionPolicyInPodTopologySpread feature flag. + type: string + topologyKey: + description: |- + TopologyKey is the key of node labels. Nodes that have a label with this key + and identical values are considered to be in the same topology. + We consider each as a "bucket", and try to put balanced number + of pods into each bucket. + We define a domain as a particular instance of a topology. + Also, we define an eligible domain as a domain whose nodes meet the requirements of + nodeAffinityPolicy and nodeTaintsPolicy. + e.g. If TopologyKey is "kubernetes.io/hostname", each Node is a domain of that topology. + And, if TopologyKey is "topology.kubernetes.io/zone", each zone is a domain of that topology. + It's a required field. + type: string + whenUnsatisfiable: + description: |- + WhenUnsatisfiable indicates how to deal with a pod if it doesn't satisfy + the spread constraint. + - DoNotSchedule (default) tells the scheduler not to schedule it. + - ScheduleAnyway tells the scheduler to schedule the pod in any location, + but giving higher precedence to topologies that would help reduce the + skew. + A constraint is considered "Unsatisfiable" for an incoming pod + if and only if every possible node assignment for that pod would violate + "MaxSkew" on some topology. + For example, in a 3-zone cluster, MaxSkew is set to 1, and pods with the same + labelSelector spread as 3/1/1: + | zone1 | zone2 | zone3 | + | P P P | P | P | + If WhenUnsatisfiable is set to DoNotSchedule, incoming pod can only be scheduled + to zone2(zone3) to become 3/2/1(3/1/2) as ActualSkew(2-1) on zone2(zone3) satisfies + MaxSkew(1). In other words, the cluster can still be imbalanced, but scheduler + won't make it *more* imbalanced. + It's a required field. + type: string + required: + - maxSkew + - topologyKey + - whenUnsatisfiable + type: object + type: array + type: object + replicas: + type: integer + servicebound: + properties: + port: + type: integer + service: + type: string + serviceType: + description: Service Type string describes ingress methods + for a service + type: string + type: object + required: + - edgebound + - servicebound + type: object + frontlas: + properties: + controlplane: + properties: + frontierPlanePort: + type: integer + port: + type: integer + service: + type: string + serviceType: + description: Service Type string describes ingress methods + for a service + type: string + type: object + image: + type: string + nodeAffinity: + description: Node affinity is a group of node affinity scheduling + rules. + properties: + preferredDuringSchedulingIgnoredDuringExecution: + description: |- + The scheduler will prefer to schedule pods to nodes that satisfy + the affinity expressions specified by this field, but it may choose + a node that violates one or more of the expressions. The node that is + most preferred is the one with the greatest sum of weights, i.e. + for each node that meets all of the scheduling requirements (resource + request, requiredDuringScheduling affinity expressions, etc.), + compute a sum by iterating through the elements of this field and adding + "weight" to the sum if the node matches the corresponding matchExpressions; the + node(s) with the highest sum are the most preferred. + items: + description: |- + An empty preferred scheduling term matches all objects with implicit weight 0 + (i.e. it's a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op). + properties: + preference: + description: A node selector term, associated with the + corresponding weight. + properties: + matchExpressions: + description: A list of node selector requirements + by node's labels. + items: + description: |- + A node selector requirement is a selector that contains values, a key, and an operator + that relates the key and values. + properties: + key: + description: The label key that the selector + applies to. + type: string + operator: + description: |- + Represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt. + type: string + values: + description: |- + An array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. If the operator is Gt or Lt, the values + array must have a single element, which will be interpreted as an integer. + This array is replaced during a strategic merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchFields: + description: A list of node selector requirements + by node's fields. + items: + description: |- + A node selector requirement is a selector that contains values, a key, and an operator + that relates the key and values. + properties: + key: + description: The label key that the selector + applies to. + type: string + operator: + description: |- + Represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt. + type: string + values: + description: |- + An array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. If the operator is Gt or Lt, the values + array must have a single element, which will be interpreted as an integer. + This array is replaced during a strategic merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + type: object + x-kubernetes-map-type: atomic + weight: + description: Weight associated with matching the corresponding + nodeSelectorTerm, in the range 1-100. + format: int32 + type: integer + required: + - preference + - weight + type: object + type: array + requiredDuringSchedulingIgnoredDuringExecution: + description: |- + If the affinity requirements specified by this field are not met at + scheduling time, the pod will not be scheduled onto the node. + If the affinity requirements specified by this field cease to be met + at some point during pod execution (e.g. due to an update), the system + may or may not try to eventually evict the pod from its node. + properties: + nodeSelectorTerms: + description: Required. A list of node selector terms. + The terms are ORed. + items: + description: |- + A null or empty node selector term matches no objects. The requirements of + them are ANDed. + The TopologySelectorTerm type implements a subset of the NodeSelectorTerm. + properties: + matchExpressions: + description: A list of node selector requirements + by node's labels. + items: + description: |- + A node selector requirement is a selector that contains values, a key, and an operator + that relates the key and values. + properties: + key: + description: The label key that the selector + applies to. + type: string + operator: + description: |- + Represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt. + type: string + values: + description: |- + An array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. If the operator is Gt or Lt, the values + array must have a single element, which will be interpreted as an integer. + This array is replaced during a strategic merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchFields: + description: A list of node selector requirements + by node's fields. + items: + description: |- + A node selector requirement is a selector that contains values, a key, and an operator + that relates the key and values. + properties: + key: + description: The label key that the selector + applies to. + type: string + operator: + description: |- + Represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt. + type: string + values: + description: |- + An array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. If the operator is Gt or Lt, the values + array must have a single element, which will be interpreted as an integer. + This array is replaced during a strategic merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + type: object + x-kubernetes-map-type: atomic + type: array + required: + - nodeSelectorTerms + type: object + x-kubernetes-map-type: atomic + type: object + pod: + description: |- + Pod is the optional set of generic Pod-level overrides applied to the + frontlas Deployment. See Frontier.Pod for semantics. + properties: + affinity: + description: Affinity is a group of affinity scheduling rules. + properties: + nodeAffinity: + description: Describes node affinity scheduling rules + for the pod. + properties: + preferredDuringSchedulingIgnoredDuringExecution: + description: |- + The scheduler will prefer to schedule pods to nodes that satisfy + the affinity expressions specified by this field, but it may choose + a node that violates one or more of the expressions. The node that is + most preferred is the one with the greatest sum of weights, i.e. + for each node that meets all of the scheduling requirements (resource + request, requiredDuringScheduling affinity expressions, etc.), + compute a sum by iterating through the elements of this field and adding + "weight" to the sum if the node matches the corresponding matchExpressions; the + node(s) with the highest sum are the most preferred. + items: + description: |- + An empty preferred scheduling term matches all objects with implicit weight 0 + (i.e. it's a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op). + properties: + preference: + description: A node selector term, associated + with the corresponding weight. + properties: + matchExpressions: + description: A list of node selector requirements + by node's labels. + items: + description: |- + A node selector requirement is a selector that contains values, a key, and an operator + that relates the key and values. + properties: + key: + description: The label key that the + selector applies to. + type: string + operator: + description: |- + Represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt. + type: string + values: + description: |- + An array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. If the operator is Gt or Lt, the values + array must have a single element, which will be interpreted as an integer. + This array is replaced during a strategic merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchFields: + description: A list of node selector requirements + by node's fields. + items: + description: |- + A node selector requirement is a selector that contains values, a key, and an operator + that relates the key and values. + properties: + key: + description: The label key that the + selector applies to. + type: string + operator: + description: |- + Represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt. + type: string + values: + description: |- + An array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. If the operator is Gt or Lt, the values + array must have a single element, which will be interpreted as an integer. + This array is replaced during a strategic merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + type: object + x-kubernetes-map-type: atomic + weight: + description: Weight associated with matching + the corresponding nodeSelectorTerm, in the + range 1-100. + format: int32 + type: integer + required: + - preference + - weight + type: object + type: array + requiredDuringSchedulingIgnoredDuringExecution: + description: |- + If the affinity requirements specified by this field are not met at + scheduling time, the pod will not be scheduled onto the node. + If the affinity requirements specified by this field cease to be met + at some point during pod execution (e.g. due to an update), the system + may or may not try to eventually evict the pod from its node. + properties: + nodeSelectorTerms: + description: Required. A list of node selector + terms. The terms are ORed. + items: + description: |- + A null or empty node selector term matches no objects. The requirements of + them are ANDed. + The TopologySelectorTerm type implements a subset of the NodeSelectorTerm. + properties: + matchExpressions: + description: A list of node selector requirements + by node's labels. + items: + description: |- + A node selector requirement is a selector that contains values, a key, and an operator + that relates the key and values. + properties: + key: + description: The label key that the + selector applies to. + type: string + operator: + description: |- + Represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt. + type: string + values: + description: |- + An array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. If the operator is Gt or Lt, the values + array must have a single element, which will be interpreted as an integer. + This array is replaced during a strategic merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchFields: + description: A list of node selector requirements + by node's fields. + items: + description: |- + A node selector requirement is a selector that contains values, a key, and an operator + that relates the key and values. + properties: + key: + description: The label key that the + selector applies to. + type: string + operator: + description: |- + Represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt. + type: string + values: + description: |- + An array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. If the operator is Gt or Lt, the values + array must have a single element, which will be interpreted as an integer. + This array is replaced during a strategic merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + type: object + x-kubernetes-map-type: atomic + type: array + required: + - nodeSelectorTerms + type: object + x-kubernetes-map-type: atomic + type: object + podAffinity: + description: Describes pod affinity scheduling rules (e.g. + co-locate this pod in the same node, zone, etc. as some + other pod(s)). + properties: + preferredDuringSchedulingIgnoredDuringExecution: + description: |- + The scheduler will prefer to schedule pods to nodes that satisfy + the affinity expressions specified by this field, but it may choose + a node that violates one or more of the expressions. The node that is + most preferred is the one with the greatest sum of weights, i.e. + for each node that meets all of the scheduling requirements (resource + request, requiredDuringScheduling affinity expressions, etc.), + compute a sum by iterating through the elements of this field and adding + "weight" to the sum if the node has pods which matches the corresponding podAffinityTerm; the + node(s) with the highest sum are the most preferred. + items: + description: The weights of all of the matched WeightedPodAffinityTerm + fields are added per-node to find the most preferred + node(s) + properties: + podAffinityTerm: + description: Required. A pod affinity term, + associated with the corresponding weight. + properties: + labelSelector: + description: |- + A label query over a set of resources, in this case pods. + If it's null, this PodAffinityTerm matches with no Pods. + properties: + matchExpressions: + description: matchExpressions is a list + of label selector requirements. The + requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label + key that the selector applies + to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + matchLabelKeys: + description: |- + MatchLabelKeys is a set of pod label keys to select which pods will + be taken into consideration. The keys are used to lookup values from the + incoming pod labels, those key-value labels are merged with `LabelSelector` as `key in (value)` + to select the group of existing pods which pods will be taken into consideration + for the incoming pod's pod (anti) affinity. Keys that don't exist in the incoming + pod labels will be ignored. The default value is empty. + The same key is forbidden to exist in both MatchLabelKeys and LabelSelector. + Also, MatchLabelKeys cannot be set when LabelSelector isn't set. + This is an alpha field and requires enabling MatchLabelKeysInPodAffinity feature gate. + items: + type: string + type: array + x-kubernetes-list-type: atomic + mismatchLabelKeys: + description: |- + MismatchLabelKeys is a set of pod label keys to select which pods will + be taken into consideration. The keys are used to lookup values from the + incoming pod labels, those key-value labels are merged with `LabelSelector` as `key notin (value)` + to select the group of existing pods which pods will be taken into consideration + for the incoming pod's pod (anti) affinity. Keys that don't exist in the incoming + pod labels will be ignored. The default value is empty. + The same key is forbidden to exist in both MismatchLabelKeys and LabelSelector. + Also, MismatchLabelKeys cannot be set when LabelSelector isn't set. + This is an alpha field and requires enabling MatchLabelKeysInPodAffinity feature gate. + items: + type: string + type: array + x-kubernetes-list-type: atomic + namespaceSelector: + description: |- + A label query over the set of namespaces that the term applies to. + The term is applied to the union of the namespaces selected by this field + and the ones listed in the namespaces field. + null selector and null or empty namespaces list means "this pod's namespace". + An empty selector ({}) matches all namespaces. + properties: + matchExpressions: + description: matchExpressions is a list + of label selector requirements. The + requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label + key that the selector applies + to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + namespaces: + description: |- + namespaces specifies a static list of namespace names that the term applies to. + The term is applied to the union of the namespaces listed in this field + and the ones selected by namespaceSelector. + null or empty namespaces list and null namespaceSelector means "this pod's namespace". + items: + type: string + type: array + topologyKey: + description: |- + This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching + the labelSelector in the specified namespaces, where co-located is defined as running on a node + whose value of the label with key topologyKey matches that of any node on which any of the + selected pods is running. + Empty topologyKey is not allowed. + type: string + required: + - topologyKey + type: object + weight: + description: |- + weight associated with matching the corresponding podAffinityTerm, + in the range 1-100. + format: int32 + type: integer + required: + - podAffinityTerm + - weight + type: object + type: array + requiredDuringSchedulingIgnoredDuringExecution: + description: |- + If the affinity requirements specified by this field are not met at + scheduling time, the pod will not be scheduled onto the node. + If the affinity requirements specified by this field cease to be met + at some point during pod execution (e.g. due to a pod label update), the + system may or may not try to eventually evict the pod from its node. + When there are multiple elements, the lists of nodes corresponding to each + podAffinityTerm are intersected, i.e. all terms must be satisfied. + items: + description: |- + Defines a set of pods (namely those matching the labelSelector + relative to the given namespace(s)) that this pod should be + co-located (affinity) or not co-located (anti-affinity) with, + where co-located is defined as running on a node whose value of + the label with key matches that of any node on which + a pod of the set of pods is running + properties: + labelSelector: + description: |- + A label query over a set of resources, in this case pods. + If it's null, this PodAffinityTerm matches with no Pods. + properties: + matchExpressions: + description: matchExpressions is a list + of label selector requirements. The requirements + are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key + that the selector applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + matchLabelKeys: + description: |- + MatchLabelKeys is a set of pod label keys to select which pods will + be taken into consideration. The keys are used to lookup values from the + incoming pod labels, those key-value labels are merged with `LabelSelector` as `key in (value)` + to select the group of existing pods which pods will be taken into consideration + for the incoming pod's pod (anti) affinity. Keys that don't exist in the incoming + pod labels will be ignored. The default value is empty. + The same key is forbidden to exist in both MatchLabelKeys and LabelSelector. + Also, MatchLabelKeys cannot be set when LabelSelector isn't set. + This is an alpha field and requires enabling MatchLabelKeysInPodAffinity feature gate. + items: + type: string + type: array + x-kubernetes-list-type: atomic + mismatchLabelKeys: + description: |- + MismatchLabelKeys is a set of pod label keys to select which pods will + be taken into consideration. The keys are used to lookup values from the + incoming pod labels, those key-value labels are merged with `LabelSelector` as `key notin (value)` + to select the group of existing pods which pods will be taken into consideration + for the incoming pod's pod (anti) affinity. Keys that don't exist in the incoming + pod labels will be ignored. The default value is empty. + The same key is forbidden to exist in both MismatchLabelKeys and LabelSelector. + Also, MismatchLabelKeys cannot be set when LabelSelector isn't set. + This is an alpha field and requires enabling MatchLabelKeysInPodAffinity feature gate. + items: + type: string + type: array + x-kubernetes-list-type: atomic + namespaceSelector: + description: |- + A label query over the set of namespaces that the term applies to. + The term is applied to the union of the namespaces selected by this field + and the ones listed in the namespaces field. + null selector and null or empty namespaces list means "this pod's namespace". + An empty selector ({}) matches all namespaces. + properties: + matchExpressions: + description: matchExpressions is a list + of label selector requirements. The requirements + are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key + that the selector applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + namespaces: + description: |- + namespaces specifies a static list of namespace names that the term applies to. + The term is applied to the union of the namespaces listed in this field + and the ones selected by namespaceSelector. + null or empty namespaces list and null namespaceSelector means "this pod's namespace". + items: + type: string + type: array + topologyKey: + description: |- + This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching + the labelSelector in the specified namespaces, where co-located is defined as running on a node + whose value of the label with key topologyKey matches that of any node on which any of the + selected pods is running. + Empty topologyKey is not allowed. + type: string + required: + - topologyKey + type: object + type: array + type: object + podAntiAffinity: + description: Describes pod anti-affinity scheduling rules + (e.g. avoid putting this pod in the same node, zone, + etc. as some other pod(s)). + properties: + preferredDuringSchedulingIgnoredDuringExecution: + description: |- + The scheduler will prefer to schedule pods to nodes that satisfy + the anti-affinity expressions specified by this field, but it may choose + a node that violates one or more of the expressions. The node that is + most preferred is the one with the greatest sum of weights, i.e. + for each node that meets all of the scheduling requirements (resource + request, requiredDuringScheduling anti-affinity expressions, etc.), + compute a sum by iterating through the elements of this field and adding + "weight" to the sum if the node has pods which matches the corresponding podAffinityTerm; the + node(s) with the highest sum are the most preferred. + items: + description: The weights of all of the matched WeightedPodAffinityTerm + fields are added per-node to find the most preferred + node(s) + properties: + podAffinityTerm: + description: Required. A pod affinity term, + associated with the corresponding weight. + properties: + labelSelector: + description: |- + A label query over a set of resources, in this case pods. + If it's null, this PodAffinityTerm matches with no Pods. + properties: + matchExpressions: + description: matchExpressions is a list + of label selector requirements. The + requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label + key that the selector applies + to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + matchLabelKeys: + description: |- + MatchLabelKeys is a set of pod label keys to select which pods will + be taken into consideration. The keys are used to lookup values from the + incoming pod labels, those key-value labels are merged with `LabelSelector` as `key in (value)` + to select the group of existing pods which pods will be taken into consideration + for the incoming pod's pod (anti) affinity. Keys that don't exist in the incoming + pod labels will be ignored. The default value is empty. + The same key is forbidden to exist in both MatchLabelKeys and LabelSelector. + Also, MatchLabelKeys cannot be set when LabelSelector isn't set. + This is an alpha field and requires enabling MatchLabelKeysInPodAffinity feature gate. + items: + type: string + type: array + x-kubernetes-list-type: atomic + mismatchLabelKeys: + description: |- + MismatchLabelKeys is a set of pod label keys to select which pods will + be taken into consideration. The keys are used to lookup values from the + incoming pod labels, those key-value labels are merged with `LabelSelector` as `key notin (value)` + to select the group of existing pods which pods will be taken into consideration + for the incoming pod's pod (anti) affinity. Keys that don't exist in the incoming + pod labels will be ignored. The default value is empty. + The same key is forbidden to exist in both MismatchLabelKeys and LabelSelector. + Also, MismatchLabelKeys cannot be set when LabelSelector isn't set. + This is an alpha field and requires enabling MatchLabelKeysInPodAffinity feature gate. + items: + type: string + type: array + x-kubernetes-list-type: atomic + namespaceSelector: + description: |- + A label query over the set of namespaces that the term applies to. + The term is applied to the union of the namespaces selected by this field + and the ones listed in the namespaces field. + null selector and null or empty namespaces list means "this pod's namespace". + An empty selector ({}) matches all namespaces. + properties: + matchExpressions: + description: matchExpressions is a list + of label selector requirements. The + requirements are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label + key that the selector applies + to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + namespaces: + description: |- + namespaces specifies a static list of namespace names that the term applies to. + The term is applied to the union of the namespaces listed in this field + and the ones selected by namespaceSelector. + null or empty namespaces list and null namespaceSelector means "this pod's namespace". + items: + type: string + type: array + topologyKey: + description: |- + This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching + the labelSelector in the specified namespaces, where co-located is defined as running on a node + whose value of the label with key topologyKey matches that of any node on which any of the + selected pods is running. + Empty topologyKey is not allowed. + type: string + required: + - topologyKey + type: object + weight: + description: |- + weight associated with matching the corresponding podAffinityTerm, + in the range 1-100. + format: int32 + type: integer + required: + - podAffinityTerm + - weight + type: object + type: array + requiredDuringSchedulingIgnoredDuringExecution: + description: |- + If the anti-affinity requirements specified by this field are not met at + scheduling time, the pod will not be scheduled onto the node. + If the anti-affinity requirements specified by this field cease to be met + at some point during pod execution (e.g. due to a pod label update), the + system may or may not try to eventually evict the pod from its node. + When there are multiple elements, the lists of nodes corresponding to each + podAffinityTerm are intersected, i.e. all terms must be satisfied. + items: + description: |- + Defines a set of pods (namely those matching the labelSelector + relative to the given namespace(s)) that this pod should be + co-located (affinity) or not co-located (anti-affinity) with, + where co-located is defined as running on a node whose value of + the label with key matches that of any node on which + a pod of the set of pods is running + properties: + labelSelector: + description: |- + A label query over a set of resources, in this case pods. + If it's null, this PodAffinityTerm matches with no Pods. + properties: + matchExpressions: + description: matchExpressions is a list + of label selector requirements. The requirements + are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key + that the selector applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + matchLabelKeys: + description: |- + MatchLabelKeys is a set of pod label keys to select which pods will + be taken into consideration. The keys are used to lookup values from the + incoming pod labels, those key-value labels are merged with `LabelSelector` as `key in (value)` + to select the group of existing pods which pods will be taken into consideration + for the incoming pod's pod (anti) affinity. Keys that don't exist in the incoming + pod labels will be ignored. The default value is empty. + The same key is forbidden to exist in both MatchLabelKeys and LabelSelector. + Also, MatchLabelKeys cannot be set when LabelSelector isn't set. + This is an alpha field and requires enabling MatchLabelKeysInPodAffinity feature gate. + items: + type: string + type: array + x-kubernetes-list-type: atomic + mismatchLabelKeys: + description: |- + MismatchLabelKeys is a set of pod label keys to select which pods will + be taken into consideration. The keys are used to lookup values from the + incoming pod labels, those key-value labels are merged with `LabelSelector` as `key notin (value)` + to select the group of existing pods which pods will be taken into consideration + for the incoming pod's pod (anti) affinity. Keys that don't exist in the incoming + pod labels will be ignored. The default value is empty. + The same key is forbidden to exist in both MismatchLabelKeys and LabelSelector. + Also, MismatchLabelKeys cannot be set when LabelSelector isn't set. + This is an alpha field and requires enabling MatchLabelKeysInPodAffinity feature gate. + items: + type: string + type: array + x-kubernetes-list-type: atomic + namespaceSelector: + description: |- + A label query over the set of namespaces that the term applies to. + The term is applied to the union of the namespaces selected by this field + and the ones listed in the namespaces field. + null selector and null or empty namespaces list means "this pod's namespace". + An empty selector ({}) matches all namespaces. + properties: + matchExpressions: + description: matchExpressions is a list + of label selector requirements. The requirements + are ANDed. + items: + description: |- + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. + properties: + key: + description: key is the label key + that the selector applies to. + type: string + operator: + description: |- + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. + type: string + values: + description: |- + values is an array of string values. If the operator is In or NotIn, + the values array must be non-empty. If the operator is Exists or DoesNotExist, + the values array must be empty. This array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object + type: object + x-kubernetes-map-type: atomic + namespaces: + description: |- + namespaces specifies a static list of namespace names that the term applies to. + The term is applied to the union of the namespaces listed in this field + and the ones selected by namespaceSelector. + null or empty namespaces list and null namespaceSelector means "this pod's namespace". + items: + type: string + type: array + topologyKey: + description: |- + This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching + the labelSelector in the specified namespaces, where co-located is defined as running on a node + whose value of the label with key topologyKey matches that of any node on which any of the + selected pods is running. + Empty topologyKey is not allowed. + type: string + required: + - topologyKey + type: object + type: array + type: object + type: object + annotations: + additionalProperties: + type: string + type: object + containerSecurityContext: + description: |- + SecurityContext holds security configuration that will be applied to a container. + Some fields are present in both SecurityContext and PodSecurityContext. When both + are set, the values in SecurityContext take precedence. + properties: + allowPrivilegeEscalation: + description: |- + AllowPrivilegeEscalation controls whether a process can gain more + privileges than its parent process. This bool directly controls if + the no_new_privs flag will be set on the container process. + AllowPrivilegeEscalation is true always when the container is: + 1) run as Privileged + 2) has CAP_SYS_ADMIN + Note that this field cannot be set when spec.os.name is windows. + type: boolean + capabilities: + description: |- + The capabilities to add/drop when running containers. + Defaults to the default set of capabilities granted by the container runtime. + Note that this field cannot be set when spec.os.name is windows. + properties: + add: + description: Added capabilities + items: + description: Capability represent POSIX capabilities + type + type: string + type: array + drop: + description: Removed capabilities + items: + description: Capability represent POSIX capabilities + type + type: string + type: array + type: object + privileged: + description: |- + Run container in privileged mode. + Processes in privileged containers are essentially equivalent to root on the host. + Defaults to false. + Note that this field cannot be set when spec.os.name is windows. + type: boolean + procMount: + description: |- + procMount denotes the type of proc mount to use for the containers. + The default is DefaultProcMount which uses the container runtime defaults for + readonly paths and masked paths. + This requires the ProcMountType feature flag to be enabled. + Note that this field cannot be set when spec.os.name is windows. + type: string + readOnlyRootFilesystem: + description: |- + Whether this container has a read-only root filesystem. + Default is false. + Note that this field cannot be set when spec.os.name is windows. + type: boolean + runAsGroup: + description: |- + The GID to run the entrypoint of the container process. + Uses runtime default if unset. + May also be set in PodSecurityContext. If set in both SecurityContext and + PodSecurityContext, the value specified in SecurityContext takes precedence. + Note that this field cannot be set when spec.os.name is windows. + format: int64 + type: integer + runAsNonRoot: + description: |- + Indicates that the container must run as a non-root user. + If true, the Kubelet will validate the image at runtime to ensure that it + does not run as UID 0 (root) and fail to start the container if it does. + If unset or false, no such validation will be performed. + May also be set in PodSecurityContext. If set in both SecurityContext and + PodSecurityContext, the value specified in SecurityContext takes precedence. + type: boolean + runAsUser: + description: |- + The UID to run the entrypoint of the container process. + Defaults to user specified in image metadata if unspecified. + May also be set in PodSecurityContext. If set in both SecurityContext and + PodSecurityContext, the value specified in SecurityContext takes precedence. + Note that this field cannot be set when spec.os.name is windows. + format: int64 + type: integer + seLinuxOptions: + description: |- + The SELinux context to be applied to the container. + If unspecified, the container runtime will allocate a random SELinux context for each + container. May also be set in PodSecurityContext. If set in both SecurityContext and + PodSecurityContext, the value specified in SecurityContext takes precedence. + Note that this field cannot be set when spec.os.name is windows. + properties: + level: + description: Level is SELinux level label that applies + to the container. + type: string + role: + description: Role is a SELinux role label that applies + to the container. + type: string + type: + description: Type is a SELinux type label that applies + to the container. + type: string + user: + description: User is a SELinux user label that applies + to the container. + type: string + type: object + seccompProfile: + description: |- + The seccomp options to use by this container. If seccomp options are + provided at both the pod & container level, the container options + override the pod options. + Note that this field cannot be set when spec.os.name is windows. + properties: + localhostProfile: + description: |- + localhostProfile indicates a profile defined in a file on the node should be used. + The profile must be preconfigured on the node to work. + Must be a descending path, relative to the kubelet's configured seccomp profile location. + Must be set if type is "Localhost". Must NOT be set for any other type. + type: string + type: + description: |- + type indicates which kind of seccomp profile will be applied. + Valid options are: + + + Localhost - a profile defined in a file on the node should be used. + RuntimeDefault - the container runtime default profile should be used. + Unconfined - no profile should be applied. + type: string + required: + - type + type: object + windowsOptions: + description: |- + The Windows specific settings applied to all containers. + If unspecified, the options from the PodSecurityContext will be used. + If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence. + Note that this field cannot be set when spec.os.name is linux. + properties: + gmsaCredentialSpec: + description: |- + GMSACredentialSpec is where the GMSA admission webhook + (https://github.com/kubernetes-sigs/windows-gmsa) inlines the contents of the + GMSA credential spec named by the GMSACredentialSpecName field. + type: string + gmsaCredentialSpecName: + description: GMSACredentialSpecName is the name of + the GMSA credential spec to use. + type: string + hostProcess: + description: |- + HostProcess determines if a container should be run as a 'Host Process' container. + All of a Pod's containers must have the same effective HostProcess value + (it is not allowed to have a mix of HostProcess containers and non-HostProcess containers). + In addition, if HostProcess is true then HostNetwork must also be set to true. + type: boolean + runAsUserName: + description: |- + The UserName in Windows to run the entrypoint of the container process. + Defaults to the user specified in image metadata if unspecified. + May also be set in PodSecurityContext. If set in both SecurityContext and + PodSecurityContext, the value specified in SecurityContext takes precedence. + type: string + type: object + type: object + imagePullPolicy: + description: PullPolicy describes a policy for if/when to + pull a container image + type: string + imagePullSecrets: + items: + description: |- + LocalObjectReference contains enough information to let you locate the + referenced object inside the same namespace. + properties: + name: + description: |- + Name of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid? + type: string + type: object + x-kubernetes-map-type: atomic + type: array + labels: + additionalProperties: + type: string + type: object + lifecycle: + description: |- + Lifecycle describes actions that the management system should take in response to container lifecycle + events. For the PostStart and PreStop lifecycle handlers, management of the container blocks + until the action is complete, unless the container process fails, in which case the handler is aborted. + properties: + postStart: + description: |- + PostStart is called immediately after a container is created. If the handler fails, + the container is terminated and restarted according to its restart policy. + Other management of the container blocks until the hook completes. + More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: |- + Command is the command line to execute inside the container, the working directory for the + command is root ('/') in the container's filesystem. The command is simply exec'd, it is + not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use + a shell, you need to explicitly call out to that shell. + Exit status of 0 is treated as live/healthy and non-zero is unhealthy. + items: + type: string + type: array + type: object + httpGet: + description: HTTPGet specifies the http request to + perform. + properties: + host: + description: |- + Host name to connect to, defaults to the pod IP. You probably want to set + "Host" in httpHeaders instead. + type: string + httpHeaders: + description: Custom headers to set in the request. + HTTP allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: |- + The header field name. + This will be canonicalized upon output, so case-variant names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: |- + Name or number of the port to access on the container. + Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: |- + Scheme to use for connecting to the host. + Defaults to HTTP. + type: string + required: + - port + type: object + sleep: + description: Sleep represents the duration that the + container should sleep before being terminated. + properties: + seconds: + description: Seconds is the number of seconds + to sleep. + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + description: |- + Deprecated. TCPSocket is NOT supported as a LifecycleHandler and kept + for the backward compatibility. There are no validation of this field and + lifecycle hooks will fail in runtime when tcp handler is specified. + properties: + host: + description: 'Optional: Host name to connect to, + defaults to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: |- + Number or name of the port to access on the container. + Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + preStop: + description: |- + PreStop is called immediately before a container is terminated due to an + API request or management event such as liveness/startup probe failure, + preemption, resource contention, etc. The handler is not called if the + container crashes or exits. The Pod's termination grace period countdown begins before the + PreStop hook is executed. Regardless of the outcome of the handler, the + container will eventually terminate within the Pod's termination grace + period (unless delayed by finalizers). Other management of the container blocks until the hook completes + or until the termination grace period is reached. + More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: |- + Command is the command line to execute inside the container, the working directory for the + command is root ('/') in the container's filesystem. The command is simply exec'd, it is + not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use + a shell, you need to explicitly call out to that shell. + Exit status of 0 is treated as live/healthy and non-zero is unhealthy. + items: + type: string + type: array + type: object + httpGet: + description: HTTPGet specifies the http request to + perform. + properties: + host: + description: |- + Host name to connect to, defaults to the pod IP. You probably want to set + "Host" in httpHeaders instead. + type: string + httpHeaders: + description: Custom headers to set in the request. + HTTP allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: |- + The header field name. + This will be canonicalized upon output, so case-variant names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: |- + Name or number of the port to access on the container. + Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: |- + Scheme to use for connecting to the host. + Defaults to HTTP. + type: string + required: + - port + type: object + sleep: + description: Sleep represents the duration that the + container should sleep before being terminated. + properties: + seconds: + description: Seconds is the number of seconds + to sleep. + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + description: |- + Deprecated. TCPSocket is NOT supported as a LifecycleHandler and kept + for the backward compatibility. There are no validation of this field and + lifecycle hooks will fail in runtime when tcp handler is specified. + properties: + host: + description: 'Optional: Host name to connect to, + defaults to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: |- + Number or name of the port to access on the container. + Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + type: object + livenessProbe: + description: |- + Probe describes a health check to be performed against a container to determine whether it is + alive or ready to receive traffic. + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: |- + Command is the command line to execute inside the container, the working directory for the + command is root ('/') in the container's filesystem. The command is simply exec'd, it is + not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use + a shell, you need to explicitly call out to that shell. + Exit status of 0 is treated as live/healthy and non-zero is unhealthy. + items: + type: string + type: array + type: object + failureThreshold: + description: |- + Minimum consecutive failures for the probe to be considered failed after having succeeded. + Defaults to 3. Minimum value is 1. + format: int32 + type: integer + grpc: + description: GRPC specifies an action involving a GRPC + port. + properties: + port: + description: Port number of the gRPC service. Number + must be in the range 1 to 65535. + format: int32 + type: integer + service: + description: |- + Service is the name of the service to place in the gRPC HealthCheckRequest + (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md). + + + If this is not specified, the default behavior is defined by gRPC. + type: string + required: + - port + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: |- + Host name to connect to, defaults to the pod IP. You probably want to set + "Host" in httpHeaders instead. + type: string + httpHeaders: + description: Custom headers to set in the request. + HTTP allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: |- + The header field name. + This will be canonicalized upon output, so case-variant names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: |- + Name or number of the port to access on the container. + Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: |- + Scheme to use for connecting to the host. + Defaults to HTTP. + type: string + required: + - port + type: object + initialDelaySeconds: + description: |- + Number of seconds after the container has started before liveness probes are initiated. + More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes + format: int32 + type: integer + periodSeconds: + description: |- + How often (in seconds) to perform the probe. + Default to 10 seconds. Minimum value is 1. + format: int32 + type: integer + successThreshold: + description: |- + Minimum consecutive successes for the probe to be considered successful after having failed. + Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1. + format: int32 + type: integer + tcpSocket: + description: TCPSocket specifies an action involving a + TCP port. + properties: + host: + description: 'Optional: Host name to connect to, defaults + to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: |- + Number or name of the port to access on the container. + Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + description: |- + Optional duration in seconds the pod needs to terminate gracefully upon probe failure. + The grace period is the duration in seconds after the processes running in the pod are sent + a termination signal and the time when the processes are forcibly halted with a kill signal. + Set this value longer than the expected cleanup time for your process. + If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this + value overrides the value provided by the pod spec. + Value must be non-negative integer. The value zero indicates stop immediately via + the kill signal (no opportunity to shut down). + This is a beta field and requires enabling ProbeTerminationGracePeriod feature gate. + Minimum value is 1. spec.terminationGracePeriodSeconds is used if unset. + format: int64 + type: integer + timeoutSeconds: + description: |- + Number of seconds after which the probe times out. + Defaults to 1 second. Minimum value is 1. + More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes + format: int32 + type: integer + type: object + nodeSelector: + additionalProperties: + type: string + type: object + podSecurityContext: + description: |- + PodSecurityContext holds pod-level security attributes and common container settings. + Some fields are also present in container.securityContext. Field values of + container.securityContext take precedence over field values of PodSecurityContext. + properties: + fsGroup: + description: |- + A special supplemental group that applies to all containers in a pod. + Some volume types allow the Kubelet to change the ownership of that volume + to be owned by the pod: + + + 1. The owning GID will be the FSGroup + 2. The setgid bit is set (new files created in the volume will be owned by FSGroup) + 3. The permission bits are OR'd with rw-rw---- + + + If unset, the Kubelet will not modify the ownership and permissions of any volume. + Note that this field cannot be set when spec.os.name is windows. + format: int64 + type: integer + fsGroupChangePolicy: + description: |- + fsGroupChangePolicy defines behavior of changing ownership and permission of the volume + before being exposed inside Pod. This field will only apply to + volume types which support fsGroup based ownership(and permissions). + It will have no effect on ephemeral volume types such as: secret, configmaps + and emptydir. + Valid values are "OnRootMismatch" and "Always". If not specified, "Always" is used. + Note that this field cannot be set when spec.os.name is windows. + type: string + runAsGroup: + description: |- + The GID to run the entrypoint of the container process. + Uses runtime default if unset. + May also be set in SecurityContext. If set in both SecurityContext and + PodSecurityContext, the value specified in SecurityContext takes precedence + for that container. + Note that this field cannot be set when spec.os.name is windows. + format: int64 + type: integer + runAsNonRoot: + description: |- + Indicates that the container must run as a non-root user. + If true, the Kubelet will validate the image at runtime to ensure that it + does not run as UID 0 (root) and fail to start the container if it does. + If unset or false, no such validation will be performed. + May also be set in SecurityContext. If set in both SecurityContext and + PodSecurityContext, the value specified in SecurityContext takes precedence. + type: boolean + runAsUser: + description: |- + The UID to run the entrypoint of the container process. + Defaults to user specified in image metadata if unspecified. + May also be set in SecurityContext. If set in both SecurityContext and + PodSecurityContext, the value specified in SecurityContext takes precedence + for that container. + Note that this field cannot be set when spec.os.name is windows. + format: int64 + type: integer + seLinuxOptions: + description: |- + The SELinux context to be applied to all containers. + If unspecified, the container runtime will allocate a random SELinux context for each + container. May also be set in SecurityContext. If set in + both SecurityContext and PodSecurityContext, the value specified in SecurityContext + takes precedence for that container. + Note that this field cannot be set when spec.os.name is windows. + properties: + level: + description: Level is SELinux level label that applies + to the container. + type: string + role: + description: Role is a SELinux role label that applies + to the container. + type: string + type: + description: Type is a SELinux type label that applies + to the container. + type: string + user: + description: User is a SELinux user label that applies + to the container. + type: string + type: object + seccompProfile: + description: |- + The seccomp options to use by the containers in this pod. + Note that this field cannot be set when spec.os.name is windows. + properties: + localhostProfile: + description: |- + localhostProfile indicates a profile defined in a file on the node should be used. + The profile must be preconfigured on the node to work. + Must be a descending path, relative to the kubelet's configured seccomp profile location. + Must be set if type is "Localhost". Must NOT be set for any other type. + type: string + type: + description: |- + type indicates which kind of seccomp profile will be applied. + Valid options are: + + + Localhost - a profile defined in a file on the node should be used. + RuntimeDefault - the container runtime default profile should be used. + Unconfined - no profile should be applied. + type: string + required: + - type + type: object + supplementalGroups: + description: |- + A list of groups applied to the first process run in each container, in addition + to the container's primary GID, the fsGroup (if specified), and group memberships + defined in the container image for the uid of the container process. If unspecified, + no additional groups are added to any container. Note that group memberships + defined in the container image for the uid of the container process are still effective, + even if they are not included in this list. + Note that this field cannot be set when spec.os.name is windows. + items: + format: int64 + type: integer + type: array + sysctls: + description: |- + Sysctls hold a list of namespaced sysctls used for the pod. Pods with unsupported + sysctls (by the container runtime) might fail to launch. + Note that this field cannot be set when spec.os.name is windows. + items: + description: Sysctl defines a kernel parameter to be + set + properties: + name: + description: Name of a property to set + type: string + value: + description: Value of a property to set + type: string + required: + - name + - value + type: object + type: array + windowsOptions: + description: |- + The Windows specific settings applied to all containers. + If unspecified, the options within a container's SecurityContext will be used. + If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence. + Note that this field cannot be set when spec.os.name is linux. + properties: + gmsaCredentialSpec: + description: |- + GMSACredentialSpec is where the GMSA admission webhook + (https://github.com/kubernetes-sigs/windows-gmsa) inlines the contents of the + GMSA credential spec named by the GMSACredentialSpecName field. + type: string + gmsaCredentialSpecName: + description: GMSACredentialSpecName is the name of + the GMSA credential spec to use. + type: string + hostProcess: + description: |- + HostProcess determines if a container should be run as a 'Host Process' container. + All of a Pod's containers must have the same effective HostProcess value + (it is not allowed to have a mix of HostProcess containers and non-HostProcess containers). + In addition, if HostProcess is true then HostNetwork must also be set to true. + type: boolean + runAsUserName: + description: |- + The UserName in Windows to run the entrypoint of the container process. + Defaults to the user specified in image metadata if unspecified. + May also be set in PodSecurityContext. If set in both SecurityContext and + PodSecurityContext, the value specified in SecurityContext takes precedence. + type: string + type: object + type: object + priorityClassName: type: string - serviceType: - description: Service Type string describes ingress methods - for a service + readinessProbe: + description: |- + Probe describes a health check to be performed against a container to determine whether it is + alive or ready to receive traffic. + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: |- + Command is the command line to execute inside the container, the working directory for the + command is root ('/') in the container's filesystem. The command is simply exec'd, it is + not run inside a shell, so traditional shell instructions ('|', etc) won't work. To use + a shell, you need to explicitly call out to that shell. + Exit status of 0 is treated as live/healthy and non-zero is unhealthy. + items: + type: string + type: array + type: object + failureThreshold: + description: |- + Minimum consecutive failures for the probe to be considered failed after having succeeded. + Defaults to 3. Minimum value is 1. + format: int32 + type: integer + grpc: + description: GRPC specifies an action involving a GRPC + port. + properties: + port: + description: Port number of the gRPC service. Number + must be in the range 1 to 65535. + format: int32 + type: integer + service: + description: |- + Service is the name of the service to place in the gRPC HealthCheckRequest + (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md). + + + If this is not specified, the default behavior is defined by gRPC. + type: string + required: + - port + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: |- + Host name to connect to, defaults to the pod IP. You probably want to set + "Host" in httpHeaders instead. + type: string + httpHeaders: + description: Custom headers to set in the request. + HTTP allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: |- + The header field name. + This will be canonicalized upon output, so case-variant names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: |- + Name or number of the port to access on the container. + Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: |- + Scheme to use for connecting to the host. + Defaults to HTTP. + type: string + required: + - port + type: object + initialDelaySeconds: + description: |- + Number of seconds after the container has started before liveness probes are initiated. + More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes + format: int32 + type: integer + periodSeconds: + description: |- + How often (in seconds) to perform the probe. + Default to 10 seconds. Minimum value is 1. + format: int32 + type: integer + successThreshold: + description: |- + Minimum consecutive successes for the probe to be considered successful after having failed. + Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1. + format: int32 + type: integer + tcpSocket: + description: TCPSocket specifies an action involving a + TCP port. + properties: + host: + description: 'Optional: Host name to connect to, defaults + to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: |- + Number or name of the port to access on the container. + Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + description: |- + Optional duration in seconds the pod needs to terminate gracefully upon probe failure. + The grace period is the duration in seconds after the processes running in the pod are sent + a termination signal and the time when the processes are forcibly halted with a kill signal. + Set this value longer than the expected cleanup time for your process. + If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this + value overrides the value provided by the pod spec. + Value must be non-negative integer. The value zero indicates stop immediately via + the kill signal (no opportunity to shut down). + This is a beta field and requires enabling ProbeTerminationGracePeriod feature gate. + Minimum value is 1. spec.terminationGracePeriodSeconds is used if unset. + format: int64 + type: integer + timeoutSeconds: + description: |- + Number of seconds after which the probe times out. + Defaults to 1 second. Minimum value is 1. + More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes + format: int32 + type: integer + type: object + resources: + description: ResourceRequirements describes the compute resource + requirements. + properties: + claims: + description: |- + Claims lists the names of resources, defined in spec.resourceClaims, + that are used by this container. + + + This is an alpha field and requires enabling the + DynamicResourceAllocation feature gate. + + + This field is immutable. It can only be set for containers. + items: + description: ResourceClaim references one entry in PodSpec.ResourceClaims. + properties: + name: + description: |- + Name must match the name of one entry in pod.spec.resourceClaims of + the Pod where this field is used. It makes that resource available + inside a container. + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: |- + Limits describes the maximum amount of compute resources allowed. + More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/ + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: |- + Requests describes the minimum amount of compute resources required. + If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, + otherwise to an implementation-defined value. Requests cannot exceed Limits. + More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/ + type: object + type: object + serviceAccountName: type: string - type: object - required: - - edgebound - - servicebound - type: object - frontlas: - properties: - controlplane: - properties: - frontierPlanePort: - type: integer - port: + terminationGracePeriodSeconds: + format: int64 type: integer - service: - type: string - serviceType: - description: Service Type string describes ingress methods - for a service - type: string - type: object - image: - type: string - nodeAffinity: - description: Node affinity is a group of node affinity scheduling - rules. - properties: - preferredDuringSchedulingIgnoredDuringExecution: - description: |- - The scheduler will prefer to schedule pods to nodes that satisfy - the affinity expressions specified by this field, but it may choose - a node that violates one or more of the expressions. The node that is - most preferred is the one with the greatest sum of weights, i.e. - for each node that meets all of the scheduling requirements (resource - request, requiredDuringScheduling affinity expressions, etc.), - compute a sum by iterating through the elements of this field and adding - "weight" to the sum if the node matches the corresponding matchExpressions; the - node(s) with the highest sum are the most preferred. + tolerations: items: description: |- - An empty preferred scheduling term matches all objects with implicit weight 0 - (i.e. it's a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op). + The pod this Toleration is attached to tolerates any taint that matches + the triple using the matching operator . properties: - preference: - description: A node selector term, associated with the - corresponding weight. + effect: + description: |- + Effect indicates the taint effect to match. Empty means match all taint effects. + When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute. + type: string + key: + description: |- + Key is the taint key that the toleration applies to. Empty means match all taint keys. + If the key is empty, operator must be Exists; this combination means to match all values and all keys. + type: string + operator: + description: |- + Operator represents a key's relationship to the value. + Valid operators are Exists and Equal. Defaults to Equal. + Exists is equivalent to wildcard for value, so that a pod can + tolerate all taints of a particular category. + type: string + tolerationSeconds: + description: |- + TolerationSeconds represents the period of time the toleration (which must be + of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, + it is not set, which means tolerate the taint forever (do not evict). Zero and + negative values will be treated as 0 (evict immediately) by the system. + format: int64 + type: integer + value: + description: |- + Value is the taint value the toleration matches to. + If the operator is Exists, the value should be empty, otherwise just a regular string. + type: string + type: object + type: array + topologySpreadConstraints: + items: + description: TopologySpreadConstraint specifies how to spread + matching pods among the given topology. + properties: + labelSelector: + description: |- + LabelSelector is used to find matching pods. + Pods that match this label selector are counted to determine the number of pods + in their corresponding topology domain. properties: matchExpressions: - description: A list of node selector requirements - by node's labels. - items: - description: |- - A node selector requirement is a selector that contains values, a key, and an operator - that relates the key and values. - properties: - key: - description: The label key that the selector - applies to. - type: string - operator: - description: |- - Represents a key's relationship to a set of values. - Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt. - type: string - values: - description: |- - An array of string values. If the operator is In or NotIn, - the values array must be non-empty. If the operator is Exists or DoesNotExist, - the values array must be empty. If the operator is Gt or Lt, the values - array must have a single element, which will be interpreted as an integer. - This array is replaced during a strategic merge patch. - items: - type: string - type: array - required: - - key - - operator - type: object - type: array - matchFields: - description: A list of node selector requirements - by node's fields. + description: matchExpressions is a list of label + selector requirements. The requirements are ANDed. items: description: |- - A node selector requirement is a selector that contains values, a key, and an operator - that relates the key and values. + A label selector requirement is a selector that contains values, a key, and an operator that + relates the key and values. properties: key: - description: The label key that the selector - applies to. + description: key is the label key that the + selector applies to. type: string operator: description: |- - Represents a key's relationship to a set of values. - Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt. + operator represents a key's relationship to a set of values. + Valid operators are In, NotIn, Exists and DoesNotExist. type: string values: description: |- - An array of string values. If the operator is In or NotIn, + values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, - the values array must be empty. If the operator is Gt or Lt, the values - array must have a single element, which will be interpreted as an integer. - This array is replaced during a strategic merge patch. + the values array must be empty. This array is replaced during a strategic + merge patch. items: type: string type: array @@ -410,106 +4558,144 @@ spec: - operator type: object type: array + matchLabels: + additionalProperties: + type: string + description: |- + matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, whose key field is "key", the + operator is "In", and the values array contains only "value". The requirements are ANDed. + type: object type: object x-kubernetes-map-type: atomic - weight: - description: Weight associated with matching the corresponding - nodeSelectorTerm, in the range 1-100. + matchLabelKeys: + description: |- + MatchLabelKeys is a set of pod label keys to select the pods over which + spreading will be calculated. The keys are used to lookup values from the + incoming pod labels, those key-value labels are ANDed with labelSelector + to select the group of existing pods over which spreading will be calculated + for the incoming pod. The same key is forbidden to exist in both MatchLabelKeys and LabelSelector. + MatchLabelKeys cannot be set when LabelSelector isn't set. + Keys that don't exist in the incoming pod labels will + be ignored. A null or empty list means only match against labelSelector. + + + This is a beta field and requires the MatchLabelKeysInPodTopologySpread feature gate to be enabled (enabled by default). + items: + type: string + type: array + x-kubernetes-list-type: atomic + maxSkew: + description: |- + MaxSkew describes the degree to which pods may be unevenly distributed. + When `whenUnsatisfiable=DoNotSchedule`, it is the maximum permitted difference + between the number of matching pods in the target topology and the global minimum. + The global minimum is the minimum number of matching pods in an eligible domain + or zero if the number of eligible domains is less than MinDomains. + For example, in a 3-zone cluster, MaxSkew is set to 1, and pods with the same + labelSelector spread as 2/2/1: + In this case, the global minimum is 1. + | zone1 | zone2 | zone3 | + | P P | P P | P | + - if MaxSkew is 1, incoming pod can only be scheduled to zone3 to become 2/2/2; + scheduling it onto zone1(zone2) would make the ActualSkew(3-1) on zone1(zone2) + violate MaxSkew(1). + - if MaxSkew is 2, incoming pod can be scheduled onto any zone. + When `whenUnsatisfiable=ScheduleAnyway`, it is used to give higher precedence + to topologies that satisfy it. + It's a required field. Default value is 1 and 0 is not allowed. + format: int32 + type: integer + minDomains: + description: |- + MinDomains indicates a minimum number of eligible domains. + When the number of eligible domains with matching topology keys is less than minDomains, + Pod Topology Spread treats "global minimum" as 0, and then the calculation of Skew is performed. + And when the number of eligible domains with matching topology keys equals or greater than minDomains, + this value has no effect on scheduling. + As a result, when the number of eligible domains is less than minDomains, + scheduler won't schedule more than maxSkew Pods to those domains. + If value is nil, the constraint behaves as if MinDomains is equal to 1. + Valid values are integers greater than 0. + When value is not nil, WhenUnsatisfiable must be DoNotSchedule. + + + For example, in a 3-zone cluster, MaxSkew is set to 2, MinDomains is set to 5 and pods with the same + labelSelector spread as 2/2/2: + | zone1 | zone2 | zone3 | + | P P | P P | P P | + The number of domains is less than 5(MinDomains), so "global minimum" is treated as 0. + In this situation, new pod with the same labelSelector cannot be scheduled, + because computed skew will be 3(3 - 0) if new Pod is scheduled to any of the three zones, + it will violate MaxSkew. + + + This is a beta field and requires the MinDomainsInPodTopologySpread feature gate to be enabled (enabled by default). format: int32 type: integer + nodeAffinityPolicy: + description: |- + NodeAffinityPolicy indicates how we will treat Pod's nodeAffinity/nodeSelector + when calculating pod topology spread skew. Options are: + - Honor: only nodes matching nodeAffinity/nodeSelector are included in the calculations. + - Ignore: nodeAffinity/nodeSelector are ignored. All nodes are included in the calculations. + + + If this value is nil, the behavior is equivalent to the Honor policy. + This is a beta-level feature default enabled by the NodeInclusionPolicyInPodTopologySpread feature flag. + type: string + nodeTaintsPolicy: + description: |- + NodeTaintsPolicy indicates how we will treat node taints when calculating + pod topology spread skew. Options are: + - Honor: nodes without taints, along with tainted nodes for which the incoming pod + has a toleration, are included. + - Ignore: node taints are ignored. All nodes are included. + + + If this value is nil, the behavior is equivalent to the Ignore policy. + This is a beta-level feature default enabled by the NodeInclusionPolicyInPodTopologySpread feature flag. + type: string + topologyKey: + description: |- + TopologyKey is the key of node labels. Nodes that have a label with this key + and identical values are considered to be in the same topology. + We consider each as a "bucket", and try to put balanced number + of pods into each bucket. + We define a domain as a particular instance of a topology. + Also, we define an eligible domain as a domain whose nodes meet the requirements of + nodeAffinityPolicy and nodeTaintsPolicy. + e.g. If TopologyKey is "kubernetes.io/hostname", each Node is a domain of that topology. + And, if TopologyKey is "topology.kubernetes.io/zone", each zone is a domain of that topology. + It's a required field. + type: string + whenUnsatisfiable: + description: |- + WhenUnsatisfiable indicates how to deal with a pod if it doesn't satisfy + the spread constraint. + - DoNotSchedule (default) tells the scheduler not to schedule it. + - ScheduleAnyway tells the scheduler to schedule the pod in any location, + but giving higher precedence to topologies that would help reduce the + skew. + A constraint is considered "Unsatisfiable" for an incoming pod + if and only if every possible node assignment for that pod would violate + "MaxSkew" on some topology. + For example, in a 3-zone cluster, MaxSkew is set to 1, and pods with the same + labelSelector spread as 3/1/1: + | zone1 | zone2 | zone3 | + | P P P | P | P | + If WhenUnsatisfiable is set to DoNotSchedule, incoming pod can only be scheduled + to zone2(zone3) to become 3/2/1(3/1/2) as ActualSkew(2-1) on zone2(zone3) satisfies + MaxSkew(1). In other words, the cluster can still be imbalanced, but scheduler + won't make it *more* imbalanced. + It's a required field. + type: string required: - - preference - - weight + - maxSkew + - topologyKey + - whenUnsatisfiable type: object type: array - requiredDuringSchedulingIgnoredDuringExecution: - description: |- - If the affinity requirements specified by this field are not met at - scheduling time, the pod will not be scheduled onto the node. - If the affinity requirements specified by this field cease to be met - at some point during pod execution (e.g. due to an update), the system - may or may not try to eventually evict the pod from its node. - properties: - nodeSelectorTerms: - description: Required. A list of node selector terms. - The terms are ORed. - items: - description: |- - A null or empty node selector term matches no objects. The requirements of - them are ANDed. - The TopologySelectorTerm type implements a subset of the NodeSelectorTerm. - properties: - matchExpressions: - description: A list of node selector requirements - by node's labels. - items: - description: |- - A node selector requirement is a selector that contains values, a key, and an operator - that relates the key and values. - properties: - key: - description: The label key that the selector - applies to. - type: string - operator: - description: |- - Represents a key's relationship to a set of values. - Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt. - type: string - values: - description: |- - An array of string values. If the operator is In or NotIn, - the values array must be non-empty. If the operator is Exists or DoesNotExist, - the values array must be empty. If the operator is Gt or Lt, the values - array must have a single element, which will be interpreted as an integer. - This array is replaced during a strategic merge patch. - items: - type: string - type: array - required: - - key - - operator - type: object - type: array - matchFields: - description: A list of node selector requirements - by node's fields. - items: - description: |- - A node selector requirement is a selector that contains values, a key, and an operator - that relates the key and values. - properties: - key: - description: The label key that the selector - applies to. - type: string - operator: - description: |- - Represents a key's relationship to a set of values. - Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt. - type: string - values: - description: |- - An array of string values. If the operator is In or NotIn, - the values array must be non-empty. If the operator is Exists or DoesNotExist, - the values array must be empty. If the operator is Gt or Lt, the values - array must have a single element, which will be interpreted as an integer. - This array is replaced during a strategic merge patch. - items: - type: string - type: array - required: - - key - - operator - type: object - type: array - type: object - x-kubernetes-map-type: atomic - type: array - required: - - nodeSelectorTerms - type: object - x-kubernetes-map-type: atomic type: object redis: properties: @@ -522,7 +4708,33 @@ spec: masterName: type: string password: + description: |- + Password 是密码明文，会被原样写进 Pod 环境变量。 + Deprecated: 生产场景请改用 PasswordSecret，避免密钥进 spec / event / describe 输出。 type: string + passwordSecret: + description: |- + PasswordSecret 引用一个 Secret 中的字段作为 Redis 密码来源。 + 设置后优先级高于 Password，env 通过 valueFrom.secretKeyRef 注入。 + properties: + key: + description: The key of the secret to select from. Must + be a valid secret key. + type: string + name: + description: |- + Name of the referent. + More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid? + type: string + optional: + description: Specify whether the Secret or its key must + be defined + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic redisType: type: string user: From 3e34eb3f45ef098447514066ecf0f546db56faf5 Mon Sep 17 00:00:00 2001 From: singchia Date: Sat, 2 May 2026 00:09:37 +0800 Subject: [PATCH 05/12] feat(operator): production-grade defaults and PodOverrides wiring Wire PodOverrides through container_builder + podtemplatespec_builder so the Reconciler can render every covered field. The builders gain typed setters for Resources, LivenessProbe, Lifecycle, SecurityContext (container), TopologySpreadConstraints, PriorityClassName, ServiceAccountName, ImagePullSecrets, PodSecurityContext, full Affinity, plus annotation and label merge helpers. Add internal/controller/podoverrides.go: per-component default tables (frontierDefaults / frontlasDefaults) plus pickXxx helpers that resolve override-or-default for each field. The defaults encode several gospec red lines and operational best practices: - ImagePullPolicy: IfNotPresent (was PullAlways) - Pod / Container SecurityContext: runAsNonRoot=true, RunAsUser=65532, drop ALL capabilities, no privilege escalation, RuntimeDefault seccomp - TerminationGracePeriodSeconds: 60 for frontier (long-lived edge connections), 30 for frontlas - Lifecycle.PreStop: sleep 10, gives kube-proxy time to remove the pod from Service Endpoints before SIGTERM - LivenessProbe and ReadinessProbe: TCP socket on listener ports for frontier (HTTP /readyz endpoint comes in M3); Frontlas keeps its /cluster/v1/health readiness and gains a TCP liveness - PodAntiAffinity: PreferredDuringScheduling (was Required, which made small clusters fail to schedule when replicas exceeded node count) Redis password env switches to corev1.EnvVarSource SecretKeyRef when Redis.PasswordSecret is set; the legacy plaintext path still works when only Redis.Password is provided. Frontier deployment also gains FRONTIER_DRAIN_SECONDS env, computed as terminationGracePeriodSeconds - 10, to be consumed by the binary's shutdown handler (next commit). 12 unit tests cover the picker logic, default safety properties (nonRoot, drop ALL caps, preferred not required affinity), and Redis env routing. Refs: docs/rfc/RFC-001-cloud-native-optimization.md (M2) Co-Authored-By: Claude Opus 4.7 (1M context) --- .../controller/frontiercluster_deployment.go | 170 +++++++----- .../internal/controller/podoverrides.go | 248 ++++++++++++++++++ .../internal/controller/podoverrides_test.go | 158 +++++++++++ .../pkg/kube/container/container_builder.go | 42 ++- .../podtemplatespec_builder.go | 85 +++++- 5 files changed, 625 insertions(+), 78 deletions(-) create mode 100644 pkg/operator/internal/controller/podoverrides.go create mode 100644 pkg/operator/internal/controller/podoverrides_test.go diff --git a/pkg/operator/internal/controller/frontiercluster_deployment.go b/pkg/operator/internal/controller/frontiercluster_deployment.go index 402a001..bd018c3 100644 --- a/pkg/operator/internal/controller/frontiercluster_deployment.go +++ b/pkg/operator/internal/controller/frontiercluster_deployment.go @@ -12,7 +12,6 @@ import ( "github.com/singchia/frontier/operator/pkg/kube/deployment" "github.com/singchia/frontier/operator/pkg/kube/podtemplatespec" corev1 "k8s.io/api/core/v1" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "sigs.k8s.io/controller-runtime/pkg/log" ) @@ -25,10 +24,13 @@ const ( NodeNameEnv = "NODE_NAME" // port for frontier and frontlas - FrontierServiceboundPortEnv = "FRONTIER_SERVICEBOUND_PORT" - FrontierEdgeboundPortEnv = "FRONTIER_EDGEBOUND_PORT" - FrontlasControlPlanePortEnv = "FRONTLAS_CONTROLPLANE_PORT" - FrontlasFrontierPlanePortEnv = "FRONTLAS_FRONTIERPLANE_PORT" + FrontierServiceboundPortEnv = "FRONTIER_SERVICEBOUND_PORT" + FrontierEdgeboundPortEnv = "FRONTIER_EDGEBOUND_PORT" + FrontlasControlPlanePortEnv = "FRONTLAS_CONTROLPLANE_PORT" + FrontlasFrontierPlanePortEnv = "FRONTLAS_FRONTIERPLANE_PORT" + + // graceful shutdown + FrontierDrainSecondsEnv = "FRONTIER_DRAIN_SECONDS" // tls for frontier FrontierEdgeboundTLSCAMountPath = "/app/conf/edgebound/tls/ca" @@ -148,11 +150,20 @@ func (r *FrontierClusterReconciler) ensureFrontierDeployment(ctx context.Context _, _, ebport := fc.FrontierEdgeboundServicePort() frontierservice, _, _, fpport := fc.FrontlasServicePort() + defaults := frontierDefaults(sbport.Port, ebport.Port) + pod := fc.Spec.Frontier.Pod + gracePeriod := pickGrace(pod.TerminationGracePeriodSeconds, defaults.terminationGracePeriodSeconds) + // drain 默认 = grace - 10s，给 Close() 自身留余量；下界 0，不会变负。 + drainSeconds := gracePeriod - 10 + if drainSeconds < 0 { + drainSeconds = 0 + } + // container container := container.Builder(). SetName("frontier"). SetImage(image). - SetImagePullPolicy(corev1.PullAlways). + SetImagePullPolicy(pickPullPolicy(pod.ImagePullPolicy, defaults.imagePullPolicy)). SetEnvs([]corev1.EnvVar{{ Name: FrontierServiceboundPortEnv, Value: strconv.Itoa(int(sbport.Port)), @@ -169,39 +180,48 @@ func (r *FrontierClusterReconciler) ensureFrontierDeployment(ctx context.Context }, { Name: FrontlasAddrEnv, Value: net.JoinHostPort(frontierservice, strconv.Itoa(int(fpport.Port))), + }, { + Name: FrontierDrainSecondsEnv, + Value: strconv.FormatInt(drainSeconds, 10), }}). SetCommand(nil). SetArgs(nil). SetVolumeMounts(volumeMounts). + SetResources(pickResources(pod.Resources, defaults.resources)). + SetLivenessProbe(pickProbe(pod.LivenessProbe, defaults.livenessProbe)). + SetReadinessProbe(pickProbe(pod.ReadinessProbe, defaults.readinessProbe)). + SetLifecycle(pickLifecycle(pod.Lifecycle, defaults.lifecycle)). + SetSecurityContext(pickContainerSecurityContext(pod.ContainerSecurityContext, defaults.containerSecurityContext)). Build() + specOver := extractPodSpecOverrides(pod) + mergedLabels := map[string]string{} + for k, v := range labels { + mergedLabels[k] = v + } + // pod - podTemplateSpec := podtemplatespec.Builder(). + podBuilder := podtemplatespec.Builder(). SetName("frontier"). SetNamespace(fc.Namespace). AddVolumes(volumes). - SetLabels(labels). + SetLabels(mergedLabels). + MergeLabels(specOver.Labels). SetMatchLabels(labels). - SetNodeAffinity(&fc.Spec.Frontier.NodeAffinity). + SetAnnotations(specOver.Annotations). SetOwnerReference(fc.OwnerReferences). AddContainer(container). - SetPodAntiAffinity(&corev1.PodAntiAffinity{ - RequiredDuringSchedulingIgnoredDuringExecution: []corev1.PodAffinityTerm{ - { - TopologyKey: "kubernetes.io/hostname", - LabelSelector: &metav1.LabelSelector{ - MatchExpressions: []metav1.LabelSelectorRequirement{ - { - Key: "app", - Operator: metav1.LabelSelectorOpIn, - Values: []string{app}, - }, - }, - }, - }, - }, - }). - Build() + SetTerminationGracePeriodSeconds(gracePeriod). + SetTolerations(specOver.Tolerations). + SetTopologySpreadConstraints(specOver.TopologySpreadConstraints). + SetNodeSelector(specOver.NodeSelector). + SetPriorityClassName(specOver.PriorityClassName). + SetServiceAccountName(specOver.ServiceAccountName). + SetImagePullSecrets(specOver.ImagePullSecrets). + SetPodSecurityContext(pickPodSecurityContext(pod.PodSecurityContext, defaults.podSecurityContext)). + SetAffinity(pickAffinity(pod.Affinity, preferredAntiAffinityByHost(app), &fc.Spec.Frontier.NodeAffinity)) + + podTemplateSpec := podBuilder.Build() deploy, err := deployment.Builder(). SetName(fc.FrontierDeploymentNamespacedName().Name). @@ -233,64 +253,49 @@ func (r *FrontierClusterReconciler) ensureFrontlasDeployment(ctx context.Context service, _, cpport, fpport := fc.FrontlasServicePort() + defaults := frontlasDefaults(cpport.Port) + pod := fc.Spec.Frontlas.Pod + // container container := container.Builder(). SetName("frontlas"). SetImage(image). - SetImagePullPolicy(corev1.PullAlways). - SetEnvs([]corev1.EnvVar{{ - Name: FrontlasControlPlanePortEnv, - Value: strconv.Itoa(int(cpport.Port)), - }, { - Name: FrontlasFrontierPlanePortEnv, - Value: strconv.Itoa(int(fpport.Port)), - }, { - Name: FrontlasRedisAddrsEnv, - Value: strings.Join(fc.Spec.Frontlas.Redis.Addrs, ","), - }, { - Name: FrontlasRedisUserEnv, - Value: fc.Spec.Frontlas.Redis.User, - }, { - Name: FrontlasRedisPasswordEnv, - Value: fc.Spec.Frontlas.Redis.Password, - }, { - Name: FrontlasRedisTypeEnv, - Value: string(fc.Spec.Frontlas.Redis.RedisType), - }, { - Name: FrontlasRedisDBEnv, - Value: strconv.Itoa(fc.Spec.Frontlas.Redis.DB), - }, { - Name: FrontlasRedisMasterName, - Value: fc.Spec.Frontlas.Redis.MasterName, - }}). - SetReadinessProbe(&corev1.Probe{ - ProbeHandler: corev1.ProbeHandler{ - /* 1.24+ - GRPC: &corev1.GRPCAction{ - Port: cpport.TargetPort.IntVal, - Service: &service, - }, - */ - HTTPGet: &corev1.HTTPGetAction{ - Port: cpport.TargetPort, - Path: "/cluster/v1/health", - }, - }, - PeriodSeconds: 5, - }). + SetImagePullPolicy(pickPullPolicy(pod.ImagePullPolicy, defaults.imagePullPolicy)). + SetEnvs(frontlasRedisEnvs(fc, cpport.Port, fpport.Port)). SetCommand(nil). SetArgs(nil). + SetResources(pickResources(pod.Resources, defaults.resources)). + SetLivenessProbe(pickProbe(pod.LivenessProbe, defaults.livenessProbe)). + SetReadinessProbe(pickProbe(pod.ReadinessProbe, defaults.readinessProbe)). + SetLifecycle(pickLifecycle(pod.Lifecycle, defaults.lifecycle)). + SetSecurityContext(pickContainerSecurityContext(pod.ContainerSecurityContext, defaults.containerSecurityContext)). Build() + specOver := extractPodSpecOverrides(pod) + mergedLabels := map[string]string{} + for k, v := range labels { + mergedLabels[k] = v + } + // pod podTemplateSpec := podtemplatespec.Builder(). SetName("frontlas"). SetNamespace(fc.Namespace). - SetLabels(labels). + SetLabels(mergedLabels). + MergeLabels(specOver.Labels). SetMatchLabels(labels). - SetNodeAffinity(&fc.Spec.Frontlas.NodeAffinity). + SetAnnotations(specOver.Annotations). SetOwnerReference(fc.OwnerReferences). AddContainer(container). + SetTerminationGracePeriodSeconds(pickGrace(pod.TerminationGracePeriodSeconds, defaults.terminationGracePeriodSeconds)). + SetTolerations(specOver.Tolerations). + SetTopologySpreadConstraints(specOver.TopologySpreadConstraints). + SetNodeSelector(specOver.NodeSelector). + SetPriorityClassName(specOver.PriorityClassName). + SetServiceAccountName(specOver.ServiceAccountName). + SetImagePullSecrets(specOver.ImagePullSecrets). + SetPodSecurityContext(pickPodSecurityContext(pod.PodSecurityContext, defaults.podSecurityContext)). + SetAffinity(pickAffinity(pod.Affinity, preferredAntiAffinityByHost(app), &fc.Spec.Frontlas.NodeAffinity)). Build() deploy, err := deployment.Builder(). @@ -310,3 +315,30 @@ func (r *FrontierClusterReconciler) ensureFrontlasDeployment(ctx context.Context _, err = deployment.CreateOrUpdate(ctx, r.client, deploy) return err } + +// frontlasRedisEnvs 把端口 + Redis 配置组装成 EnvVar， +// Password 与 PasswordSecret 互斥（PasswordSecret 优先，进 valueFrom；否则走明文兼容路径）。 +func frontlasRedisEnvs(fc v1alpha1.FrontierCluster, cpPort, fpPort int32) []corev1.EnvVar { + r := fc.Spec.Frontlas.Redis + envs := []corev1.EnvVar{ + {Name: FrontlasControlPlanePortEnv, Value: strconv.Itoa(int(cpPort))}, + {Name: FrontlasFrontierPlanePortEnv, Value: strconv.Itoa(int(fpPort))}, + {Name: FrontlasRedisAddrsEnv, Value: strings.Join(r.Addrs, ",")}, + {Name: FrontlasRedisUserEnv, Value: r.User}, + {Name: FrontlasRedisTypeEnv, Value: string(r.RedisType)}, + {Name: FrontlasRedisDBEnv, Value: strconv.Itoa(r.DB)}, + {Name: FrontlasRedisMasterName, Value: r.MasterName}, + } + if r.PasswordSecret != nil { + envs = append(envs, corev1.EnvVar{ + Name: FrontlasRedisPasswordEnv, + ValueFrom: &corev1.EnvVarSource{SecretKeyRef: r.PasswordSecret}, + }) + } else { + envs = append(envs, corev1.EnvVar{ + Name: FrontlasRedisPasswordEnv, + Value: r.Password, + }) + } + return envs +} diff --git a/pkg/operator/internal/controller/podoverrides.go b/pkg/operator/internal/controller/podoverrides.go new file mode 100644 index 0000000..226c67d --- /dev/null +++ b/pkg/operator/internal/controller/podoverrides.go @@ -0,0 +1,248 @@ +package controller + +import ( + "strconv" + + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/util/intstr" + + "github.com/singchia/frontier/operator/api/v1alpha1" +) + +func intToStr(i int) string { + return strconv.Itoa(i) +} + +func labelSelectorForApp(app string) *metav1.LabelSelector { + return &metav1.LabelSelector{ + MatchExpressions: []metav1.LabelSelectorRequirement{ + { + Key: "app", + Operator: metav1.LabelSelectorOpIn, + Values: []string{app}, + }, + }, + } +} + +// componentDefaults 是 operator 给 Frontier / Frontlas 各自的硬编码默认值。 +// 用户在 spec 里没填的项走这里。 +type componentDefaults struct { + livenessProbe *corev1.Probe + readinessProbe *corev1.Probe + lifecycle *corev1.Lifecycle + terminationGracePeriodSeconds int64 + resources *corev1.ResourceRequirements + podAntiAffinity *corev1.PodAntiAffinity + podSecurityContext *corev1.PodSecurityContext + containerSecurityContext *corev1.SecurityContext + imagePullPolicy corev1.PullPolicy +} + +// 通用默认值：非 root + 删掉所有 capability + 关掉特权升级 + RuntimeDefault seccomp。 +// 这些是 gospec "容器以非 root 用户运行" 红线 + Pod Security Standards "restricted" profile 的最小集。 +func defaultPodSecurityContext() *corev1.PodSecurityContext { + uid := int64(65532) + runAsNonRoot := true + return &corev1.PodSecurityContext{ + RunAsNonRoot: &runAsNonRoot, + RunAsUser: &uid, + RunAsGroup: &uid, + FSGroup: &uid, + SeccompProfile: &corev1.SeccompProfile{ + Type: corev1.SeccompProfileTypeRuntimeDefault, + }, + } +} + +func defaultContainerSecurityContext() *corev1.SecurityContext { + allowPrivEsc := false + runAsNonRoot := true + return &corev1.SecurityContext{ + AllowPrivilegeEscalation: &allowPrivEsc, + RunAsNonRoot: &runAsNonRoot, + Capabilities: &corev1.Capabilities{ + Drop: []corev1.Capability{"ALL"}, + }, + } +} + +// preStop 缓冲：让 K8s 把 Pod 从 Service Endpoints 摘除一段时间，再发 SIGTERM。 +// 长连接型 frontier 必须有，避免在 endpoint 还没传播到 kube-proxy 时连接被秒断。 +func defaultLifecycle(preStopSleepSeconds int) *corev1.Lifecycle { + return &corev1.Lifecycle{ + PreStop: &corev1.LifecycleHandler{ + Exec: &corev1.ExecAction{ + Command: []string{"/bin/sh", "-c", "sleep " + intToStr(preStopSleepSeconds)}, + }, + }, + } +} + +// 跨 host 反亲和（preferred 而非 required，避免在小集群里调度不出去）。 +func preferredAntiAffinityByHost(appLabel string) *corev1.PodAntiAffinity { + return &corev1.PodAntiAffinity{ + PreferredDuringSchedulingIgnoredDuringExecution: []corev1.WeightedPodAffinityTerm{ + { + Weight: 100, + PodAffinityTerm: corev1.PodAffinityTerm{ + TopologyKey: "kubernetes.io/hostname", + LabelSelector: labelSelectorForApp(appLabel), + }, + }, + }, + } +} + +// frontierDefaults / frontlasDefaults 由 ensureXxxDeployment 调用，参数化各自端口。 +func frontierDefaults(servicePort, edgePort int32) componentDefaults { + return componentDefaults{ + livenessProbe: &corev1.Probe{ + ProbeHandler: corev1.ProbeHandler{ + TCPSocket: &corev1.TCPSocketAction{Port: intstr.FromInt32(edgePort)}, + }, + InitialDelaySeconds: 10, + PeriodSeconds: 20, + TimeoutSeconds: 3, + FailureThreshold: 3, + }, + readinessProbe: &corev1.Probe{ + ProbeHandler: corev1.ProbeHandler{ + TCPSocket: &corev1.TCPSocketAction{Port: intstr.FromInt32(servicePort)}, + }, + InitialDelaySeconds: 5, + PeriodSeconds: 10, + TimeoutSeconds: 3, + FailureThreshold: 3, + }, + lifecycle: defaultLifecycle(10), + terminationGracePeriodSeconds: 60, + podSecurityContext: defaultPodSecurityContext(), + containerSecurityContext: defaultContainerSecurityContext(), + imagePullPolicy: corev1.PullIfNotPresent, + } +} + +func frontlasDefaults(controlPort int32) componentDefaults { + return componentDefaults{ + livenessProbe: &corev1.Probe{ + ProbeHandler: corev1.ProbeHandler{ + TCPSocket: &corev1.TCPSocketAction{Port: intstr.FromInt32(controlPort)}, + }, + InitialDelaySeconds: 10, + PeriodSeconds: 20, + TimeoutSeconds: 3, + FailureThreshold: 3, + }, + readinessProbe: &corev1.Probe{ + ProbeHandler: corev1.ProbeHandler{ + HTTPGet: &corev1.HTTPGetAction{ + Port: intstr.FromInt32(controlPort), + Path: "/cluster/v1/health", + }, + }, + PeriodSeconds: 5, + }, + lifecycle: defaultLifecycle(5), + terminationGracePeriodSeconds: 30, + podSecurityContext: defaultPodSecurityContext(), + containerSecurityContext: defaultContainerSecurityContext(), + imagePullPolicy: corev1.PullIfNotPresent, + } +} + +// pickXxx 是用户覆盖优先 / 默认兜底的小工具。 +// 任一字段被显式置 nil/零值都视为 "用默认"——这与 K8s +// 资源 spec "未设置走默认" 的语义一致。 + +func pickResources(o, def *corev1.ResourceRequirements) *corev1.ResourceRequirements { + if o != nil { + return o + } + return def +} + +func pickProbe(o, def *corev1.Probe) *corev1.Probe { + if o != nil { + return o + } + return def +} + +func pickLifecycle(o, def *corev1.Lifecycle) *corev1.Lifecycle { + if o != nil { + return o + } + return def +} + +func pickContainerSecurityContext(o, def *corev1.SecurityContext) *corev1.SecurityContext { + if o != nil { + return o + } + return def +} + +func pickPodSecurityContext(o, def *corev1.PodSecurityContext) *corev1.PodSecurityContext { + if o != nil { + return o + } + return def +} + +func pickPullPolicy(o, def corev1.PullPolicy) corev1.PullPolicy { + if o != "" { + return o + } + return def +} + +func pickGrace(o *int64, def int64) int64 { + if o != nil { + return *o + } + return def +} + +// pickAffinity：用户给 affinity 优先；否则用 operator 默认 PodAntiAffinity 打底， +// 同时保留 spec..NodeAffinity（v1alpha1 老字段）走 NodeAffinity 槽位。 +// 当用户传完整 PodOverrides.Affinity 时，老的 NodeAffinity 字段被忽略。 +func pickAffinity(o *corev1.Affinity, defaultAnti *corev1.PodAntiAffinity, legacyNodeAff *corev1.NodeAffinity) *corev1.Affinity { + if o != nil { + return o + } + out := &corev1.Affinity{ + PodAntiAffinity: defaultAnti, + } + if legacyNodeAff != nil && (len(legacyNodeAff.PreferredDuringSchedulingIgnoredDuringExecution) > 0 || legacyNodeAff.RequiredDuringSchedulingIgnoredDuringExecution != nil) { + out.NodeAffinity = legacyNodeAff + } + return out +} + +// 把 PodOverrides 里和 PodSpec 直接对应的"列表/标量"取出来，方便 deployment.go +// 一次塞进 podtemplatespec builder。 +type podSpecOverrides struct { + NodeSelector map[string]string + Tolerations []corev1.Toleration + TopologySpreadConstraints []corev1.TopologySpreadConstraint + PriorityClassName string + ServiceAccountName string + ImagePullSecrets []corev1.LocalObjectReference + Annotations map[string]string + Labels map[string]string +} + +func extractPodSpecOverrides(o v1alpha1.PodOverrides) podSpecOverrides { + return podSpecOverrides{ + NodeSelector: o.NodeSelector, + Tolerations: o.Tolerations, + TopologySpreadConstraints: o.TopologySpreadConstraints, + PriorityClassName: o.PriorityClassName, + ServiceAccountName: o.ServiceAccountName, + ImagePullSecrets: o.ImagePullSecrets, + Annotations: o.Annotations, + Labels: o.Labels, + } +} diff --git a/pkg/operator/internal/controller/podoverrides_test.go b/pkg/operator/internal/controller/podoverrides_test.go new file mode 100644 index 0000000..e104979 --- /dev/null +++ b/pkg/operator/internal/controller/podoverrides_test.go @@ -0,0 +1,158 @@ +package controller + +import ( + "testing" + + "github.com/stretchr/testify/assert" + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/api/resource" + + "github.com/singchia/frontier/operator/api/v1alpha1" +) + +func TestPickPullPolicy(t *testing.T) { + assert.Equal(t, corev1.PullIfNotPresent, pickPullPolicy("", corev1.PullIfNotPresent)) + assert.Equal(t, corev1.PullAlways, pickPullPolicy(corev1.PullAlways, corev1.PullIfNotPresent)) + assert.Equal(t, corev1.PullNever, pickPullPolicy(corev1.PullNever, corev1.PullIfNotPresent)) +} + +func TestPickResources_NilFallsBackToDefault(t *testing.T) { + def := &corev1.ResourceRequirements{ + Requests: corev1.ResourceList{ + corev1.ResourceCPU: resource.MustParse("100m"), + }, + } + assert.Same(t, def, pickResources(nil, def)) + + custom := &corev1.ResourceRequirements{ + Limits: corev1.ResourceList{ + corev1.ResourceCPU: resource.MustParse("2"), + }, + } + assert.Same(t, custom, pickResources(custom, def)) +} + +func TestPickGrace_DefaultWhenNil(t *testing.T) { + assert.Equal(t, int64(60), pickGrace(nil, 60)) + v := int64(120) + assert.Equal(t, int64(120), pickGrace(&v, 60)) +} + +func TestPickAffinity_UserAffinityFullyWins(t *testing.T) { + defaultAnti := preferredAntiAffinityByHost("foo") + userAff := &corev1.Affinity{ + NodeAffinity: &corev1.NodeAffinity{ + RequiredDuringSchedulingIgnoredDuringExecution: &corev1.NodeSelector{}, + }, + } + got := pickAffinity(userAff, defaultAnti, nil) + assert.Same(t, userAff, got) + assert.Nil(t, got.PodAntiAffinity, "用户 Affinity 全量替换默认，不再叠加 default anti-affinity") +} + +func TestPickAffinity_DefaultsKeepLegacyNodeAffinity(t *testing.T) { + defaultAnti := preferredAntiAffinityByHost("foo") + legacy := &corev1.NodeAffinity{ + PreferredDuringSchedulingIgnoredDuringExecution: []corev1.PreferredSchedulingTerm{ + {Weight: 1}, + }, + } + got := pickAffinity(nil, defaultAnti, legacy) + assert.NotNil(t, got) + assert.Same(t, defaultAnti, got.PodAntiAffinity) + assert.Same(t, legacy, got.NodeAffinity) +} + +func TestPickAffinity_EmptyLegacyAffinityIsIgnored(t *testing.T) { + defaultAnti := preferredAntiAffinityByHost("foo") + emptyLegacy := &corev1.NodeAffinity{} // 用户没填 NodeAffinity 时会传一个零值结构体进来 + got := pickAffinity(nil, defaultAnti, emptyLegacy) + assert.Nil(t, got.NodeAffinity, "空 NodeAffinity 不应进 PodSpec") + assert.Same(t, defaultAnti, got.PodAntiAffinity) +} + +func TestPreferredAntiAffinityByHost_IsPreferredNotRequired(t *testing.T) { + a := preferredAntiAffinityByHost("frontier") + assert.NotNil(t, a) + assert.Empty(t, a.RequiredDuringSchedulingIgnoredDuringExecution, "必须是 preferred 而非 required；否则小集群副本超 node 数即调度失败") + assert.Len(t, a.PreferredDuringSchedulingIgnoredDuringExecution, 1) + assert.Equal(t, int32(100), a.PreferredDuringSchedulingIgnoredDuringExecution[0].Weight) + assert.Equal(t, "kubernetes.io/hostname", a.PreferredDuringSchedulingIgnoredDuringExecution[0].PodAffinityTerm.TopologyKey) +} + +func TestDefaultPodSecurityContext_NonRoot(t *testing.T) { + psc := defaultPodSecurityContext() + assert.NotNil(t, psc.RunAsNonRoot) + assert.True(t, *psc.RunAsNonRoot, "默认必须 nonRoot——gospec 安全红线") + assert.NotNil(t, psc.RunAsUser) + assert.NotZero(t, *psc.RunAsUser) + assert.NotNil(t, psc.SeccompProfile) + assert.Equal(t, corev1.SeccompProfileTypeRuntimeDefault, psc.SeccompProfile.Type) +} + +func TestDefaultContainerSecurityContext_DropAll(t *testing.T) { + c := defaultContainerSecurityContext() + assert.NotNil(t, c.AllowPrivilegeEscalation) + assert.False(t, *c.AllowPrivilegeEscalation) + assert.NotNil(t, c.Capabilities) + assert.Contains(t, c.Capabilities.Drop, corev1.Capability("ALL")) +} + +func TestFrontlasRedisEnvs_PasswordSecretWinsOverPlaintext(t *testing.T) { + fc := v1alpha1.FrontierCluster{} + fc.Spec.Frontlas.Redis = v1alpha1.Redis{ + Addrs: []string{"r:6379"}, + Password: "should-not-leak", + PasswordSecret: &corev1.SecretKeySelector{ + LocalObjectReference: corev1.LocalObjectReference{Name: "redis-creds"}, + Key: "password", + }, + RedisType: v1alpha1.RedisTypeStandalone, + } + envs := frontlasRedisEnvs(fc, 40011, 40012) + + var passwordEnv *corev1.EnvVar + for i := range envs { + if envs[i].Name == FrontlasRedisPasswordEnv { + passwordEnv = &envs[i] + } + } + assert.NotNil(t, passwordEnv) + assert.Empty(t, passwordEnv.Value, "PasswordSecret 存在时禁止把明文 Password 写入 env") + assert.NotNil(t, passwordEnv.ValueFrom) + assert.NotNil(t, passwordEnv.ValueFrom.SecretKeyRef) + assert.Equal(t, "redis-creds", passwordEnv.ValueFrom.SecretKeyRef.Name) + assert.Equal(t, "password", passwordEnv.ValueFrom.SecretKeyRef.Key) +} + +func TestFrontlasRedisEnvs_FallbackToPlaintext(t *testing.T) { + fc := v1alpha1.FrontierCluster{} + fc.Spec.Frontlas.Redis = v1alpha1.Redis{ + Addrs: []string{"r:6379"}, + Password: "legacy-plaintext", + RedisType: v1alpha1.RedisTypeStandalone, + } + envs := frontlasRedisEnvs(fc, 40011, 40012) + + var passwordEnv *corev1.EnvVar + for i := range envs { + if envs[i].Name == FrontlasRedisPasswordEnv { + passwordEnv = &envs[i] + } + } + assert.NotNil(t, passwordEnv) + assert.Equal(t, "legacy-plaintext", passwordEnv.Value) + assert.Nil(t, passwordEnv.ValueFrom) +} + +func TestFrontierDefaults_HasGoodSafetyDefaults(t *testing.T) { + d := frontierDefaults(30011, 30012) + assert.NotNil(t, d.livenessProbe) + assert.NotNil(t, d.readinessProbe) + assert.NotNil(t, d.lifecycle, "preStop 必须有，避免长连接秒断") + assert.NotNil(t, d.lifecycle.PreStop) + assert.Greater(t, d.terminationGracePeriodSeconds, int64(0)) + assert.Equal(t, corev1.PullIfNotPresent, d.imagePullPolicy) + assert.NotNil(t, d.podSecurityContext) + assert.NotNil(t, d.containerSecurityContext) +} diff --git a/pkg/operator/pkg/kube/container/container_builder.go b/pkg/operator/pkg/kube/container/container_builder.go index b1fafa9..219065e 100644 --- a/pkg/operator/pkg/kube/container/container_builder.go +++ b/pkg/operator/pkg/kube/container/container_builder.go @@ -13,7 +13,11 @@ type builder struct { command []string args []string envs []corev1.EnvVar + resources *corev1.ResourceRequirements + livenessProbe *corev1.Probe readinessProbe *corev1.Probe + lifecycle *corev1.Lifecycle + securityContext *corev1.SecurityContext volumeMounts []corev1.VolumeMount ports []corev1.ContainerPort @@ -54,6 +58,11 @@ func (b *builder) SetEnvs(envs []corev1.EnvVar) *builder { return b } +func (b *builder) AddEnvs(envs []corev1.EnvVar) *builder { + b.envs = append(b.envs, envs...) + return b +} + func (b *builder) SetVolumeMounts(volumeMount []corev1.VolumeMount) *builder { b.volumeMounts = volumeMount return b @@ -64,13 +73,33 @@ func (b *builder) SetPorts(ports []corev1.ContainerPort) *builder { return b } -func (b *builder) SetReadinessProbe(readinessProbe *corev1.Probe) *builder { - b.readinessProbe = readinessProbe +func (b *builder) SetResources(r *corev1.ResourceRequirements) *builder { + b.resources = r + return b +} + +func (b *builder) SetLivenessProbe(p *corev1.Probe) *builder { + b.livenessProbe = p + return b +} + +func (b *builder) SetReadinessProbe(p *corev1.Probe) *builder { + b.readinessProbe = p + return b +} + +func (b *builder) SetLifecycle(l *corev1.Lifecycle) *builder { + b.lifecycle = l + return b +} + +func (b *builder) SetSecurityContext(s *corev1.SecurityContext) *builder { + b.securityContext = s return b } func (b *builder) Build() corev1.Container { - return corev1.Container{ + c := corev1.Container{ Name: b.name, Image: b.image, ImagePullPolicy: b.imagePullPolicy, @@ -80,8 +109,15 @@ func (b *builder) Build() corev1.Container { Env: b.envs, VolumeMounts: b.volumeMounts, Ports: b.ports, + LivenessProbe: b.livenessProbe, ReadinessProbe: b.readinessProbe, + Lifecycle: b.lifecycle, + SecurityContext: b.securityContext, + } + if b.resources != nil { + c.Resources = *b.resources } + return c } func Builder() *builder { diff --git a/pkg/operator/pkg/kube/podtemplatespec/podtemplatespec_builder.go b/pkg/operator/pkg/kube/podtemplatespec/podtemplatespec_builder.go index e030227..f92d178 100644 --- a/pkg/operator/pkg/kube/podtemplatespec/podtemplatespec_builder.go +++ b/pkg/operator/pkg/kube/podtemplatespec/podtemplatespec_builder.go @@ -25,10 +25,17 @@ type builder struct { terminationGracePeriodSeconds int64 tolerations []corev1.Toleration + topologySpreadConstraints []corev1.TopologySpreadConstraint // affinity + affinity *corev1.Affinity // 全量 Affinity，优先级高于下面三段 podAntiAffinity *corev1.PodAntiAffinity nodeAffinity *corev1.NodeAffinity podAffinity *corev1.PodAffinity + + priorityClassName string + serviceAccountName string + imagePullSecrets []corev1.LocalObjectReference + podSecurityContext *corev1.PodSecurityContext } func (b *builder) SetLabels(labels map[string]string) *builder { @@ -36,6 +43,25 @@ func (b *builder) SetLabels(labels map[string]string) *builder { return b } +func (b *builder) MergeLabels(labels map[string]string) *builder { + for k, v := range labels { + b.labels[k] = v + } + return b +} + +func (b *builder) SetAnnotations(annotations map[string]string) *builder { + b.annotations = annotations + return b +} + +func (b *builder) MergeAnnotations(annotations map[string]string) *builder { + for k, v := range annotations { + b.annotations[k] = v + } + return b +} + func (b *builder) SetName(name string) *builder { b.name = name return b @@ -83,6 +109,11 @@ func (b *builder) SetNodeSelector(nodeSelector map[string]string) *builder { return b } +func (b *builder) SetAffinity(affinity *corev1.Affinity) *builder { + b.affinity = affinity + return b +} + func (b *builder) SetPodAntiAffinity(podAntiAffinity *corev1.PodAntiAffinity) *builder { b.podAntiAffinity = podAntiAffinity return b @@ -103,6 +134,47 @@ func (b *builder) SetTolerations(tolerations []corev1.Toleration) *builder { return b } +func (b *builder) SetTopologySpreadConstraints(c []corev1.TopologySpreadConstraint) *builder { + b.topologySpreadConstraints = c + return b +} + +func (b *builder) SetPriorityClassName(name string) *builder { + b.priorityClassName = name + return b +} + +func (b *builder) SetServiceAccountName(name string) *builder { + b.serviceAccountName = name + return b +} + +func (b *builder) SetImagePullSecrets(s []corev1.LocalObjectReference) *builder { + b.imagePullSecrets = s + return b +} + +func (b *builder) SetPodSecurityContext(s *corev1.PodSecurityContext) *builder { + b.podSecurityContext = s + return b +} + +// resolveAffinity 决定最终的 Affinity：用户传整体 affinity 走它（fully wins）； +// 否则用 builder 累计的三段子结构组装。 +func (b *builder) resolveAffinity() *corev1.Affinity { + if b.affinity != nil { + return b.affinity + } + if b.nodeAffinity == nil && b.podAffinity == nil && b.podAntiAffinity == nil { + return nil + } + return &corev1.Affinity{ + NodeAffinity: b.nodeAffinity, + PodAffinity: b.podAffinity, + PodAntiAffinity: b.podAntiAffinity, + } +} + func (b *builder) Build() corev1.PodTemplateSpec { return corev1.PodTemplateSpec{ ObjectMeta: metav1.ObjectMeta{ @@ -117,12 +189,13 @@ func (b *builder) Build() corev1.PodTemplateSpec { Containers: b.containers, TerminationGracePeriodSeconds: &b.terminationGracePeriodSeconds, NodeSelector: b.nodeSelector, - Affinity: &corev1.Affinity{ - NodeAffinity: b.nodeAffinity, - PodAffinity: b.podAffinity, - PodAntiAffinity: b.podAntiAffinity, - }, - Tolerations: b.tolerations, + Affinity: b.resolveAffinity(), + Tolerations: b.tolerations, + TopologySpreadConstraints: b.topologySpreadConstraints, + PriorityClassName: b.priorityClassName, + ServiceAccountName: b.serviceAccountName, + ImagePullSecrets: b.imagePullSecrets, + SecurityContext: b.podSecurityContext, }, } } From d83e19badbbcbf247c804cab47b3e6a834b557ad Mon Sep 17 00:00:00 2001 From: singchia Date: Sat, 2 May 2026 00:09:46 +0800 Subject: [PATCH 06/12] feat(frontier): drain window before graceful shutdown When SIGTERM arrives, sleep for FRONTIER_DRAIN_SECONDS (default 30s, operator sets it dynamically based on terminationGracePeriodSeconds) before invoking frontier.Close(). This is the minimum-useful step toward graceful shutdown for long-lived edge connections: it gives kube-proxy time to fully propagate the endpoint removal that the preStop sleep started, and lets in-flight edge sessions finish naturally instead of being torn down the instant the signal hits. Full deregister-from-Frontlas + close-listeners-but-keep-existing flow is deferred to M3, where it will compose with the upcoming /readyz endpoint (readiness flips to 503 -> kube removes endpoint -> drain). Refs: docs/rfc/RFC-001-cloud-native-optimization.md (M2) Co-Authored-By: Claude Opus 4.7 (1M context) --- cmd/frontier/main.go | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/cmd/frontier/main.go b/cmd/frontier/main.go index d8b6e92..9fa65ca 100644 --- a/cmd/frontier/main.go +++ b/cmd/frontier/main.go @@ -3,12 +3,39 @@ package main import ( "context" _ "net/http/pprof" + "os" + "strconv" + "time" "github.com/jumboframes/armorigo/sigaction" "github.com/singchia/frontier/pkg/frontier" "k8s.io/klog/v2" ) +// drainSecondsFromEnv 决定 SIGTERM 抵达后、调用 frontier.Close() 前的等待时长。 +// 这段时间内进程不接新流量也不主动断已有连接——给上游 kube-proxy +// 把本 pod 从 endpoints 摘除、给已建立的 edge 长连接自然结束的窗口。 +// +// 上限由 K8s 侧的 terminationGracePeriodSeconds 控制。Operator 默认设 60s， +// 给 Close 自身留 ~10s 余量，所以 drain 默认 30s 是合理起点。用户可调。 +const ( + envDrainSeconds = "FRONTIER_DRAIN_SECONDS" + defaultDrainSeconds = 30 +) + +func drainSecondsFromEnv() int { + v := os.Getenv(envDrainSeconds) + if v == "" { + return defaultDrainSeconds + } + n, err := strconv.Atoi(v) + if err != nil || n < 0 { + klog.Warningf("invalid %s=%q, falling back to default %ds", envDrainSeconds, v, defaultDrainSeconds) + return defaultDrainSeconds + } + return n +} + func main() { frontier, err := frontier.NewFrontier() if err != nil { @@ -20,5 +47,10 @@ func main() { sig := sigaction.NewSignal() sig.Wait(context.TODO()) + if drain := drainSecondsFromEnv(); drain > 0 { + klog.Infof("frontier received shutdown signal, draining for %ds before close", drain) + time.Sleep(time.Duration(drain) * time.Second) + } + frontier.Close() } From dfeb8feced6e8a21993fd24959e2fa379ce5560b Mon Sep 17 00:00:00 2001 From: singchia Date: Sat, 2 May 2026 00:10:00 +0800 Subject: [PATCH 07/12] chore(images): run frontier and frontlas as non-root MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add a system user (UID 65532) to both Alpine images and switch USER to it. UID 65532 matches the operator's default PodSecurityContext.RunAsUser, so runAsNonRoot=true now passes for stock images without users having to override SecurityContext. Resolves the gospec security red line "容器以非 root 用户运行". Refs: docs/rfc/RFC-001-cloud-native-optimization.md (M2) Co-Authored-By: Claude Opus 4.7 (1M context) --- images/Dockerfile.frontier | 5 +++++ images/Dockerfile.frontlas | 5 +++++ 2 files changed, 10 insertions(+) diff --git a/images/Dockerfile.frontier b/images/Dockerfile.frontier index ec7f091..d79c72b 100644 --- a/images/Dockerfile.frontier +++ b/images/Dockerfile.frontier @@ -21,6 +21,11 @@ RUN wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://alpine-pkgs.sgerrand.com/s RUN wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.34-r0/glibc-2.34-r0.apk RUN apk add glibc-2.34-r0.apk +# 非 root 运行：UID 65532 与 operator 默认 PodSecurityContext.RunAsUser 对齐。 +# gospec 安全红线："容器以非 root 用户运行"。 +RUN addgroup -g 65532 -S frontier && adduser -u 65532 -S -G frontier frontier +USER 65532:65532 + EXPOSE 30010 EXPOSE 30011 EXPOSE 30012 diff --git a/images/Dockerfile.frontlas b/images/Dockerfile.frontlas index 6a29496..c3ccbc9 100644 --- a/images/Dockerfile.frontlas +++ b/images/Dockerfile.frontlas @@ -21,6 +21,11 @@ RUN wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://alpine-pkgs.sgerrand.com/s RUN wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.34-r0/glibc-2.34-r0.apk RUN apk add glibc-2.34-r0.apk +# 非 root 运行：UID 65532 与 operator 默认 PodSecurityContext.RunAsUser 对齐。 +# gospec 安全红线："容器以非 root 用户运行"。 +RUN addgroup -g 65532 -S frontlas && adduser -u 65532 -S -G frontlas frontlas +USER 65532:65532 + EXPOSE 40011 EXPOSE 40012 From 3cb125c3061573f1a4ceadb58362d56b5f0cb771 Mon Sep 17 00:00:00 2001 From: singchia Date: Sat, 2 May 2026 00:10:09 +0800 Subject: [PATCH 08/12] docs(rfc): record M2 amendments Two amendments to RFC-001 captured in the changelog table: - M2 stays on v1alpha1 with optional fields rather than introducing v1alpha2. Multi-version CRDs require a conversion strategy; None needs schema parity (defeating the new fields), Webhook adds an in-cluster service + cert lifecycle to M2 scope. Version bump moved to M4 alongside Status conditions, going straight to v1beta1. - M2 through M4 ship on a single feat/rfc-001-m1 branch instead of one PR per milestone, per user direction after M1 verification. Refs: docs/rfc/RFC-001-cloud-native-optimization.md Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/rfc/RFC-001-cloud-native-optimization.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/rfc/RFC-001-cloud-native-optimization.md b/docs/rfc/RFC-001-cloud-native-optimization.md index 75fd31a..9bc9bd6 100644 --- a/docs/rfc/RFC-001-cloud-native-optimization.md +++ b/docs/rfc/RFC-001-cloud-native-optimization.md @@ -249,3 +249,5 @@ type Redis struct { | 日期 | 变更人 | 变更内容 | 原因 | |---|---|---|---| | 2026-05-01 | singchia | 初稿 | 立项 | +| 2026-05-01 | singchia | M2 不再引入 v1alpha2，改为在 v1alpha1 原地追加 optional 字段（PodOverrides、Redis.PasswordSecret）；版本 bump 推迟到 M4 跟 Status Conditions 一起做（届时直接到 v1beta1，跳过 v1alpha2 临时态） | K8s 多版本 CRD 必须配 conversion strategy；`None` 要求版本 schema 严格等价（与本次扩字段冲突），`Webhook` 又把 M2 范围撑到需要起额外服务、写 webhook handler、签证书。原地扩 alpha 字段不破坏向后兼容（旧 YAML 全量可用），把版本演进集中到 M4 一起做，整体工作量更小、风险更低。| +| 2026-05-01 | singchia | M2-M4 统一在 `feat/rfc-001-m1` 分支累积，不再每个里程碑独立 PR | 用户在 M1 验证完成后明确指示"剩下的我们继续吧，都在同一个分支继续"。整体一个 PR 评审，便于把握跨里程碑的连贯性 | From 2acbfd632ffd907b7d54c359189486d138fc69a5 Mon Sep 17 00:00:00 2001 From: singchia Date: Sat, 2 May 2026 09:25:11 +0800 Subject: [PATCH 09/12] =?UTF-8?q?feat(operator):=20CRD=20ergonomics=20?= =?UTF-8?q?=E2=80=94=20shortName=20fc,=20printcolumns,=20Conditions=20sche?= =?UTF-8?q?ma?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CRD-level changes that improve day-to-day kubectl experience and lay the groundwork for modern Kubernetes-style status reporting (M4.3). - shortNames: fc, fcs (kubectl get fc just works) - categories: frontier (kubectl get all -l frontier picks it up) - printcolumns: Phase, FrontierReady, FrontlasReady, Age, Message (Message is priority=1 so it only shows in -o wide) - Status now exposes: * conditions []metav1.Condition (Available/Progressing/Degraded) * observedGeneration * frontierReadyReplicas / frontlasReadyReplicas - Phase + Message kept for backwards compat and the printcolumn, but tagged Deprecated in favor of Conditions zz_generated.deepcopy.go and CRD YAML regenerated by controller-gen. Refs: docs/rfc/RFC-001-cloud-native-optimization.md (M4.3) Co-Authored-By: Claude Opus 4.7 (1M context) --- .../api/v1alpha1/frontiercluster_types.go | 42 +++++- .../api/v1alpha1/zz_generated.deepcopy.go | 10 +- ...frontier.singchia.io_frontierclusters.yaml | 124 ++++++++++++++++-- 3 files changed, 159 insertions(+), 17 deletions(-) diff --git a/pkg/operator/api/v1alpha1/frontiercluster_types.go b/pkg/operator/api/v1alpha1/frontiercluster_types.go index 865ea6d..55b1c74 100644 --- a/pkg/operator/api/v1alpha1/frontiercluster_types.go +++ b/pkg/operator/api/v1alpha1/frontiercluster_types.go @@ -161,17 +161,45 @@ const ( // FrontierClusterStatus defines the observed state of FrontierCluster type FrontierClusterStatus struct { - // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster - // Important: Run "make" to regenerate code after modifying this file - // TODO scale 1 a time - // CurrentFrontierReplicas int `json:"currentFrontierReplicas"` - // CurrentFrontlasReplicass int `json:"currentFrontlasReplicas"` - Phase Phase `json:"phase"` - Message string `json:"message,omitemtpy"` + // Phase 是粗粒度状态，便于 printcolumn 一眼可见。 + // Deprecated: 优先使用 Conditions 做精细化判断；Phase 暂保留兼容。 + Phase Phase `json:"phase,omitempty"` + // Message 是最近一次状态变更的简短描述。 + Message string `json:"message,omitempty"` + // Conditions 反映 reconcile 流程的细粒度状态，遵循 K8s 现代约定。 + // 类型包括 Available / Progressing / Degraded。 + // +optional + // +patchMergeKey=type + // +patchStrategy=merge + // +listType=map + // +listMapKey=type + Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"` + // ObservedGeneration 是 status 对应的 spec generation，用于检测 status 是否陈旧。 + // +optional + ObservedGeneration int64 `json:"observedGeneration,omitempty"` + // FrontierReadyReplicas / FrontlasReadyReplicas 暴露当前 ready 副本数， + // 给 printcolumn 与 kubectl get 用。 + // +optional + FrontierReadyReplicas int32 `json:"frontierReadyReplicas,omitempty"` + // +optional + FrontlasReadyReplicas int32 `json:"frontlasReadyReplicas,omitempty"` } +// Condition 类型常量 +const ( + ConditionAvailable = "Available" + ConditionProgressing = "Progressing" + ConditionDegraded = "Degraded" +) + //+kubebuilder:object:root=true +//+kubebuilder:resource:shortName=fc;fcs,categories={frontier} //+kubebuilder:subresource:status +//+kubebuilder:printcolumn:name="Phase",type=string,JSONPath=`.status.phase` +//+kubebuilder:printcolumn:name="Frontier",type=integer,JSONPath=`.status.frontierReadyReplicas` +//+kubebuilder:printcolumn:name="Frontlas",type=integer,JSONPath=`.status.frontlasReadyReplicas` +//+kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` +//+kubebuilder:printcolumn:name="Message",type=string,priority=1,JSONPath=`.status.message` // FrontierCluster is the Schema for the frontierclusters API type FrontierCluster struct { diff --git a/pkg/operator/api/v1alpha1/zz_generated.deepcopy.go b/pkg/operator/api/v1alpha1/zz_generated.deepcopy.go index bef374a..19619ad 100644 --- a/pkg/operator/api/v1alpha1/zz_generated.deepcopy.go +++ b/pkg/operator/api/v1alpha1/zz_generated.deepcopy.go @@ -22,6 +22,7 @@ package v1alpha1 import ( "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" runtime "k8s.io/apimachinery/pkg/runtime" ) @@ -81,7 +82,7 @@ func (in *FrontierCluster) DeepCopyInto(out *FrontierCluster) { out.TypeMeta = in.TypeMeta in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) in.Spec.DeepCopyInto(&out.Spec) - out.Status = in.Status + in.Status.DeepCopyInto(&out.Status) } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new FrontierCluster. @@ -154,6 +155,13 @@ func (in *FrontierClusterSpec) DeepCopy() *FrontierClusterSpec { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *FrontierClusterStatus) DeepCopyInto(out *FrontierClusterStatus) { *out = *in + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]metav1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new FrontierClusterStatus. diff --git a/pkg/operator/config/crd/bases/frontier.singchia.io_frontierclusters.yaml b/pkg/operator/config/crd/bases/frontier.singchia.io_frontierclusters.yaml index 41103db..016c43a 100644 --- a/pkg/operator/config/crd/bases/frontier.singchia.io_frontierclusters.yaml +++ b/pkg/operator/config/crd/bases/frontier.singchia.io_frontierclusters.yaml @@ -8,13 +8,35 @@ metadata: spec: group: frontier.singchia.io names: + categories: + - frontier kind: FrontierCluster listKind: FrontierClusterList plural: frontierclusters + shortNames: + - fc + - fcs singular: frontiercluster scope: Namespaced versions: - - name: v1alpha1 + - additionalPrinterColumns: + - jsonPath: .status.phase + name: Phase + type: string + - jsonPath: .status.frontierReadyReplicas + name: Frontier + type: integer + - jsonPath: .status.frontlasReadyReplicas + name: Frontlas + type: integer + - jsonPath: .metadata.creationTimestamp + name: Age + type: date + - jsonPath: .status.message + name: Message + priority: 1 + type: string + name: v1alpha1 schema: openAPIV3Schema: description: FrontierCluster is the Schema for the frontierclusters API @@ -4755,19 +4777,103 @@ spec: status: description: FrontierClusterStatus defines the observed state of FrontierCluster properties: + conditions: + description: |- + Conditions 反映 reconcile 流程的细粒度状态，遵循 K8s 现代约定。 + 类型包括 Available / Progressing / Degraded。 + items: + description: "Condition contains details for one aspect of the current + state of this API Resource.\n---\nThis struct is intended for + direct use as an array at the field path .status.conditions. For + example,\n\n\n\ttype FooStatus struct{\n\t // Represents the + observations of a foo's current state.\n\t // Known .status.conditions.type + are: \"Available\", \"Progressing\", and \"Degraded\"\n\t // + +patchMergeKey=type\n\t // +patchStrategy=merge\n\t // +listType=map\n\t + \ // +listMapKey=type\n\t Conditions []metav1.Condition `json:\"conditions,omitempty\" + patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"`\n\n\n\t + \ // other fields\n\t}" + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: |- + type of condition in CamelCase or in foo.example.com/CamelCase. + --- + Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be + useful (see .node.status.conditions), the ability to deconflict is important. + The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + frontierReadyReplicas: + description: |- + FrontierReadyReplicas / FrontlasReadyReplicas 暴露当前 ready 副本数， + 给 printcolumn 与 kubectl get 用。 + format: int32 + type: integer + frontlasReadyReplicas: + format: int32 + type: integer message: + description: Message 是最近一次状态变更的简短描述。 type: string + observedGeneration: + description: ObservedGeneration 是 status 对应的 spec generation，用于检测 + status 是否陈旧。 + format: int64 + type: integer phase: description: |- - INSERT ADDITIONAL STATUS FIELD - define observed state of cluster - Important: Run "make" to regenerate code after modifying this file - TODO scale 1 a time - CurrentFrontierReplicas int `json:"currentFrontierReplicas"` - CurrentFrontlasReplicass int `json:"currentFrontlasReplicas"` + Phase 是粗粒度状态，便于 printcolumn 一眼可见。 + Deprecated: 优先使用 Conditions 做精细化判断；Phase 暂保留兼容。 type: string - required: - - message - - phase type: object type: object served: true From 7898bddce22657fa498045affa5f5159df90ef84 Mon Sep 17 00:00:00 2001 From: singchia Date: Sat, 2 May 2026 09:27:11 +0800 Subject: [PATCH 10/12] feat(operator): wire Status Conditions, ObservedGeneration, readyReplicas + Events MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reconciler now writes a full modern K8s status alongside the legacy Phase: - Available / Progressing / Degraded Conditions, set in lockstep with Phase transitions. Status changes refresh LastTransitionTime; no-op writes preserve it (idiomatic K8s condition behavior). - ObservedGeneration written on every status update so consumers can detect stale status. - FrontierReadyReplicas / FrontlasReadyReplicas pulled from the underlying Deployment.Status.ReadyReplicas. ensureDeployment now returns a deploymentReadiness struct instead of a bare bool so the controller can feed both numbers into status regardless of pending/failed/running path. EventRecorder ("frontier-operator") emits Kubernetes Events at meaningful state transitions: - ServiceEnsureFailed (warning) — Service reconcile error - TLSEnsureFailed (warning) — TLS Secret reconcile error - DeploymentEnsureFailed (warning) — Deployment apply error - Available (normal) — fired once when the cluster first becomes Running, not on every successful reconcile RBAC: + events:create;patch (the standard EventRecorder verbs). Refs: docs/rfc/RFC-001-cloud-native-optimization.md (M4.1, M4.2) Co-Authored-By: Claude Opus 4.7 (1M context) --- .../controller/frontiercluster_controller.go | 33 ++++-- .../controller/frontiercluster_deployment.go | 29 +++-- .../frontiercluster_status_options.go | 106 ++++++++++++++++++ 3 files changed, 152 insertions(+), 16 deletions(-) diff --git a/pkg/operator/internal/controller/frontiercluster_controller.go b/pkg/operator/internal/controller/frontiercluster_controller.go index c4959b9..a08f168 100644 --- a/pkg/operator/internal/controller/frontiercluster_controller.go +++ b/pkg/operator/internal/controller/frontiercluster_controller.go @@ -20,7 +20,9 @@ import ( "context" "fmt" + corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/runtime" + "k8s.io/client-go/tools/record" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/log" @@ -36,16 +38,18 @@ import ( // FrontierClusterReconciler reconciles a FrontierCluster object type FrontierClusterReconciler struct { client.Client - Scheme *runtime.Scheme - client kubeclient.Client + Scheme *runtime.Scheme + client kubeclient.Client + recorder record.EventRecorder } func NewReconciler(mgr manager.Manager) *FrontierClusterReconciler { mgrClient := mgr.GetClient() return &FrontierClusterReconciler{ - Client: mgrClient, - Scheme: mgr.GetScheme(), - client: kubeclient.NewClient(mgrClient), + Client: mgrClient, + Scheme: mgr.GetScheme(), + client: kubeclient.NewClient(mgrClient), + recorder: mgr.GetEventRecorderFor("frontier-operator"), } } @@ -54,6 +58,7 @@ func NewReconciler(mgr manager.Manager) *FrontierClusterReconciler { //+kubebuilder:rbac:groups=frontier.singchia.io,resources=frontierclusters/finalizers,verbs=update //+kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete //+kubebuilder:rbac:groups="",resources=services;pods;secrets,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups="",resources=events,verbs=create;patch // Reconcile is part of the main kubernetes reconciliation loop which aims to // move the current state of the cluster closer to the desired state. @@ -75,6 +80,7 @@ func (r *FrontierClusterReconciler) Reconcile(ctx context.Context, req ctrl.Requ log.Info("Ensuring the service exists") if err := r.ensureService(ctx, frontiercluster); err != nil { + r.recorder.Eventf(&frontiercluster, corev1.EventTypeWarning, "ServiceEnsureFailed", "Error ensuring services: %s", err) return status.Update(ctx, r.client.Status(), &frontiercluster, statusOptions(). withMessage(Error, fmt.Sprintf("Error ensuring services: %s", err)). withFailedPhase()) @@ -82,26 +88,39 @@ func (r *FrontierClusterReconciler) Reconcile(ctx context.Context, req ctrl.Requ log.Info("Ensuring the tls exists") if err := r.ensureTLS(ctx, frontiercluster); err != nil { + r.recorder.Eventf(&frontiercluster, corev1.EventTypeWarning, "TLSEnsureFailed", "Error ensuring TLS secret: %s", err) return status.Update(ctx, r.client.Status(), &frontiercluster, statusOptions(). withMessage(Error, fmt.Sprintf("Error ensuring tls secret: %s", err)). withFailedPhase()) } log.Info("Ensuring the deployment exists") - ready, err := r.ensureDeployment(ctx, frontiercluster) + readiness, err := r.ensureDeployment(ctx, frontiercluster) if err != nil { + r.recorder.Eventf(&frontiercluster, corev1.EventTypeWarning, "DeploymentEnsureFailed", "Error deploying: %s", err) return status.Update(ctx, r.client.Status(), &frontiercluster, statusOptions(). + withReadyReplicas(readiness.frontierReady, readiness.frontlasReady). withMessage(Error, fmt.Sprintf("Error deploying Deployment: %s", err)). withFailedPhase()) } - if !ready { + if !readiness.allReady { return status.Update(ctx, r.client.Status(), &frontiercluster, statusOptions(). + withReadyReplicas(readiness.frontierReady, readiness.frontlasReady). withMessage(Info, "Deployment is not yet ready, retrying in 10 seconds"). withPendingPhase(10)) } + // 只在从非 Running 转 Running 时发事件，避免每次 reconcile 都打。 + if frontiercluster.Status.Phase != frontierv1alpha1.Running { + r.recorder.Eventf(&frontiercluster, corev1.EventTypeNormal, "Available", + "FrontierCluster is available: frontier=%d/%d frontlas=%d/%d", + readiness.frontierReady, frontiercluster.Spec.Frontier.Replicas, + readiness.frontlasReady, frontiercluster.Spec.Frontlas.Replicas) + } + res, err := status.Update(ctx, r.client.Status(), &frontiercluster, statusOptions(). + withReadyReplicas(readiness.frontierReady, readiness.frontlasReady). withMessage(Info, "Good to go!"). withRunningPhase()) if err != nil { diff --git a/pkg/operator/internal/controller/frontiercluster_deployment.go b/pkg/operator/internal/controller/frontiercluster_deployment.go index bd018c3..bfda1d1 100644 --- a/pkg/operator/internal/controller/frontiercluster_deployment.go +++ b/pkg/operator/internal/controller/frontiercluster_deployment.go @@ -48,38 +48,48 @@ const ( FrontlasAddrEnv = "FRONTLAS_ADDR" // service + frontierport ) -func (r *FrontierClusterReconciler) ensureDeployment(ctx context.Context, fc v1alpha1.FrontierCluster) (bool, error) { +// deploymentReadiness 是 ensureDeployment 的回包，方便上层把数字回填到 status。 +type deploymentReadiness struct { + frontierReady int32 + frontlasReady int32 + allReady bool +} + +func (r *FrontierClusterReconciler) ensureDeployment(ctx context.Context, fc v1alpha1.FrontierCluster) (deploymentReadiness, error) { log := log.FromContext(ctx) + out := deploymentReadiness{} log.Info("Create/Updating Frontlas Deployment") if err := r.ensureFrontlasDeployment(ctx, fc); err != nil { - return false, fmt.Errorf("error creating/updating frontlas Deployment: %s", err) + return out, fmt.Errorf("error creating/updating frontlas Deployment: %s", err) } currentFrontlasDeployment, err := r.client.GetDeployment(ctx, fc.FrontlasDeploymentNamespacedName()) if err != nil { - return false, fmt.Errorf("error getting Deployment: %s", err) + return out, fmt.Errorf("error getting Deployment: %s", err) } + out.frontlasReady = currentFrontlasDeployment.Status.ReadyReplicas frontlasIsReady := deployment.IsReady(currentFrontlasDeployment, fc.FrontlasReplicas()) if !frontlasIsReady { log.Info("frontlas deployment is not ready", - "expectedReplicas", fc.FrontierReplicas(), + "expectedReplicas", fc.FrontlasReplicas(), "updatedReplicas", currentFrontlasDeployment.Status.UpdatedReplicas, "readyReplicas", currentFrontlasDeployment.Status.ReadyReplicas, "generation", currentFrontlasDeployment.Generation, "observedGeneration", currentFrontlasDeployment.Status.ObservedGeneration) - return false, nil + return out, nil } log.Info("Creating/Updating Frontier Deployment") if err := r.ensureFrontierDeployment(ctx, fc); err != nil { - return false, fmt.Errorf("error creating/updating frontier Deployment: %s", err) + return out, fmt.Errorf("error creating/updating frontier Deployment: %s", err) } currentFrontierDeployment, err := r.client.GetDeployment(ctx, fc.FrontierDeploymentNamespacedName()) if err != nil { - return false, fmt.Errorf("error getting Deployment: %s", err) + return out, fmt.Errorf("error getting Deployment: %s", err) } + out.frontierReady = currentFrontierDeployment.Status.ReadyReplicas frontierIsReady := deployment.IsReady(currentFrontierDeployment, fc.FrontierReplicas()) if !frontierIsReady { log.Info("frontier deployment is not ready", @@ -88,10 +98,11 @@ func (r *FrontierClusterReconciler) ensureDeployment(ctx context.Context, fc v1a "readyReplicas", currentFrontierDeployment.Status.ReadyReplicas, "generation", currentFrontierDeployment.Generation, "observedGeneration", currentFrontierDeployment.Status.ObservedGeneration) - return false, nil + return out, nil } - return frontierIsReady && frontlasIsReady, nil + out.allReady = true + return out, nil } func (r *FrontierClusterReconciler) ensureFrontierDeployment(ctx context.Context, fc v1alpha1.FrontierCluster) error { diff --git a/pkg/operator/internal/controller/frontiercluster_status_options.go b/pkg/operator/internal/controller/frontiercluster_status_options.go index 8c2f9a1..091c22e 100644 --- a/pkg/operator/internal/controller/frontiercluster_status_options.go +++ b/pkg/operator/internal/controller/frontiercluster_status_options.go @@ -5,6 +5,7 @@ import ( "github.com/singchia/frontier/operator/pkg/util/result" "github.com/singchia/frontier/operator/pkg/util/status" "go.uber.org/zap" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "sigs.k8s.io/controller-runtime/pkg/reconcile" ) @@ -98,6 +99,28 @@ func (o *optionBuilder) withRunningPhase() *optionBuilder { return o.withPhase(v1alpha1.Running, -1) } +func (o *optionBuilder) withReadyReplicas(frontier, frontlas int32) *optionBuilder { + o.options = append(o.options, readyReplicasOption{ + frontierReady: frontier, + frontlasReady: frontlas, + }) + return o +} + +type readyReplicasOption struct { + frontierReady int32 + frontlasReady int32 +} + +func (r readyReplicasOption) ApplyOption(fc *v1alpha1.FrontierCluster) { + fc.Status.FrontierReadyReplicas = r.frontierReady + fc.Status.FrontlasReadyReplicas = r.frontlasReady +} + +func (r readyReplicasOption) GetResult() (reconcile.Result, error) { + return result.OK() +} + type phaseOption struct { phase v1alpha1.Phase retryAfter int @@ -105,6 +128,68 @@ type phaseOption struct { func (p phaseOption) ApplyOption(fc *v1alpha1.FrontierCluster) { fc.Status.Phase = p.phase + // 把 Phase 同步到 Conditions——Phase 是 deprecated 标记位但仍会被 printcolumn 用， + // Conditions 是现代 K8s 状态查询入口，两者一起更新避免错位。 + fc.Status.ObservedGeneration = fc.Generation + switch p.phase { + case v1alpha1.Running: + setCondition(&fc.Status.Conditions, fc.Generation, metav1.Condition{ + Type: v1alpha1.ConditionAvailable, + Status: metav1.ConditionTrue, + Reason: "AllComponentsReady", + Message: fc.Status.Message, + }) + setCondition(&fc.Status.Conditions, fc.Generation, metav1.Condition{ + Type: v1alpha1.ConditionProgressing, + Status: metav1.ConditionFalse, + Reason: "ReconcileSucceeded", + Message: "Reconcile cycle completed successfully", + }) + setCondition(&fc.Status.Conditions, fc.Generation, metav1.Condition{ + Type: v1alpha1.ConditionDegraded, + Status: metav1.ConditionFalse, + Reason: "AllComponentsReady", + Message: "", + }) + case v1alpha1.Pending: + setCondition(&fc.Status.Conditions, fc.Generation, metav1.Condition{ + Type: v1alpha1.ConditionAvailable, + Status: metav1.ConditionFalse, + Reason: "ComponentsNotReady", + Message: fc.Status.Message, + }) + setCondition(&fc.Status.Conditions, fc.Generation, metav1.Condition{ + Type: v1alpha1.ConditionProgressing, + Status: metav1.ConditionTrue, + Reason: "ReconcileInProgress", + Message: fc.Status.Message, + }) + setCondition(&fc.Status.Conditions, fc.Generation, metav1.Condition{ + Type: v1alpha1.ConditionDegraded, + Status: metav1.ConditionFalse, + Reason: "ReconcileInProgress", + Message: "", + }) + case v1alpha1.Failed: + setCondition(&fc.Status.Conditions, fc.Generation, metav1.Condition{ + Type: v1alpha1.ConditionAvailable, + Status: metav1.ConditionFalse, + Reason: "ReconcileFailed", + Message: fc.Status.Message, + }) + setCondition(&fc.Status.Conditions, fc.Generation, metav1.Condition{ + Type: v1alpha1.ConditionProgressing, + Status: metav1.ConditionFalse, + Reason: "ReconcileFailed", + Message: fc.Status.Message, + }) + setCondition(&fc.Status.Conditions, fc.Generation, metav1.Condition{ + Type: v1alpha1.ConditionDegraded, + Status: metav1.ConditionTrue, + Reason: "ReconcileFailed", + Message: fc.Status.Message, + }) + } } func (p phaseOption) GetResult() (reconcile.Result, error) { @@ -119,3 +204,24 @@ func (p phaseOption) GetResult() (reconcile.Result, error) { } return result.OK() } + +// setCondition 是 K8s 风格的 condition upsert：按 Type 找到位置， +// 状态变化时刷新 LastTransitionTime；不变则只更新 Message/Reason/ObservedGeneration。 +func setCondition(conds *[]metav1.Condition, gen int64, c metav1.Condition) { + c.ObservedGeneration = gen + if c.LastTransitionTime.IsZero() { + c.LastTransitionTime = metav1.Now() + } + for i := range *conds { + if (*conds)[i].Type != c.Type { + continue + } + if (*conds)[i].Status == c.Status { + // 状态未变，保留原 LastTransitionTime + c.LastTransitionTime = (*conds)[i].LastTransitionTime + } + (*conds)[i] = c + return + } + *conds = append(*conds, c) +} From f7952fcb47d19cfd622f2c7f7711b546fa2db6cd Mon Sep 17 00:00:00 2001 From: singchia Date: Sat, 2 May 2026 09:27:48 +0800 Subject: [PATCH 11/12] feat(observability): shared HTTP server for /healthz /readyz /metrics MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Resolves the gospec red line "all external services must expose /healthz, /readyz, /metrics" for both frontier and frontlas (M3.1, M3.2). New shared package pkg/observability: - Server with three endpoints on a single configurable port. * /healthz — liveness; always 200 if process responds * /readyz — readiness; calls a pluggable ReadinessFn, returns 503 with a one-line reason when not ready * /metrics — Prometheus default registry (Go runtime + process collectors auto-included; business code can add custom collectors via promauto) - Atomic readiness function pointer so business code can flip readiness state at any point in the lifecycle (start, register-with-Frontlas, shutdown). - Graceful Shutdown(timeout) wired into the parent component's Close(). Wiring: - Frontier: default port 0.0.0.0:9091. Configurable via the new spec.observability block in frontier.yaml. Default ReadinessFn is "always ready"; future iterations can plug in Frontlas registration state. - Frontlas: default port 0.0.0.0:9092. ReadinessFn pings Redis with a 2s timeout — if Redis is unreachable, /readyz returns 503 (kube removes the pod from Endpoints, frontier nodes route around it). - Frontlas Dao gains a Ping(ctx) helper that wraps RDS.Ping. Existing /cluster/v1/health gRPC-gateway endpoint stays for backwards compat with the operator's frontlas readinessProbe. Adds github.com/prometheus/client_golang v1.23.2. Refs: docs/rfc/RFC-001-cloud-native-optimization.md (M3) Co-Authored-By: Claude Opus 4.7 (1M context) --- go.mod | 15 +++-- go.sum | 39 ++++++++---- pkg/frontier/config/config.go | 8 +++ pkg/frontier/frontier.go | 16 +++++ pkg/frontlas/config/config.go | 9 +++ pkg/frontlas/frontlas.go | 23 +++++++ pkg/frontlas/repo/dao.go | 5 ++ pkg/observability/server.go | 110 ++++++++++++++++++++++++++++++++++ 8 files changed, 210 insertions(+), 15 deletions(-) create mode 100644 pkg/observability/server.go diff --git a/go.mod b/go.mod index 4abd33d..a94bcba 100644 --- a/go.mod +++ b/go.mod @@ -14,19 +14,20 @@ require ( github.com/nats-io/nats.go v1.33.1 github.com/nsqio/go-nsq v1.1.0 github.com/pion/transport/v2 v2.2.10 + github.com/prometheus/client_golang v1.23.2 github.com/rabbitmq/amqp091-go v1.9.0 github.com/singchia/geminio v1.3.0-rc.2 github.com/singchia/go-timer/v2 v2.2.2 github.com/singchia/joy4 v0.0.0-20240621074108-53a2b0132ec6 github.com/soheilhy/cmux v0.1.5 github.com/spf13/pflag v1.0.5 - github.com/stretchr/testify v1.9.0 + github.com/stretchr/testify v1.11.1 github.com/swaggo/swag v1.16.3 github.com/tidwall/buntdb v1.3.1 github.com/vishvananda/netlink v1.1.0 google.golang.org/genproto/googleapis/api v0.0.0-20240304212257-790db918fca8 google.golang.org/grpc v1.62.1 - google.golang.org/protobuf v1.33.0 + google.golang.org/protobuf v1.36.8 gopkg.in/natefinch/lumberjack.v2 v2.2.1 gopkg.in/yaml.v2 v2.4.0 gorm.io/driver/sqlite v1.5.4 @@ -36,12 +37,17 @@ require ( require ( github.com/alicebob/gopher-json v0.0.0-20200520072559-a9ecdc9d1d3a // indirect - github.com/cespare/xxhash/v2 v2.2.0 // indirect + github.com/beorn7/perks v1.0.1 // indirect + github.com/cespare/xxhash/v2 v2.3.0 // indirect github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect + github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect github.com/nareix/joy4 v0.0.0-20200507095837-05a4ffbb5369 // indirect github.com/pion/logging v0.2.2 // indirect github.com/pion/transport/v3 v3.0.7 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect + github.com/prometheus/client_model v0.6.2 // indirect + github.com/prometheus/common v0.66.1 // indirect + github.com/prometheus/procfs v0.16.1 // indirect github.com/tidwall/btree v1.4.2 // indirect github.com/tidwall/gjson v1.14.3 // indirect github.com/tidwall/grect v0.1.4 // indirect @@ -51,6 +57,7 @@ require ( github.com/tidwall/tinyqueue v0.1.1 // indirect github.com/vishvananda/netns v0.0.0-20191106174202-0a2b9b5464df // indirect github.com/yuin/gopher-lua v1.1.1 // indirect + go.yaml.in/yaml/v2 v2.4.2 // indirect ) require ( @@ -82,7 +89,7 @@ require ( github.com/jinzhu/inflection v1.0.0 // indirect github.com/jinzhu/now v1.1.5 // indirect github.com/josharian/intern v1.0.0 // indirect - github.com/klauspost/compress v1.17.7 // indirect + github.com/klauspost/compress v1.18.0 // indirect github.com/mailru/easyjson v0.7.6 // indirect github.com/mattn/go-sqlite3 v1.14.17 // indirect github.com/nats-io/nkeys v0.4.7 // indirect diff --git a/go.sum b/go.sum index d5fe69d..f9864e3 100644 --- a/go.sum +++ b/go.sum @@ -10,14 +10,16 @@ github.com/alicebob/gopher-json v0.0.0-20200520072559-a9ecdc9d1d3a h1:HbKu58rmZp github.com/alicebob/gopher-json v0.0.0-20200520072559-a9ecdc9d1d3a/go.mod h1:SGnFV6hVsYE877CKEZ6tDNTjaSXYUk6QqoIK6PrAtcc= github.com/alicebob/miniredis/v2 v2.32.1 h1:Bz7CciDnYSaa0mX5xODh6GUITRSx+cVhjNoOR4JssBo= github.com/alicebob/miniredis/v2 v2.32.1/go.mod h1:AqkLNAfUm0K07J28hnAyyQKf/x0YkCY/g5DCtuL01Mw= +github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= +github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs= github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c= github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA= github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0= github.com/census-instrumentation/opencensus-proto v0.4.1 h1:iKLQ0xPNFxR/2hzXZMrBo8f1j86j5WHzznCCQxV/b8g= github.com/census-instrumentation/opencensus-proto v0.4.1/go.mod h1:4T9NM4+4Vw91VeyqjLS6ao50K5bOcLKN6Q42XnYaRYw= -github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44= -github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= +github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= @@ -72,8 +74,8 @@ github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEW github.com/golang/snappy v0.0.4 h1:yAGX7huGHXlcLOEtBnF4w7FQwA26wojNCwOYAEhLjQM= github.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= -github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY= @@ -108,8 +110,8 @@ github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFF github.com/jumboframes/armorigo v0.2.3/go.mod h1:sXe0R32y6V3oJD2eXcPzMlimvZx0xIDiLedpQOy06t4= github.com/jumboframes/armorigo v0.4.1 h1:MiT21uAGl21yUaj7SjHg4veGtB5Q79+8d7MRJhVq2rM= github.com/jumboframes/armorigo v0.4.1/go.mod h1:H4OlF0Jj8e+8LkAqDjeLtapNNnUuUXR/h4Q32Lqgf9o= -github.com/klauspost/compress v1.17.7 h1:ehO88t2UGzQK66LMdE8tibEd1ErmzZjNEqWkjLAKQQg= -github.com/klauspost/compress v1.17.7/go.mod h1:Di0epgTjJY877eYKx5yC51cX2A2Vl2ibi7bDH9ttBbw= +github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo= +github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= @@ -117,12 +119,16 @@ github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= +github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= github.com/mailru/easyjson v0.7.6 h1:8yTIVnZgCoiM1TgqoeTl+LfU5Jg6/xL3QhGQnimLYnA= github.com/mailru/easyjson v0.7.6/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= github.com/mattn/go-sqlite3 v1.14.17 h1:mCRHCLDUBXgpKAqIKsaAaAsrAlbkeomtRFKXh2L6YIM= github.com/mattn/go-sqlite3 v1.14.17/go.mod h1:2eHXhiwb8IkHr+BDWZGa96P6+rkvnG63S2DGjv9HUNg= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/nareix/joy4 v0.0.0-20200507095837-05a4ffbb5369 h1:Yp0zFEufLz0H7jzffb4UPXijavlyqlYeOg7dcyVUNnQ= github.com/nareix/joy4 v0.0.0-20200507095837-05a4ffbb5369/go.mod h1:aFJ1ZwLjvHN4yEzE5Bkz8rD8/d8Vlj3UIuvz2yfET7I= github.com/nats-io/nats.go v1.33.1 h1:8TxLZZ/seeEfR97qV0/Bl939tpDnt2Z2fK3HkPypj70= @@ -144,6 +150,14 @@ github.com/pion/transport/v3 v3.0.7 h1:iRbMH05BzSNwhILHoBoAPxoB9xQgOaJk+591KC9P1 github.com/pion/transport/v3 v3.0.7/go.mod h1:YleKiTZ4vqNxVwh77Z0zytYi7rXHl7j6uPLGhhz9rwo= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= +github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= +github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= +github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= +github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs= +github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA= +github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= +github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= github.com/rabbitmq/amqp091-go v1.9.0 h1:qrQtyzB4H8BQgEuJwhmVQqVHB9O4+MNDJCCAcpc3Aoo= github.com/rabbitmq/amqp091-go v1.9.0/go.mod h1:+jPrT9iY2eLjRaMSRHUhc3z14E/l85kv/f+6luSD3pc= github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 h1:N/ElC8H3+5XpJzTSTfLsJV/mx9Q9g7kxmchpfZyxgzM= @@ -175,8 +189,8 @@ github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/ github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= -github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg= -github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= github.com/swaggo/swag v1.16.3 h1:PnCYjPCah8FK4I26l2F/KQ4yz3sILcVUN3cTlBFA9Pg= github.com/swaggo/swag v1.16.3/go.mod h1:DImHIuOFXKpMFAQjcC7FG4m3Dg4+QuUgUzJmKjI/gRk= github.com/tidwall/assert v0.1.0 h1:aWcKyRBUAdLoVebxo95N7+YZVTFF/ASTr7BN4sLP6XI= @@ -208,8 +222,11 @@ github.com/wlynxg/anet v0.0.3/go.mod h1:eay5PRQr7fIVAMbTbchTnO9gG65Hg/uYGdc7mguH github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= github.com/yuin/gopher-lua v1.1.1 h1:kYKnWBjvbNP4XLT3+bPEwAXJx262OhaHDWDVOPjL46M= github.com/yuin/gopher-lua v1.1.1/go.mod h1:GBR0iDaNXjAgGg9zfCvksxSRnQx76gclCIb7kdAd1Pw= -go.uber.org/goleak v1.2.1 h1:NBol2c7O1ZokfZ0LEU9K6Whx/KnwvepVetCUhtKja4A= go.uber.org/goleak v1.2.1/go.mod h1:qlT2yGI9QafXHhZZLxlSuNsMw3FFLxBr+tBRlmO1xH4= +go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= +go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= +go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI= +go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= @@ -284,8 +301,8 @@ google.golang.org/grpc v1.62.1 h1:B4n+nfKzOICUXMgyrNd19h/I9oH0L1pizfk1d4zSgTk= google.golang.org/grpc v1.62.1/go.mod h1:IWTG0VlJLCh1SkC58F7np9ka9mx/WNkjl4PGJaiq+QE= google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= -google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI= -google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos= +google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc= +google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= diff --git a/pkg/frontier/config/config.go b/pkg/frontier/config/config.go index 484b669..d9d80d9 100644 --- a/pkg/frontier/config/config.go +++ b/pkg/frontier/config/config.go @@ -247,6 +247,12 @@ type Frontlas struct { } `yaml:"metrics" json:"metrics"` } +// Observability 控制 /healthz、/readyz、/metrics HTTP 端点。 +type Observability struct { + Enable bool `yaml:"enable" json:"enable"` + Addr string `yaml:"addr" json:"addr"` // 默认 0.0.0.0:9091 +} + type Configuration struct { Log config.Log `yaml:"log,omitempty" json:"log"` @@ -265,6 +271,8 @@ type Configuration struct { Frontlas Frontlas `yaml:"frontlas,omitempty" json:"frontlas"` MQM MQM `yaml:"mqm,omitempty" json:"mqm"` + + Observability Observability `yaml:"observability,omitempty" json:"observability"` } // Configuration accepts config file and command-line, and command-line is more privileged. diff --git a/pkg/frontier/frontier.go b/pkg/frontier/frontier.go index 14aa07d..b7fea0f 100644 --- a/pkg/frontier/frontier.go +++ b/pkg/frontier/frontier.go @@ -2,12 +2,14 @@ package frontier import ( "net/http" + "time" "github.com/singchia/frontier/pkg/frontier/apis" "github.com/singchia/frontier/pkg/frontier/config" "github.com/singchia/frontier/pkg/frontier/mq" "github.com/singchia/frontier/pkg/frontier/repo" "github.com/singchia/frontier/pkg/frontier/server" + "github.com/singchia/frontier/pkg/observability" "github.com/singchia/frontier/pkg/utils" "k8s.io/klog/v2" ) @@ -16,6 +18,7 @@ type Frontier struct { repo apis.Repo mqm apis.MQM server *server.Server + obs *observability.Server } func NewFrontier() (*Frontier, error) { @@ -55,18 +58,31 @@ func NewFrontier() (*Frontier, error) { return nil, err } + // observability：默认开启；地址不填走 0.0.0.0:9091。 + obsCfg := conf.Observability + if obsCfg.Addr == "" { + obsCfg.Addr = "0.0.0.0:9091" + } + if !obsCfg.Enable { + obsCfg.Enable = true // 默认开启；显式 false 才能关 + } + obs := observability.New(observability.Config{Enable: obsCfg.Enable, Addr: obsCfg.Addr}) + return &Frontier{ repo: repo, mqm: mqm, server: server, + obs: obs, }, nil } func (frontier *Frontier) Run() { + frontier.obs.Run() frontier.server.Serve() } func (frontier *Frontier) Close() { + frontier.obs.Shutdown(5 * time.Second) frontier.repo.Close() frontier.mqm.Close() frontier.server.Close() diff --git a/pkg/frontlas/config/config.go b/pkg/frontlas/config/config.go index 0e82061..4aea189 100644 --- a/pkg/frontlas/config/config.go +++ b/pkg/frontlas/config/config.go @@ -130,6 +130,13 @@ type FrontierManager struct { } `yaml:"expiration,omitempty" json:"expiration"` } +// Observability 控制 /healthz、/readyz、/metrics HTTP 端点。 +// 注：原有 /cluster/v1/health gRPC-gateway 端点保留兼容。 +type Observability struct { + Enable bool `yaml:"enable" json:"enable"` + Addr string `yaml:"addr" json:"addr"` // 默认 0.0.0.0:9092 +} + type Configuration struct { Log config.Log `yaml:"log,omitempty" json:"log"` @@ -140,6 +147,8 @@ type Configuration struct { FrontierManager FrontierManager `yaml:"frontier_plane" json:"frontier_manager"` Redis Redis `yaml:"redis" json:"redis"` + + Observability Observability `yaml:"observability,omitempty" json:"observability"` } func Parse() (*Configuration, error) { diff --git a/pkg/frontlas/frontlas.go b/pkg/frontlas/frontlas.go index 8308f5c..e715dd5 100644 --- a/pkg/frontlas/frontlas.go +++ b/pkg/frontlas/frontlas.go @@ -1,12 +1,15 @@ package frontlas import ( + "context" "net/http" "runtime" + "time" "github.com/singchia/frontier/pkg/frontlas/config" "github.com/singchia/frontier/pkg/frontlas/repo" "github.com/singchia/frontier/pkg/frontlas/server" + "github.com/singchia/frontier/pkg/observability" "github.com/singchia/frontier/pkg/utils" "k8s.io/klog/v2" ) @@ -14,6 +17,7 @@ import ( type Frontlas struct { repo *repo.Dao server *server.Server + obs *observability.Server } func NewFrontlas() (*Frontlas, error) { @@ -53,17 +57,36 @@ func NewFrontlas() (*Frontlas, error) { return nil, err } + // observability：默认开启；地址不填走 0.0.0.0:9092（与 frontier 9091 错开）。 + obsCfg := conf.Observability + if obsCfg.Addr == "" { + obsCfg.Addr = "0.0.0.0:9092" + } + if !obsCfg.Enable { + obsCfg.Enable = true + } + obs := observability.New(observability.Config{Enable: obsCfg.Enable, Addr: obsCfg.Addr}) + // readiness 反映 Redis 是否可达。 + obs.SetReadiness(func(ctx context.Context) error { + c, cancel := context.WithTimeout(ctx, 2*time.Second) + defer cancel() + return repo.Ping(c) + }) + return &Frontlas{ repo: repo, server: server, + obs: obs, }, nil } func (frontlas *Frontlas) Run() { + frontlas.obs.Run() frontlas.server.Serve() } func (frontlas *Frontlas) Close() { + frontlas.obs.Shutdown(5 * time.Second) frontlas.repo.Close() frontlas.server.Close() klog.Infof("frontlas ends") diff --git a/pkg/frontlas/repo/dao.go b/pkg/frontlas/repo/dao.go index 9ff7cc0..6da6bb5 100644 --- a/pkg/frontlas/repo/dao.go +++ b/pkg/frontlas/repo/dao.go @@ -165,3 +165,8 @@ func NewDao(conf *config.Configuration) (*Dao, error) { func (dao *Dao) Close() error { return dao.rds.Close() } + +// Ping 探测 Redis 是否可达，给 /readyz 用。 +func (dao *Dao) Ping(ctx context.Context) error { + return dao.rds.Ping(ctx).Err() +} diff --git a/pkg/observability/server.go b/pkg/observability/server.go new file mode 100644 index 0000000..872dbb3 --- /dev/null +++ b/pkg/observability/server.go @@ -0,0 +1,110 @@ +// Package observability 给 frontier 和 frontlas 提供统一的可观测性 HTTP 端点： +// +// /healthz - liveness：进程存活即 200 +// /readyz - readiness：业务侧准备就绪后 200，否则 503 +// /metrics - Prometheus exporter，使用默认 registry（含 Go runtime + process collector） +// +// gospec 红线："所有对外服务必须暴露 /healthz、/readyz、/metrics"。 +package observability + +import ( + "context" + "errors" + "net/http" + "sync/atomic" + "time" + + "github.com/prometheus/client_golang/prometheus/promhttp" + "k8s.io/klog/v2" +) + +// Config 控制 observability HTTP server 行为。 +type Config struct { + // Enable 关闭时整个 server 不启动；默认 true（zero value 反向初始化在 New 中处理）。 + Enable bool + // Addr 是 HTTP 监听地址，例如 "0.0.0.0:9091"。 + Addr string +} + +// ReadinessFn 在每次 /readyz 被请求时调用，返回 nil 表示 ready。 +// 业务侧通过它注入"必要依赖是否就绪"的判断（例如 Frontlas 已注册、Redis 可达）。 +// 默认为永远 ready。 +type ReadinessFn func(ctx context.Context) error + +// Server 提供生命周期受控的 HTTP server。 +type Server struct { + cfg Config + srv *http.Server + readiness atomic.Pointer[ReadinessFn] +} + +// New 构造一个 Server。注册 Prometheus 默认 registry 上 promauto 增量定义的所有指标。 +// 如果想注册自定义 collector，业务包直接 prometheus.MustRegister 即可，这里不需要传引用。 +func New(cfg Config) *Server { + s := &Server{cfg: cfg} + mux := http.NewServeMux() + mux.HandleFunc("/healthz", s.handleHealthz) + mux.HandleFunc("/readyz", s.handleReadyz) + mux.Handle("/metrics", promhttp.Handler()) + s.srv = &http.Server{ + Addr: cfg.Addr, + Handler: mux, + ReadHeaderTimeout: 5 * time.Second, + } + defaultReady := ReadinessFn(func(context.Context) error { return nil }) + s.readiness.Store(&defaultReady) + return s +} + +// SetReadiness 替换当前的就绪检查函数。可以在进程生命周期里多次调用， +// 比如启动初期返回 NotReady、注册到 Frontlas 后切到 Ready、SIGTERM 后切回 NotReady。 +func (s *Server) SetReadiness(fn ReadinessFn) { + if fn == nil { + fn = func(context.Context) error { return nil } + } + s.readiness.Store(&fn) +} + +// Run 在独立 goroutine 启动监听。Run 返回前同步检查 cfg.Enable。 +func (s *Server) Run() { + if !s.cfg.Enable { + klog.Infof("observability server disabled") + return + } + go func() { + klog.Infof("observability server listening on %s", s.cfg.Addr) + if err := s.srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) { + klog.Errorf("observability server stopped with error: %s", err) + } + }() +} + +// Shutdown 触发 graceful shutdown，最多等 timeout。 +func (s *Server) Shutdown(timeout time.Duration) { + if !s.cfg.Enable { + return + } + ctx, cancel := context.WithTimeout(context.Background(), timeout) + defer cancel() + if err := s.srv.Shutdown(ctx); err != nil { + klog.Warningf("observability server shutdown: %s", err) + } +} + +func (s *Server) handleHealthz(w http.ResponseWriter, _ *http.Request) { + // liveness 仅证明进程仍在响应请求；不调用 readinessFn。 + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte("ok")) +} + +func (s *Server) handleReadyz(w http.ResponseWriter, r *http.Request) { + fn := *s.readiness.Load() + if err := fn(r.Context()); err != nil { + w.Header().Set("Content-Type", "text/plain; charset=utf-8") + w.WriteHeader(http.StatusServiceUnavailable) + _, _ = w.Write([]byte("not ready: " + err.Error())) + return + } + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte("ready")) +} From 36fc078d5e9d8525e7cf461527fe9f29c03d6a9d Mon Sep 17 00:00:00 2001 From: singchia Date: Sat, 2 May 2026 09:28:18 +0800 Subject: [PATCH 12/12] docs(website): comprehensive Kubernetes Operator & CRD guide MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replaces the 68-line stub at /docs/operator with a full reference covering the M1+M2+M3+M4 deliverables. Twelve sections: 1. Overview — two-tier architecture, what one CR gets you 2. Installation — install.yaml + RBAC walkthrough 3. Quick start — minimal CR + verification commands 4. CRD field reference — three tables (Frontier, Frontlas, PodOverrides) documenting every spec field, type, default, and intent 5. Common scenarios — mTLS, production resources/scheduling, private registry, Prometheus + cert-manager annotations, SecurityContext override for legacy images 6. Status & Conditions — Available/Progressing/Degraded semantics 7. Observability endpoints — /healthz, /readyz, /metrics ports for both frontier and frontlas 8. Common kubectl operations — short name fc, patches, JSON path tricks 9. Operator behavior — reconcile order, owner references, graceful shutdown env, event types 10. Troubleshooting — six failure modes mapped to root cause + remediation 11. Known limitations — no kubectl scale, alpha API, Helm gaps, no validation webhook 12. Roadmap — pointer to RFC-001 in the repo Verified with `npx next build` — page generates statically alongside the existing /docs/* routes. Refs: docs/rfc/RFC-001-cloud-native-optimization.md (user-facing deliverable for the cloud-native optimization initiative) Co-Authored-By: Claude Opus 4.7 (1M context) --- website/src/app/docs/operator/page.tsx | 352 ++++++++++++++++++++++--- 1 file changed, 317 insertions(+), 35 deletions(-) diff --git a/website/src/app/docs/operator/page.tsx b/website/src/app/docs/operator/page.tsx index 461769f..d3b190d 100644 --- a/website/src/app/docs/operator/page.tsx +++ b/website/src/app/docs/operator/page.tsx @@ -2,68 +2,350 @@ export default function OperatorGuide() { return (
-

Kubernetes Operator

+

Kubernetes Operator & CRD

- The official Kubernetes Operator for deploying Frontier clusters. + Run Frontier on Kubernetes with a single FrontierCluster resource. The operator handles deployments, TLS secrets, services, probes, security context, and graceful shutdown.

+ +

1. Overview

+

+ The Frontier operator manages a two-tier deployment: Frontier (data plane, stateless edge gateway) and Frontlas (control plane, Redis-backed coordinator). Both are reconciled from a single namespaced custom resource FrontierCluster in the API group frontier.singchia.io/v1alpha1. +

- The Frontier Operator automates the deployment, provisioning, and scaling of Frontier and Frontlas clusters inside Kubernetes. + With one CR you get: two Deployments, three Services (servicebound, edgebound, controlplane), TLS material copied into operator-managed Secrets, sane production defaults (probes, preStop, non-root, drop-all caps, preferred anti-affinity), and a status that surfaces ready replicas and Conditions.

-

1. Install the CRD and Operator

-

First, clone the repository and apply the Custom Resource Definitions (CRDs):

- 
{`git clone https://github.com/singchia/frontier.git
-cd frontier/dist/crd
-kubectl apply -f install.yaml`}
+

2. Installation

+

2.1 Install the CRD and operator

+ 
{`# Apply the CRD + operator deployment + RBAC in one shot
+kubectl apply -f https://raw.githubusercontent.com/singchia/frontier/main/pkg/operator/dist/install.yaml
 
-        
Verify that the CRD is installed:

-        
{`kubectl get crd frontierclusters.frontier.singchia.io`}

+# Or from a local checkout
+git clone https://github.com/singchia/frontier.git
+kubectl apply -f frontier/pkg/operator/dist/install.yaml`}
-

Verify that the Operator is running:

- 
{`kubectl get all -n frontier-system`}
+

Verify the CRD and operator pod:

+ 
{`kubectl get crd frontierclusters.frontier.singchia.io
+kubectl get all -n frontier-operator-system`}
-

2. Deploy a FrontierCluster

-

Create a file named frontiercluster.yaml. This example deploys 2 Frontier gateways and 1 Frontlas control plane relying on an existing Redis sentinel.

+

2.2 RBAC

+

The bundled install.yaml creates a ClusterRole granting the operator:

+
    +
  • frontier.singchia.io/frontierclusters — full CRUD + status
    • +
  • apps/deployments — full CRUD (manages frontier & frontlas Deployments)
    • +
  • core/services, secrets, pods — full CRUD (services, TLS material, pod inspection)
    • +
  • core/events — create + patch (used by EventRecorder)
    • +
+

If you tighten this further, keep at least get;list;watch on those resources or reconcile will fail.

+

3. Quick start

+

Minimum viable cluster: 2 frontier replicas, 1 frontlas replica, external Redis. Save as frontiercluster.yaml:

{`apiVersion: frontier.singchia.io/v1alpha1
 kind: FrontierCluster
 metadata:
-  name: my-frontier-cluster
+  name: prod
+  namespace: frontier
 spec:
   frontier:
     replicas: 2
-    servicebound:
-      port: 30011
-    edgebound:
-      port: 30012
   frontlas:
     replicas: 1
-    controlplane:
-      port: 40011
     redis:
       addrs:
-        - rfs-redisfailover:26379
-      password: your-password
-      masterName: mymaster
-      redisType: sentinel`}
+ - redis.frontier:6379 + passwordSecret: + name: redis-creds + key: password + redisType: standalone`} + + 
{`kubectl create namespace frontier
+kubectl -n frontier create secret generic redis-creds --from-literal=password=...
+kubectl apply -f frontiercluster.yaml`}
+ +

Wait for it to come up, then check:

+ 
{`kubectl -n frontier get fc                  # short name 'fc' is registered
+NAME   PHASE     FRONTIER   FRONTLAS   AGE
+prod   Running   2          1          47s
+
+kubectl -n frontier describe fc prod         # see Conditions + Events
+kubectl -n frontier get pods                 # frontier + frontlas pods
+kubectl -n frontier get svc                  # 3 services rendered`}
+ +

4. CRD field reference

+

Everything below lives under spec. All fields outside frontier.servicebound, frontier.edgebound, frontlas.controlplane, and frontlas.redis are optional — sensible defaults apply.

+ +

4.1 spec.frontier

+ + + + + + + + + + + + + + + + + + + + + +
FieldTypeDefaultNotes
replicasint1Frontier pod count
imagestringsingchia/frontier:1.1.0Override to pin a specific tag
servicebound.portint30011Service-side TCP/gRPC port
servicebound.servicestring{`-servicebound-svc`}Service name override
servicebound.serviceTypestringClusterIPClusterIP / NodePort / LoadBalancer
edgebound.portint30012Edge-side port (typically external)
edgebound.serviceNamestring{`-edgebound-svc`}Service name override
edgebound.serviceTypestringNodePortUse LoadBalancer for cloud egress
edgebound.tls.enabledboolfalseEnable TLS on edgebound
edgebound.tls.optionalboolfalseIf true, both TLS and plain accepted
edgebound.tls.mtlsboolfalseEnable client cert verification
edgebound.tls.certificateKeySecretRef.namestring—User Secret with tls.crt, tls.key
edgebound.tls.caCertificateSecretRef.namestring—User Secret with ca.crt (mTLS only)
nodeAffinityNodeAffinitynilLegacy — use pod.affinity instead
podPodOverridessee §4.3Production-grade overrides for the frontier pod
+ +

4.2 spec.frontlas

+ + + + + + + + + + + + + + + + + + + + + +
FieldTypeDefaultNotes
replicasint1Frontlas pod count
imagestringsingchia/frontlas:1.1.0Image override
controlplane.portint40011Service-side control plane port
controlplane.frontierPlanePortint40012Port used by frontier nodes to talk to frontlas
controlplane.servicestring{`-frontlas-svc`}Service name override
controlplane.serviceTypestringClusterIPInternal only by default
redis.addrs[]stringrequiredOne or more Redis addrs
redis.redisTypestringrequiredstandalone / sentinel / cluster
redis.dbint0DB index (standalone only)
redis.userstring""For Redis ACL
redis.passwordstring""Deprecated — use passwordSecret
redis.passwordSecretSecretKeySelectornilRecommended. Wins over password; injected via valueFrom.secretKeyRef
redis.masterNamestring""Sentinel only
nodeAffinityNodeAffinitynilLegacy — use pod.affinity
podPodOverridessee §4.3Production-grade overrides for the frontlas pod
-

Apply the cluster configuration:

- 
{`kubectl apply -f frontiercluster.yaml`}
+

4.3 spec.frontier.pod / spec.frontlas.pod (PodOverrides)

+

Every override is optional. When unset, the operator applies a production-grade default.

+ + + + + + + + + + + + + + + + + + + + + + + +
FieldTypeOperator defaultUse case
resourcesResourceRequirementsnil (BestEffort QoS)Set CPU/memory requests + limits for production
nodeSelectormap[string]stringnilPin to nodes by label
tolerations[]TolerationnilRun on tainted nodes
topologySpreadConstraints[]TopologySpreadConstraintnilCross-zone / cross-node spread
affinityAffinityonly PodAntiAffinity (preferred host spread)Setting this fully replaces the default and the legacy nodeAffinity field
priorityClassNamestring""Critical workload priority
serviceAccountNamestringdefaultBind workload identity
imagePullSecrets[]LocalObjectReferencenilPrivate registry credentials
imagePullPolicystringIfNotPresentUse Always in dev when pinning latest
annotationsmap[string]stringnilPod annotations — cert-manager, Prometheus scrape config, sidecar opt-in
labelsmap[string]stringapp=…Extra labels (merged with selector labels)
podSecurityContextPodSecurityContextrunAsNonRoot=true, UID/GID/FSGroup=65532, RuntimeDefault seccompOverride only when an image needs root or a different UID
containerSecurityContextSecurityContextdrop ALL caps, AllowPrivilegeEscalation=false, runAsNonRoot=trueOverride to add a specific capability back
terminationGracePeriodSecondsint64frontier=60, frontlas=30Long-lived edge connections need at least 60
livenessProbeProbeTCP socket on edge port (frontier) / control port (frontlas)Replace with HTTP probe in M3+
readinessProbeProbeTCP socket on service port (frontier) / HTTP /cluster/v1/health (frontlas)HTTP /readyz available since M3
lifecycleLifecyclepreStop: sleep 10 (frontier) / sleep 5 (frontlas)Lets kube-proxy remove pod from Service Endpoints before SIGTERM
-

3. Verify the Cluster

-

Within a minute, your HA cluster will be ready. You can check the status of the pods:

- 
{`kubectl get all -l app=frontiercluster-frontier
-kubectl get all -l app=frontiercluster-frontlas`}
+

5. Common scenarios

-

4. Connect your workloads

+

5.1 Edge mTLS

+

Provide both a server cert/key and a CA. The operator copies them into namespace-scoped Secrets and mounts them into the frontier pod at /app/conf/edgebound/tls/secret and /app/conf/edgebound/tls/ca.

+ 
{`apiVersion: v1
+kind: Secret
+metadata:
+  name: edge-server-cert
+type: kubernetes.io/tls
+data:
+  tls.crt: ...    # PEM cert
+  tls.key: ...    # PEM key
+---
+apiVersion: v1
+kind: Secret
+metadata:
+  name: edge-ca
+data:
+  ca.crt: ...     # PEM CA
+---
+apiVersion: frontier.singchia.io/v1alpha1
+kind: FrontierCluster
+metadata:
+  name: prod
+spec:
+  frontier:
+    edgebound:
+      port: 8443
+      serviceType: LoadBalancer
+      tls:
+        enabled: true
+        mtls: true
+        certificateKeySecretRef:
+          name: edge-server-cert
+        caCertificateSecretRef:
+          name: edge-ca
+  frontlas: { ... }`}
+ +

5.2 Production resources + scheduling

+ 
{`spec:
+  frontier:
+    replicas: 6
+    pod:
+      resources:
+        requests: { cpu: "500m", memory: "512Mi" }
+        limits:   { cpu: "2",    memory: "2Gi" }
+      tolerations:
+        - key: workload
+          operator: Equal
+          value: edge-gateway
+          effect: NoSchedule
+      topologySpreadConstraints:
+        - maxSkew: 1
+          topologyKey: topology.kubernetes.io/zone
+          whenUnsatisfiable: ScheduleAnyway
+          labelSelector:
+            matchLabels: { app: prod-frontier }
+      priorityClassName: frontend-critical
+      terminationGracePeriodSeconds: 120`}
+ +

5.3 Private image registry

+ 
{`spec:
+  frontier:
+    image: my-registry.example.com/frontier:1.2.4
+    pod:
+      imagePullSecrets:
+        - name: my-registry-creds
+      imagePullPolicy: IfNotPresent
+  frontlas:
+    image: my-registry.example.com/frontlas:1.2.4
+    pod:
+      imagePullSecrets:
+        - name: my-registry-creds`}
+ +

5.4 Annotations for Prometheus + cert-manager

+ 
{`spec:
+  frontier:
+    pod:
+      annotations:
+        prometheus.io/scrape: "true"
+        prometheus.io/port: "9091"
+        prometheus.io/path: "/metrics"
+  frontlas:
+    pod:
+      annotations:
+        cert-manager.io/inject-ca-from: frontier/frontier-ca`}
+ +

5.5 Override SecurityContext for legacy images

+

If your custom frontier image needs root or a non-65532 UID, opt out of the default explicitly:

+ 
{`spec:
+  frontier:
+    pod:
+      podSecurityContext: {}                    # drop the default nonRoot/UID
+      containerSecurityContext:
+        runAsNonRoot: false
+        capabilities:
+          drop: []                              # keep capabilities`}
+ +

6. Status & Conditions

+

The CRD has a status subresource (read-only for users):

+ 
{`status:
+  phase: Running                # Pending | Running | Failed (deprecated, kept for printcolumn)
+  message: "Good to go!"
+  observedGeneration: 7         # spec.generation that this status reflects
+  frontierReadyReplicas: 6
+  frontlasReadyReplicas: 2
+  conditions:
+    - type: Available
+      status: "True"
+      reason: AllComponentsReady
+      lastTransitionTime: "2026-05-02T01:23:45Z"
+      observedGeneration: 7
+    - type: Progressing
+      status: "False"
+      reason: ReconcileSucceeded
+    - type: Degraded
+      status: "False"`}
+ +

Three conditions are maintained:

+
    +
  • Available — True when both Deployments report all replicas ready.
    • +
  • Progressing — True while the operator is still reconciling toward desired state.
    • +
  • Degraded — True when reconcile failed (TLS Secret missing, deployment error, etc.). Inspect kubectl describe fc for the Events stream.
    • +
+ +

7. Observability endpoints (since M3)

+

Both frontier and frontlas expose three HTTP endpoints on a separate port:

+ + + + + + + + + +
EndpointFrontier portFrontlas portSemantics
/healthz90919092Liveness — 200 if process responds
/readyz90919092Readiness — 503 with details when not ready (e.g. Redis unreachable for frontlas)
/metrics90919092Prometheus default registry — Go runtime + process metrics
+

Configure via the observability block in frontier.yaml / frontlas.yaml:

+ 
{`observability:
+  enable: true
+  addr: 0.0.0.0:9091`}
+

The default behavior is on; set enable: false to disable.

+ +

8. Common operations

+ 
{`# CRUD with the short name
+kubectl get fc
+kubectl describe fc prod
+kubectl edit fc prod
+kubectl delete fc prod
+
+# Inspect Conditions
+kubectl get fc prod -o jsonpath='{.status.conditions}' | jq
+
+# Watch reconcile events
+kubectl describe fc prod | tail -20
+
+# Patch the replica count without an editor
+kubectl patch fc prod --type=merge -p '{"spec":{"frontier":{"replicas":4}}}'`}
+ +

9. Operator behavior

    -
  • Microservices: Should connect to service/frontiercluster-frontlas-svc:40011
    • -
  • Edge Nodes: Can connect to the NodePort where :30012 is exposed externally.
    • +
  • Reconcile order. Service → TLS Secrets → Frontlas Deployment → (wait until ready) → Frontier Deployment.
    • +
  • Owner references. Deployments + Services + operator-managed Secrets all carry the FrontierCluster as owner; deleting the CR cascades to all of them.
    • +
  • Graceful shutdown. Frontier honors FRONTIER_DRAIN_SECONDS (operator injects terminationGracePeriodSeconds - 10): on SIGTERM it waits this many seconds before tearing connections down, letting kube-proxy fully drop the pod from Service Endpoints first.
    • +
  • Events. Each meaningful state transition emits a Kubernetes Event: ServiceEnsureFailed, TLSEnsureFailed, DeploymentEnsureFailed, Available (one-shot when the cluster first becomes ready).
+ +

10. Troubleshooting

+ + + + + + + + + + + + +
SymptomLikely causeWhere to look
Frontier pod CrashLoopBackOff with connect: connection refused on the frontier-plane portFrontlas not yet ready, or Redis unreachable from frontlaskubectl describe fc Conditions; kubectl logs deploy/{``}-frontlas
Frontier pod fails to start: container can't run as nonRootCustom image without a non-root USER directiveOverride spec.frontier.pod.podSecurityContext + containerSecurityContext, or use singchia/frontier:1.2.4+ which ships with USER 65532
Status stays Pending for minutesOne of the Deployments not converging on ready replicaskubectl describe fc + kubectl get pods + pod Events
TLS-enabled cluster can't serve mTLSMissing ca.crt in the user CA Secret, or the operator-managed Secret was deleted manuallyOperator log: Error ensuring tls secret; check user Secret keys exactly match tls.crt, tls.key, ca.crt
Cluster keeps re-reconciling but never settlesSome required spec field changed (e.g. ServiceType) and K8s rejects the updateOperator log + kubectl get events
Redis password is visible in kubectl describe podUsing deprecated spec.frontlas.redis.password instead of passwordSecretMove to passwordSecret — injected via valueFrom.secretKeyRef with no plaintext leak
+ +

11. Known limitations

+
    +
  • No kubectl scale — the spec has two replica fields (frontier & frontlas) so the scale subresource isn't enabled. Patch spec.frontier.replicas directly. HPA targets the underlying Deployments instead.
    • +
  • v1alpha1 — no compatibility guarantees between alpha versions. The next bump goes to v1beta1 alongside conversion machinery.
    • +
  • Helm chart only ships frontier templates — the operator path (this page) is the recommended deployment route. Helm-only users should bring their own frontlas + Redis manifests until the chart catches up.
    • +
  • No webhook validation — bad input (e.g. negative replicas, invalid redisType) is caught at reconcile time, not at kubectl apply.
    • +
+ +

12. Roadmap

+

This page reflects RFC-001 “Cloud-native optimization” through M3 (observability) and M4 (Status conditions + EventRecorder + CRD ergonomics). Open RFC content lives at docs/rfc/RFC-001-cloud-native-optimization.md in the repository.

+
); -} \ No newline at end of file +}