Skip to content

kra/ci config normalization #1420

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kra wants to merge 10 commits into
base: main
Choose a base branch
from
Open

kra/ci config normalization #1420

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
9b3a872
Update docker image source for minio and minio-init from docker hub t…
kra Sep 15, 2026
775e1da
update docker-compose.ci.yml also
kra Sep 15, 2026
77de55c
Normalize config between the CI and non-CI docker versions
kra Sep 15, 2026
a83b3b9
Normalize config between the CI and non-CI docker versions
kra Sep 15, 2026
dd585a3
make order match docker-compose.yml, not docker-compose.ci.yml, just …
kra Sep 30, 2026
c4579ad
Merge branch 'kra/ci-config-normalization' of github.com:RolnickLab/a…
kra Sep 30, 2026
8295d91
Normalize config between the CI and non-CI docker versions
kra Sep 15, 2026
5eb16a3
Merge branch 'kra/ci-config-normalization' of github.com:RolnickLab/a…
kra Oct 1, 2026
d629045
Merge branch 'main' into kra/ci-config-normalization
kra Oct 5, 2026
869ed0d
remove container_name from ci docker compose file to avoid a containe…
kra Oct 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 10 additions & 14 deletions .envs/.ci/.django
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
# General
# ------------------------------------------------------------------------------
USE_DOCKER=yes
DJANGO_SETTINGS_MODULE="config.settings.local"
USE_DOCKER=yes

# RabbitMQ
CELERY_BROKER_URL=amqp://rabbituser:rabbitpass@rabbitmq:5672/
CELERY_RESULT_BACKEND=rpc:// # Use RabbitMQ for results backend
RABBITMQ_DEFAULT_USER=rabbituser
RABBITMQ_DEFAULT_PASS=rabbitpass

# Redis
# ------------------------------------------------------------------------------
REDIS_URL=redis://redis:6379/0

DJANGO_CSRF_TRUSTED_ORIGINS=http://localhost:3000,http://localhost:4000
Expand All @@ -17,19 +21,11 @@ MINIO_STORAGE_USE_HTTPS=False
MINIO_TEST_BUCKET=ami-test-ci
MINIO_BROWSER_REDIRECT_URL=http://minio:9001

# NATS
NATS_URL=nats://nats:4222

DEFAULT_PROCESSING_SERVICE_NAME=Test Processing Service
DEFAULT_PROCESSING_SERVICE_ENDPOINT=http://ml_backend:2000

# RabbitMQ
# ------------------------------------------------------------------------------
CELERY_BROKER_URL=amqp://rabbituser:rabbitpass@rabbitmq:5672/
CELERY_RESULT_BACKEND=rpc:// # Use RabbitMQ for results backend
RABBITMQ_DEFAULT_USER=rabbituser
RABBITMQ_DEFAULT_PASS=rabbitpass

# Email backend to use.
DJANGO_EMAIL_BACKEND=django.core.mail.backends.locmem.EmailBackend

# NATS
# ------------------------------------------------------------------------------
NATS_URL=nats://nats:4222
21 changes: 11 additions & 10 deletions .envs/.local/.django
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,17 @@ IPYTHONDIR=/app/.ipython
DJANGO_SUPERUSER_EMAIL=antenna@insectai.org
DJANGO_SUPERUSER_PASSWORD=localadmin

# RabbitMQ
CELERY_BROKER_URL=amqp://rabbituser:rabbitpass@rabbitmq:5672/
CELERY_RESULT_BACKEND=rpc:// # Use RabbitMQ for results backend
RABBITMQ_DEFAULT_USER=rabbituser
RABBITMQ_DEFAULT_PASS=rabbitpass
RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS=-rabbit consumer_timeout 604800000


# Redis
REDIS_URL=redis://redis:6379/0

# NATS
NATS_URL=nats://nats:4222

# Upload size limit (integer, binary MiB — multiplied by 1024*1024). 100 MiB default
# is fine for local dev. Raise only if testing large ML result payloads locally.
# DJANGO_DATA_UPLOAD_MAX_MEMORY_MB=100
Expand All @@ -28,13 +33,6 @@ CELERY_FLOWER_PASSWORD=BEQgmCtgyrFieKNoGTsux9YIye0I7P5Q7vEgfJD2C4jxmtHDetFaE2jhS
# each prefork worker is a separate Python process with its own DB connection.
# CELERY_WORKER_CONCURRENCY=4

# RabbitMQ
CELERY_BROKER_URL=amqp://rabbituser:rabbitpass@rabbitmq:5672/
CELERY_RESULT_BACKEND=rpc:// # Use RabbitMQ for results backend
RABBITMQ_DEFAULT_USER=rabbituser
RABBITMQ_DEFAULT_PASS=rabbitpass
RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS=-rabbit consumer_timeout 604800000

# Email backend to use.
DJANGO_EMAIL_BACKEND=django.core.mail.backends.console.EmailBackend

Expand All @@ -60,6 +58,9 @@ MINIO_STORAGE_USE_HTTPS=False
MINIO_TEST_BUCKET=ami-test
MINIO_BROWSER_REDIRECT_URL=http://minio:9001

# NATS
NATS_URL=nats://nats:4222

# Default processing service (local)
DEFAULT_PROCESSING_SERVICE_NAME=Local Processing Service
DEFAULT_PROCESSING_SERVICE_ENDPOINT=http://ml_backend:2000
Expand Down
10 changes: 5 additions & 5 deletions docker-compose.ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,24 +12,24 @@ services:
build:
context: .
dockerfile: ./compose/local/django/Dockerfile
volumes:
- .:/app:z
extra_hosts:
- "host.docker.internal:host-gateway"
depends_on:
postgres:
condition: service_started
redis:
condition: service_started
nats:
condition: service_started
# Wait for bucket setup to succeed so a failure stops the job here, not in later tests.
minio-init:
condition: service_completed_successfully
ml_backend:
condition: service_started
rabbitmq:
condition: service_started
nats:
condition: service_started
volumes:
- .:/app:z
env_file:
- ./.envs/.ci/.django
- ./.envs/.ci/.postgres
Expand Down Expand Up @@ -60,9 +60,9 @@ services:

celeryworker:
<<: *django
command: /start-celeryworker
depends_on:
- rabbitmq
command: /start-celeryworker

rabbitmq:
image: rabbitmq:3-management
Expand Down
Loading