diff --git a/README.md b/README.md index f1e89f2..37a3a9d 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ This repository contains the _HotPocket_ project. **Exported services:** * The app: https://app.hotpocket.work.bthlabs.net:8000/ -* The admin: https://app.hotpocket.work.bthlabs.net:8000/ +* The admin: https://admin.hotpocket.work.bthlabs.net:8000/ * Keycloak: https://auth.hotpocket.work.bthlabs.net:8443/ * Postgres: postgres://postgres.hotpocket.work.bthlabs.net:5432/ * RabbitMQ: amqp://rabbitmq.hotpocket.work.bthlabs.net:5672/ @@ -43,7 +43,120 @@ and admin using OIDC. ## Deployment -TODO +There are two deployment images - `aio` and `deployment`. + +### The AIO image + +The `aio` image is pre-configured for running small instances in a single +container: + +* It defaults to SQLite database. +* It defaults to running all background tasks in the foreground. +* It defaults to accepting traffic with any `Host` HTTP header. + +The `aio` image is recommended for self-hosting with minimal use, e.g. by a +single user. + +**Example:** + +``` +$ docker run --rm -it \ + -v `realpath run/`:/srv/run \ + -e HOTPOCKET_BACKEND_SECRET_KEY=thisisntright \ + -e HOTPOCKET_BACKEND_INITIAL_ACCOUNT_USERNAME=hotpocket \ + -e HOTPOCKET_BACKEND_INITIAL_ACCOUNT_PASSWORD=hotpocketm4st3r \ + -p 8000:8000 \ + docker-hosted.nexus.bthlabs.pl/hotpocket/backend:aio-v1.0.0rc1-01 +``` + +The command above will set up and start the application. The SQLite file will +be placed in `run/hotpocket-backend-aio.sqlite` and database migrations will +be ran. The initial superuser account will be created with the specified +credentials. The Web app will be reachable at `http://127.0.0.1:8000/`. +The admin will be reachable at `http://127.0.0.1:8000/admin/`. + +The `DJANGO_SETTINGS_MODULE` environment variable defaults to +`hotpocket_backend.settings.deployment.webapp`. This should be set to +`hotpocket_backend.settings.deployment.admin` in the Admin container. + +**NOTE:** The command above specifies wildly insecure `SECRET_KEY` which is +used among other things to secure the session cookie. Please *please* +**please** don't run it like this. Not even in your homelab :). + +The `deployment/aio/docker-compose.yaml` file can be used as a starting +point for AIO deployments. + +### The Deployment image + +The `deployment` image doesn't make any assumptions about the env and in turn +will require the operator to configure database, Celery broker and result +backend etc. The final deployment will require services for at least the Web +app, the Celery worker and Celery Beat. Admin is optional. + +The `DJANGO_SETTINGS_MODULE` environment variable defaults to +`hotpocket_backend.settings.deployment.aio`. + +The `deployment/fullstack/docker-compose.yaml` file can be used as a +starting point for full-stack deployments. + +### Configuration environment variables + +HotPocket deployment images provide extensive set of environment variables +that can be used to configure the services. + +| Variable | Default | Description | +|----------------------------------------------|-----------------------------------------------------------------|---------------------------------------------------------------------------------------------| +| `HOTPOCKET_BACKEND_ENV` | `deployment` or `aio` | The environment name. See below. | +| `HOTPOCKET_BACKEND_APP` | `webapp` | The app name. See below. | +| `HOTPOCKET_BACKEND_DEBUG` | `false` | Django `DEBUG` setting. **Do not enable in production**. Only effective in the AIO image. | +| `HOTPOCKET_BACKEND_ALLOWED_HOSTS` | N/A or `*` | Django `ALLOWED_HOSTS` setting. **Required in the Deployment image.** | +| `HOTPOCKET_BACKEND_SECRET_KEY` | N/A | Django `SECRET_KEY` setting. Recommended different for the Web app and Admin. **Required**. | +| `HOTPOCKET_BACKEND_DATABASE_ENGINE` | `django.db.backends.postgresql` or `django.db.backends.sqlite3` | The database configuration engine. | +| `HOTPOCKET_BACKEND_DATABASE_NAME` | N/A or `/srv/run/hotpocket-backend-aio.sqlite` | The database name. | +| `HOTPOCKET_BACKEND_DATABASE_USER` | N/A or N/A | The database user. | +| `HOTPOCKET_BACKEND_DATABASE_PASSWORD` | N/A | The database password. | +| `HOTPOCKET_BACKEND_DATABASE_HOST` | N/A | The database host. | +| `HOTPOCKET_BACKEND_DATABASE_PORT` | `5432` or N/A | The database port. | +| `HOTPOCKET_BACKEND_MODEL_AUTH_IS_DISABLED` | `false` | Set to `true` to disable username and password login. | +| `HOTPOCKET_BACKEND_OIDC_PAYLOAD` | N/A | The OIDC configuration payload. | +| `HOTPOCKET_BACKEND_CELERY_BROKER_URL` | N/A | The Celery broker URL. | +| `HOTPOCKET_BACKEND_CELERY_RESULT_BACKEND` | N/A | The Celery result backend URL. | +| `HOTPOCKET_BACKEND_CELERY_IGNORE_RESULT` | `false` | Set to `true` to prevent Celery from saving task results. | +| `HOTPOCKET_BACKEND_CELERY_ALWAYS_EAGER` | `false` | Set to `true` to run Celery tasks in the foreground. | +| `HOTPOCKET_BACKEND_UPLOADS_PATH` | `/srv/uploads` or `/srv/run/uploads` | The absolute path to user-uploaded files. | +| `HOTPOCKET_BACKEND_GUNICORN_WORKERS` | `4` or `2` | The number of Gunicorn workers to run for Web servers. | +| `HOTPOCKET_BACKEND_RUN_MIGRATIONS` | `false` or `true` | Set to `true` to run database muigrations when the container starts. | +| `HOTPOCKET_BACKEND_INITIAL_ACCOUNT_USERNAME` | N/A | Username for the initial account. | +| `HOTPOCKET_BACKEND_INITIAL_ACCOUNT_PASSWORD` | N/A | Password for the initial account. | + +**Env and App settings** + +The `HOTPOCKET_BACKEND_ENV` and `HOTPOCKET_BACKEND_APP` variables are used +internally to resolve other settings and identify the running app. +`HOTPOCKET_BACKEND_ENV` should only be changed if when creating heavily +customized version of the project. `HOTPOCKET_BACKEND_APP` should generally be +set to `admin` only in the Admin container. + +**OIDC login configuration** + +The `HOTPOCKET_BACKEND_OIDC_PAYLOAD` can be used to enable OIDC login. It must +be a JSON string that deserializes to the following object: + +``` +{ + "endpoint": "https://some.oidc.host/some-realm/", + "key": "client-key", + "secret": "client-secret", + "scope": ["roles"], + "display_name": "My OIDC server" +} +``` + +The `scope` field specified additional scopes to request from IdP. It can be +ommited and defaults to `["roles"]`. The `display_name` field specifies the +method's name in the UI and defaults to `OIDC`. + +**NOTE:** Currently, only Keycloak has been tested with this login method. ## Author diff --git a/deployment/aio/.gitignore b/deployment/aio/.gitignore new file mode 100644 index 0000000..7b5bdc2 --- /dev/null +++ b/deployment/aio/.gitignore @@ -0,0 +1,2 @@ +run/*.sqlite +uploads/ diff --git a/deployment/aio/docker-compose.yaml b/deployment/aio/docker-compose.yaml new file mode 100644 index 0000000..959dd1e --- /dev/null +++ b/deployment/aio/docker-compose.yaml @@ -0,0 +1,12 @@ +services: + backend: + image: "docker-hosted.nexus.bthlabs.pl/hotpocket/backend:aio-v1.0.0rc1-01" + environment: + HOTPOCKET_BACKEND_SECRET_KEY: "thisisntright" + HOTPOCKET_BACKEND_INITIAL_ACCOUNT_USERNAME: "hotpocket" + HOTPOCKET_BACKEND_INITIAL_ACCOUNT_PASSWORD: "hotpocketm4st3r" + ports: + - "8000:8000" + volumes: + - "./run:/srv/run" + restart: "unless-stopped" diff --git a/deployment/aio/run/.placeholder b/deployment/aio/run/.placeholder new file mode 100644 index 0000000..e69de29 diff --git a/deployment/fullstack/.gitignore b/deployment/fullstack/.gitignore new file mode 100644 index 0000000..68948b7 --- /dev/null +++ b/deployment/fullstack/.gitignore @@ -0,0 +1,2 @@ +run/celery-beat-schedule +uploads/*.csv diff --git a/deployment/fullstack/docker-compose.yaml b/deployment/fullstack/docker-compose.yaml new file mode 100644 index 0000000..53c4fe7 --- /dev/null +++ b/deployment/fullstack/docker-compose.yaml @@ -0,0 +1,77 @@ +x-backend-environment: &x-backend-environment + HOTPOCKET_BACKEND_DATABASE_NAME: "hotpocket_backend_staging" + HOTPOCKET_BACKEND_DATABASE_USER: "hotpocket" + HOTPOCKET_BACKEND_DATABASE_PASSWORD: "hotpocketm4st3r" + HOTPOCKET_BACKEND_DATABASE_HOST: "databases.bthlab.bthlabs.net" + HOTPOCKET_BACKEND_CELERY_BROKER_URL: "amqp://hotpocket:hotpocketm4st3r@databases.bthlab.bthlabs.net/hotpocket_backend_staging" + HOTPOCKET_BACKEND_CELERY_RESULT_BACKEND: "db+postgresql+psycopg://hotpocket:hotpocketm4st3r@databases.bthlab.bthlabs.net/hotpocket_backend_staging" + +services: + webapp: + image: "docker-hosted.nexus.bthlabs.pl/hotpocket/backend:deployment-v1.0.0rc1-01" + environment: + <<: *x-backend-environment + HOTPOCKET_BACKEND_ALLOWED_HOSTS: "app.staging.hotpocket.bthlab.bthlabs.net" + HOTPOCKET_BACKEND_SECRET_KEY: "thisisntright" + ports: + - "8000:8000" + volumes: + - "./run:/srv/run" + - "./uploads:/srv/uploads" + restart: "unless-stopped" + + admin: + image: "docker-hosted.nexus.bthlabs.pl/hotpocket/backend:deployment-v1.0.0rc1-01" + environment: + <<: *x-backend-environment + HOTPOCKET_BACKEND_APP: "admin" + HOTPOCKET_BACKEND_ALLOWED_HOSTS: "app.staging.hotpocket.bthlab.bthlabs.net" + HOTPOCKET_BACKEND_SECRET_KEY: "thisisntright" + ports: + - "8001:8000" + volumes: + - "./run:/srv/run" + - "./uploads:/srv/uploads" + restart: "unless-stopped" + + celery-worker: + image: "docker-hosted.nexus.bthlabs.pl/hotpocket/backend:deployment-v1.0.0rc1-01" + command: + - "/srv/venv/bin/celery" + - "-A" + - "hotpocket_backend.celery:app" + - "worker" + - "-l" + - "INFO" + - "-Q" + - "celery,webapp" + - "-c" + - "2" + environment: + <<: *x-backend-environment + HOTPOCKET_BACKEND_ALLOWED_HOSTS: "app.staging.hotpocket.bthlab.bthlabs.net" + HOTPOCKET_BACKEND_SECRET_KEY: "thisisntright" + volumes: + - "./run:/srv/run" + - "./uploads:/srv/uploads" + restart: "unless-stopped" + + celery-beat: + image: "docker-hosted.nexus.bthlabs.pl/hotpocket/backend:deployment-v1.0.0rc1-01" + command: + - "/srv/venv/bin/celery" + - "-A" + - "hotpocket_backend.celery:app" + - "beat" + - "-l" + - "INFO" + - "-s" + - "/srv/run/celery-beat-schedule" + environment: + <<: *x-backend-environment + HOTPOCKET_BACKEND_ALLOWED_HOSTS: "app.staging.hotpocket.bthlab.bthlabs.net" + HOTPOCKET_BACKEND_SECRET_KEY: "thisisntright" + volumes: + - "./run:/srv/run" + - "./uploads:/srv/uploads" + restart: "unless-stopped" diff --git a/deployment/fullstack/run/.placeholder b/deployment/fullstack/run/.placeholder new file mode 100644 index 0000000..e69de29 diff --git a/deployment/fullstack/uploads/.placeholder b/deployment/fullstack/uploads/.placeholder new file mode 100644 index 0000000..e69de29 diff --git a/services/backend/Dockerfile b/services/backend/Dockerfile index 68f0eb8..e342c66 100644 --- a/services/backend/Dockerfile +++ b/services/backend/Dockerfile @@ -72,6 +72,10 @@ ARG APP_USER_UID ARG APP_USER_GID ARG IMAGE_ID +ENV DJANGO_SETTINGS_MODULE=hotpocket_backend.settings.deployment.webapp +ENV HOTPOCKET_BACKEND_ENV=deployment +ENV HOTPOCKET_BACKEND_APP=webapp + VOLUME ["/srv/run", "/srv/uploads"] FROM deployment-base AS aio @@ -84,7 +88,12 @@ ENV DJANGO_SETTINGS_MODULE=hotpocket_backend.settings.aio ENV HOTPOCKET_BACKEND_ENV=aio ENV HOTPOCKET_BACKEND_APP=webapp ENV HOTPOCKET_BACKEND_DEBUG=false -ENV HOTPOCKET_BACKEND_DATABASE_PAYLOAD={"engine":"django.db.backends.sqlite3","name":"/srv/run/hotpocket-backend-aio.sqlite"} +ENV HOTPOCKET_BACKEND_DATABASE_ENGINE=django.db.backends.sqlite3 +ENV HOTPOCKET_BACKEND_DATABASE_NAME=/srv/run/hotpocket-backend-aio.sqlite +ENV HOTPOCKET_BACKEND_DATABASE_USER= +ENV HOTPOCKET_BACKEND_DATABASE_PASSWORD= +ENV HOTPOCKET_BACKEND_DATABASE_HOST= +ENV HOTPOCKET_BACKEND_DATABASE_PORT= ENV HOTPOCKET_BACKEND_CELERY_IGNORE_RESULT=true ENV HOTPOCKET_BACKEND_CELERY_ALWAYS_EAGER=true ENV HOTPOCKET_BACKEND_GUNICORN_WORKERS=2 diff --git a/services/backend/docker-compose-aio.yaml b/services/backend/docker-compose-aio.yaml index 4a9c61e..bffd5e5 100644 --- a/services/backend/docker-compose-aio.yaml +++ b/services/backend/docker-compose-aio.yaml @@ -1,5 +1,5 @@ services: - backend-aio-webapp: + webapp: build: context: ".." dockerfile: "backend/Dockerfile" diff --git a/services/backend/hotpocket_backend/secrets/deployment/common.py b/services/backend/hotpocket_backend/secrets/deployment/common.py index 4c9d52d..e15da4f 100644 --- a/services/backend/hotpocket_backend/secrets/deployment/common.py +++ b/services/backend/hotpocket_backend/secrets/deployment/common.py @@ -17,10 +17,10 @@ class DeploymentDatabaseSecrets(DatabaseSecrets): payload: str = LiteralField.new( json.dumps({ 'engine': os.getenv('HOTPOCKET_BACKEND_DATABASE_ENGINE', 'django.db.backends.postgresql'), - 'name': os.getenv('HOTPOCKET_BACKEND_DATABASE_NAME', 'hotpocket'), - 'user': os.getenv('HOTPOCKET_BACKEND_DATABASE_USER', 'hotpocket'), - 'password': os.getenv('HOTPOCKET_BACKEND_DATABASE_PASSWORD', 'hotpocketm4st3r'), - 'host': os.getenv('HOTPOCKET_BACKEND_DATABASE_HOST', 'postgres.hotpocket.work.bthlabs.net'), + 'name': os.getenv('HOTPOCKET_BACKEND_DATABASE_NAME', ''), + 'user': os.getenv('HOTPOCKET_BACKEND_DATABASE_USER', ''), + 'password': os.getenv('HOTPOCKET_BACKEND_DATABASE_PASSWORD', ''), + 'host': os.getenv('HOTPOCKET_BACKEND_DATABASE_HOST', ''), 'port': os.getenv('HOTPOCKET_BACKEND_DATABASE_PORT', '5432'), }), )