Problem Description

Currently, all our application Dockerfiles are based off a public barebones base image FROM --platform=linux/amd64 python:3.11.5-slim AS base.

Each time we start our applications locally i.e. docker-compose --profile infra --profile api up, we're relying on the local Docker image cache, which will be invalidated for any small changes to the application, even if it's just a change to just python application files, causing the images to have to be built from scratch before the application can start.

Building the images from scratch takes ~12-18 mins, esp for the api image, because we're doing a bunch of local system installations like RUN apt install -y ocrmypdf.

We should manually push images for our applications - excluding the code/build folders, so the dev images can be built much more quickly - i.e. no need to re-install all the apt packages from scratch because those are not changing.

Ideally, there should be a script that we can run to manually build base images for each application and push to Amazon ECR with a version number (e.g. api-base-v0.1.1), and in the Docker Compose file, refer to the latest current version of each application's base image.

Acceptance Criteria

We'll share access to relevant parts of our codebase, and you'll prepare PRs that reduce our API server startup time from >15 min to less than 5min. This should work not just in Github Actions in CI/CD, but also locally when folks run docker compose up.

For context, we already spend 10+ minutes building the ocrmypdf image. Simply caching this should enable us to hit the goal already.

Technical Details

Deliverables we expect are:

1. A script to manually build images and push them up to Amazon ECR.
2. Updated Dockerfiles that check Amazon ECR and pull images.

Link to Project

Will provide more details over email.