docker: all dependencies are reinstalled on each run

[VallariAg]

current behaviour: all dependencies of teuthology-api (configured in setup.cfg) are reinstalled everytime docker container is started

expected behaviour: install dependencies before copying files (so changes in file won't require re-intalling dependencies) or cache

[dikwickley]

hey @VallariAg I would like to work on this.

[VallariAg]

@dikwickley sure, go ahead!

[dikwickley]

I tried to reproduce this but can't seem to do it.

aniket@air teuthology-api % docker build -t t-api .

[+] Building 314.3s (11/11) FINISHED                                                                                           docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                           0.0s
 => => transferring dockerfile: 530B                                                                                                           0.0s
 => [internal] load .dockerignore                                                                                                              0.0s
 => => transferring context: 2B                                                                                                                0.0s
 => [internal] load metadata for docker.io/library/ubuntu:focal                                                                                3.8s
 => [1/6] FROM docker.io/library/ubuntu:focal@sha256:ed4a42283d9943135ed87d4ee34e542f7f5ad9ecf2f244870e23122f703f91c2                         10.6s
 => => resolve docker.io/library/ubuntu:focal@sha256:ed4a42283d9943135ed87d4ee34e542f7f5ad9ecf2f244870e23122f703f91c2                          0.0s
 => => sha256:ed4a42283d9943135ed87d4ee34e542f7f5ad9ecf2f244870e23122f703f91c2 1.13kB / 1.13kB                                                 0.0s
 => => sha256:a80d11b67ef30474bcccab048020ee25aee659c4caaca70794867deba5d392b6 424B / 424B                                                     0.0s
 => => sha256:0341906bdafc976cd73b05ea0e3df2e4884c6b6816197a2ffbd2367061c19acf 2.32kB / 2.32kB                                                 0.0s
 => => sha256:915eebb74587f0e5d3919cb77720c143be9a85a8d2d5cd44675d84c8c3a2b74a 25.97MB / 25.97MB                                               9.8s
 => => extracting sha256:915eebb74587f0e5d3919cb77720c143be9a85a8d2d5cd44675d84c8c3a2b74a                                                      0.6s
 => [internal] load build context                                                                                                              6.9s
 => => transferring context: 511.35MB                                                                                                          6.7s
 => [2/6] RUN apt-get update &&     apt-get install -y     git     qemu-utils     python3-dev     libssl-dev     ipmitool     python3-pip     83.7s
 => [3/6] COPY .teuthology.yaml /root                                                                                                          0.0s
 => [4/6] WORKDIR /teuthology_api                                                                                                              0.0s 
 => [5/6] COPY . /teuthology_api/                                                                                                              4.1s 
 => [6/6] RUN pip3 install -e .                                                                                                              202.6s 
 => exporting to image                                                                                                                         9.4s 
 => => exporting layers                                                                                                                        9.4s 
 => => writing image sha256:6ea4f5746ffc7e4b939f129f62e65c3d4ff9e8df485dfac00bd0482e4782a616                                                   0.0s 
 => => naming to docker.io/library/t-api                                                                                                       0.0s

What's Next?
  View a summary of image vulnerabilities and recommendations → docker scout quickview
aniket@air teuthology-api % docker images ls
REPOSITORY   TAG       IMAGE ID   CREATED   SIZE
aniket@air teuthology-api % docker image ls 
REPOSITORY    TAG       IMAGE ID       CREATED          SIZE
t-api         latest    6ea4f5746ffc   20 seconds ago   1.71GB
hello-world   latest    b038788ddb22   6 months ago     9.14kB
aniket@air teuthology-api % docker run -it 6ea
+ trap exit TERM
+ HOST=0.0.0.0
+ PORT=8080
+ cd /teuthology_api/src/
+ [  = development ]
+ gunicorn -c /teuthology_api/gunicorn_config.py teuthology_api.main:app
[2023-11-08 13:03:01 +0000] [8] [INFO] Starting gunicorn 21.2.0
[2023-11-08 13:03:01 +0000] [8] [INFO] Listening at: http://0.0.0.0:8080 (8)
[2023-11-08 13:03:01 +0000] [8] [INFO] Using worker: uvicorn.workers.UvicornWorker
[2023-11-08 13:03:02 +0000] [10] [INFO] Booting worker with pid: 10
[2023-11-08 13:03:02 +0000] [11] [INFO] Booting worker with pid: 11
[2023-11-08 13:03:02 +0000] [12] [INFO] Booting worker with pid: 12
[2023-11-08 13:03:02 +0000] [13] [INFO] Booting worker with pid: 13
[2023-11-08 13:03:02 +0000] [14] [INFO] Booting worker with pid: 14
[2023-11-08 13:03:02 +0000] [15] [INFO] Booting worker with pid: 15
[2023-11-08 13:03:02 +0000] [16] [INFO] Booting worker with pid: 16
[2023-11-08 13:03:02 +0000] [17] [INFO] Booting worker with pid: 17
[2023-11-08 13:03:04 +0000] [13] [INFO] Started server process [13]
[2023-11-08 13:03:04 +0000] [13] [INFO] Waiting for application startup.
[2023-11-08 13:03:04 +0000] [16] [INFO] Started server process [16]
[2023-11-08 13:03:04 +0000] [12] [INFO] Started server process [12]
[2023-11-08 13:03:04 +0000] [12] [INFO] Waiting for application startup.
[2023-11-08 13:03:04 +0000] [16] [INFO] Waiting for application startup.
[2023-11-08 13:03:04 +0000] [14] [INFO] Started server process [14]
[2023-11-08 13:03:04 +0000] [13] [INFO] Application startup complete.
[2023-11-08 13:03:04 +0000] [14] [INFO] Waiting for application startup.
[2023-11-08 13:03:04 +0000] [12] [INFO] Application startup complete.
[2023-11-08 13:03:04 +0000] [16] [INFO] Application startup complete.
[2023-11-08 13:03:04 +0000] [14] [INFO] Application startup complete.
[2023-11-08 13:03:04 +0000] [11] [INFO] Started server process [11]
[2023-11-08 13:03:04 +0000] [11] [INFO] Waiting for application startup.
[2023-11-08 13:03:04 +0000] [15] [INFO] Started server process [15]
[2023-11-08 13:03:04 +0000] [15] [INFO] Waiting for application startup.
[2023-11-08 13:03:04 +0000] [10] [INFO] Started server process [10]
[2023-11-08 13:03:04 +0000] [11] [INFO] Application startup complete.
[2023-11-08 13:03:04 +0000] [10] [INFO] Waiting for application startup.
[2023-11-08 13:03:04 +0000] [15] [INFO] Application startup complete.
[2023-11-08 13:03:04 +0000] [10] [INFO] Application startup complete.
[2023-11-08 13:03:04 +0000] [17] [INFO] Started server process [17]
[2023-11-08 13:03:04 +0000] [17] [INFO] Waiting for application startup.
[2023-11-08 13:03:04 +0000] [17] [INFO] Application startup complete.

As you can see, the dependencies were installed during the docker build step.
After the image is built, running the container in interactive mode does not reinstall the dependencies.

Were you referring to caching the dependencies during the build step? or am i missing something while trying to reproduce this.

[VallariAg]

Yes, I am referring to caching them during the build step. Any changes in the root directory of teuthology-api will lead to reinstalling of all dependencies in the next build.
This might be helpful: https://docs.docker.com/build/cache/

[dikwickley]

Alright got it.

[Devansh3712]

I searched about caching pip dependencies so it only re-installs if there is a change in requirements.txt, this might be useful in our use case - https://stackoverflow.com/a/57282479