Skip to content

Commit

Permalink
Add docker-compose.dev.yml and dev/test scripts
Browse files Browse the repository at this point in the history
  • Loading branch information
wil93 committed Oct 5, 2024
1 parent f0d969b commit 11723de
Show file tree
Hide file tree
Showing 7 changed files with 137 additions and 21 deletions.
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@ Dockerfile
docker-compose*.yml
.vagrant/
codecov/
.dev/
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,4 @@ lib/
log/
.vagrant/
codecov/
.dev/
2 changes: 2 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ RUN sudo python3 prerequisites.py --yes --cmsuser=cmsuser install

RUN sudo sed 's|/cmsuser:your_password_here@localhost:5432/cmsdb"|/postgres@testdb:5432/cmsdbfortesting"|' ./config/cms.conf.sample \
| sudo tee /usr/local/etc/cms-testdb.conf
RUN sudo sed 's|/cmsuser:your_password_here@localhost:5432/cmsdb"|/postgres@devdb:5432/cmsdb"|' ./config/cms.conf.sample \
| sudo tee /usr/local/etc/cms-devdb.conf

ENV LANG C.UTF-8

Expand Down
6 changes: 6 additions & 0 deletions cms-dev.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -x

GIT_BRANCH_NAME=$(git rev-parse --abbrev-ref HEAD)

docker compose -p $GIT_BRANCH_NAME -f docker-compose.dev.yml run --build --rm --service-ports devcms
6 changes: 6 additions & 0 deletions cms-test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -x

GIT_BRANCH_NAME=$(git rev-parse --abbrev-ref HEAD)

docker compose -p $GIT_BRANCH_NAME -f docker-compose.test.yml run --build --rm testcms
25 changes: 25 additions & 0 deletions docker-compose.dev.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
services:
devdb:
image: postgres
environment:
POSTGRES_HOST_AUTH_METHOD: trust
volumes:
- "./.dev/postgres-data:/var/lib/postgresql/data"

devcms:
build: .
depends_on:
- "devdb"
environment:
CMS_CONFIG: /usr/local/etc/cms-devdb.conf
volumes:
- "./.dev/home:/home/cmsuser"
- ".:/home/cmsuser/cms"
privileged: true
cgroup: host
command: >
wait-for-it devdb:5432 -- bash
ports:
- 8888:8888
- 8889:8889
- 8890:8890
117 changes: 96 additions & 21 deletions docs/Docker image.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,19 @@ Docker image

We provide a docker image (defined in :gh_blob:`Dockerfile`) that can be used to
easily get an instance of CMS running locally with all the necessary
dependencies. We also provide a :gh_blob:`docker-compose.test.yml` files that
uses said docker image to run the tests.
dependencies. We also provide:

* :gh_blob:`cms-test.sh`: This file uses :gh_blob:`docker-compose.test.yml` to
spawn a volatile database (not persisted on disk) as well as a CMS instance
that automatically runs all tests.

* :gh_blob:`cms-dev.sh`: This file uses :gh_blob:`docker-compose.dev.yml` to
spawn a database (persisted in the local ``.dev/postgres-data`` folder
within the repository) as well as a CMS instance that only runs `bash`,
leaving you with a shell from where you can start cms services and servers.
The DB port and CMS server ports are also automatically forwarded on the
host machine (respectively to ``15432`` for the database, and ``8888-8890``
for CMS).

Make sure that you have a recent version of Docker installed, as well as Docker
Compose.
Expand All @@ -14,45 +25,109 @@ Compose.
Running tests
=============

First you need to build the docker image, then you use it to run the tests.
You can simply run:

.. sourcecode:: bash

./cms-test.sh

Or, you can issue the full command (that is defined in ``cms-test.sh``) which
is similar to:

.. sourcecode:: bash

docker compose -p someprojectname -f docker-compose.test.yml run --build --rm testcms

.. note::

Some versions of docker require to specify ``-p``, some version will fill it
for you based on the current folder's name (which for us would be equivalent
to passing ``-p cms``).

The ``-p`` flag is used as a namespace for the containers that will be
created. When you're running tests on a separate branch, it can be useful to
include the branch name there, to avoid any conflict. (You can also omit the
flag and specify the name via the ``COMPOSE_PROJECT_NAME`` environment
variable.)
include the branch name there, to avoid any conflict. The ``cms-test.sh``
script uses the **name of the current git branch** and passes it to ``-p``.

Note also that if you are not part of the ``docker`` group then you'll need
to run every docker command with ``sudo``, including ``sudo ./cms-test.sh``.
We recommend adding yourself to the ``docker`` group.

What the ``cms-test.sh`` command does is: first build a fresh CMS image when
necessary, and then create (assuming you didn't specify the ``-p`` flag) a
``cms-testdb-1`` container for the database, and a
``cms-testcms-run-<random_string>`` container for CMS.

The database container **will not** be automatically deleted, while the CMS
container will be automatically deleted upon exiting (because of the ``--rm``
flag).

To delete the ``cms-testdb-1`` container after testing you can run:

.. sourcecode:: bash

If you are not part of the ``docker`` group, then you need to run every
docker command with ``sudo``.
docker rm -f cms-testdb-1

To build the image:
Developing CMS
==============

To run a local development instance of CMS, you can simply run:

.. sourcecode:: bash

docker compose -p cms -f docker-compose.test.yml build testcms
./cms-dev.sh

To run the tests:
Or, you can issue the full command (that is defined in ``cms-dev.sh``) which is
similar to:

.. sourcecode:: bash

docker compose -p cms -f docker-compose.test.yml run --rm testcms
docker compose -p someprojectname -f docker-compose.dev.yml run --build --rm --service-ports devcms

The command will build a fresh CMS image when necessary, and drop you into a
bash prompt where the repository is mounted on ``~/cms`` for ease of
development. You can edit the code from the host (i.e. outside the container)
and then reinstall CMS (``python3 setup.py install``) directly from inside the
container, without having to rebuild the image every time.

Another option is to add the ``--build`` flag to the ``run`` command, to perform
a new build before running the tests:
Upon running ``cms-dev.sh`` for the first time, the database will initially be
empty. You need to initialize it (notice that the following commands are
indicated with a ``>>>`` prompt because they are meant to be executed **inside**
the container, from the prompt that you get to after running ``cms-dev.sh``)
like so:

.. sourcecode:: bash

docker compose -p cms -f docker-compose.test.yml run --build --rm testcms
>>> createdb -h devdb -U postgres cmsdb
>>> cmsInitDB

This command will create (assuming you used ``-p cms``) a ``cms-testdb-1``
container for the database which **will not** be automatically deleted, and a
``cms-testcms-run-<random_string>`` container for CMS which will be
automatically deleted (because of the ``--rm`` flag) upon exiting.
Then you probably want to download a test contest and import it, for example
like this:

To delete the ``cms-testdb-1`` container after testing you can run:
.. sourcecode:: bash

>>> git clone https://github.com/cms-dev/con_test.git
>>> cd con_test
>>> cmsImportUser --all
>>> cmsImportContest -i .

If this succeeds, you can then run one of the servers, for example the
ContestWebServer, like so:

.. sourcecode:: bash

docker rm -f cms-testdb-1
>>> cmsContestWebServer

When it prompts you to choose a contest ID, you can simply hit Enter.

When the server is finally running, you can check (from the host machine) that
the server is reachable at http://localhost:8888/

You can also verify that upon exiting the container's bash shell and reentering
it (by running ``cms-dev.sh`` again) you won't need to re-import the contest, as
the database is persisted on disk on the host machine. Even manually destroying
and recreating the database container will retain the same data. If for some
reason you need to reset the database, we recommend using the ``dropdb -h devdb
-U postgres cmsdb`` command inside the container. To remove any trace of the
database data, you can delete the ``.dev/postgres-data`` folder within the git
repository.

0 comments on commit 11723de

Please sign in to comment.