-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Setup Gitlab pipeline and provisioning on new Upstream project to enable two production infrastructures #1996
Open
rija
wants to merge
41
commits into
gigascience:develop
Choose a base branch
from
rija:alt-upstream
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rija
force-pushed
the
alt-upstream
branch
2 times, most recently
from
August 9, 2024 09:27
b6c838f
to
c6f56a5
Compare
This was referenced Aug 13, 2024
This was referenced Aug 20, 2024
rija
changed the title
Enabling of Gitlab pipeline and provisioning on Upstream for blue/green deployment
Setup Gitlab pipeline and provisioning on two Upstream projects to enable two production infrastructures
Aug 20, 2024
rija
changed the title
Setup Gitlab pipeline and provisioning on two Upstream projects to enable two production infrastructures
Setup Gitlab pipeline and provisioning on new Upstream project to enable two production infrastructures
Aug 20, 2024
pli888
reviewed
Sep 2, 2024
pli888
reviewed
Sep 2, 2024
pli888
reviewed
Sep 3, 2024
pli888
reviewed
Sep 3, 2024
pli888
reviewed
Sep 3, 2024
pli888
reviewed
Sep 3, 2024
pli888
reviewed
Sep 4, 2024
pli888
reviewed
Sep 4, 2024
pli888
reviewed
Sep 4, 2024
kencho51
approved these changes
Oct 9, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull request for issue:
This is a pull request for the following functionalities:
emacs
andwget
tmux
*.tar.bz2
archives by installingbzip2
andlbzip2
How to test?
Deployment to your AWS staging environment
After checking out this PR's branch, pushing it to your Gitlab pipeline,
you can follow the "Setting up your Staging environment" section of the updated
docs/SETUP_PROVISIONING.md
document.Deploying to hot standby from Upstream's alt-gigadb-website pipeline
The hot standby infrastructure is already built, but you can test re-provisioning and deploying to it:
Follow the instructions specific to the
upstream/alt-gigadb-website
project indocs/sop/PROVISIONING_PRODUCTION.md
for the staging environment.Then follow the instructions in
docs/RELEASE_PROCESS.md
to create a fake release (choose a tag label that's obviously fake e.g:v00-rija-testing
), that's going to be deployed live only to theupstream/alt-gigadb-website
project, and not to our current production.Once that release has been deployed to staging, you can resume the instructions in
docs/sop/PROVISIONING_PRODUCTION.md
from the "Provisioning live for upstream/alt-gigadb-website" section.When guided to the blue/green deployment process, consider your fake release as a simple one (no infrastructure change and no database change): You can deploy to the Hot stand-by (currently the
upstream/alt-gigadb-website
) by following the instructions in "Deployment to a specific live environment" section fromdocs/sop/PROVISIONING_PRODUCTION.md
If everything goes well, you should be able to play with the new infrastructure:
When both pipelines are successful, navigate to the staging urls (check versions in footer, should match your fake release)
https://staging.gigadb.org
https://alt-staging.gigadb.org
Test you can connect to the bastion server for both staging environments with the centos user using the SSH key from the first two steps:
centos@bastion-stg.gigadb.host
centos@bastion.alt-staging.gigadb.host
Test the deployment to live on Hot stand-by
https://alt-live.gigadb.org
and check version in footercentos@bastion.alt-live.gigadb.host
users_playbook.yml
for the exact same username you already have onupstream/gigadb-website
, you should then notice that you can ssh with that user using the same private key (because the public key is already in Gitlab variables)bastion.alt-live.gigadb.host
, you can access/share/dropbox
, and its content should be the same as what's onupstream/gigadb-website
's EFS.centos
user, execcrontab -l
and notice that:upstream/gigadb-website
blue/green deployment switchover
The last part of
docs/sop/DEPLOYING_TO_PRODUCTION.md
described the proposed plan for doing the blue/green deployment. Feel free to comment.Changes to composer.json
composer.json
is now a regular file, that's versioned and manually editable, so that automated dependencies security checks can be performed.After checking out this branch, you should be able to execute
./up.sh
as usual and everything should work as usual.Addition of more basic tools
The new
ops/infrastructure/data_cliapp_playbook.yml
playbook will install:On your AWS deployment of this branch (from the first section of the "how to test?") , you can check they work by executing the commands below in order on a bastion server:
C-x C-c
to exit emacsC-d
to exit tmuxC-d
to log off SSHHow have functionalities been implemented?
Blue/green deployment:
See the "Upstream projects" section of
docs/sop/DEPLOYING_TO_PRODUCTION.md
Fixing the circular dependency issue:
Move all the bastion playbooks tasks that depend on the building and pulling of docker containers by the Gitlab pipeline into a new playbook
ops/infrastructure/data_cliapp_playbook.yml
which, unlike the other host configuration playbooks, is to be executed after running the Gitlab pipeline.Any issues with implementation?
N/a
Any changes to automated tests?
N/a
Any changes to documentation?
docs/sop/AWS_SETUP.md
is replaced and augmented withdocs/sop/DEPLOYING_TO_PRODUCTION.md
docs/SETUP_PROVISIONING.md
is updated to take into account the breaking down of the bastion playbook into two distinct playbookdocs/RELEASE_PROCESS.md
was moved todocs/sop/RELEASE_PROCESS.md
and updated to reflect current practice and blue/green deployment changesdocs/sop/PRODUCTION_DEPLOY.md
was renameddocs/sop/PROVISIONING_PRODUCTION.md
and updated to integrate two parallel infrastructures and blue/green deployment approachAny technical debt repayment?
N/a
Any improvements to CI/CD pipeline?
The
ops/infrastructure/bastion_playbook.yml
was broken up for two reasons:ops/infrastructure/data_cliapp_playbook.yml
docs/SETUP_PROVISIONING.md
can be performed in order without ambiguity and clear boundaries.