Hendrik/xgboost
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[CI] Optimize external Docker build cache (#4334)

Browse Source 
* When building pull requests, use Docker cache for master branch

Docker build caches are per-branch, so new pull requests will initially
have no build cache, causing the Docker containers to be built from
scratch. New pull requests should use the cache associated with the
master branch. This makes sense, since most pull requests do not modify
the Dockerfile.

* Add comments
This commit is contained in:
Philip Hyunsu Cho 2019-04-04 15:59:07 -07:00 committed by GitHub
parent 37c75aac41
commit 70be1e38c2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 16 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
tests/ci_build/ci_build.sh
View File

@ -122,12 +122,22 @@ echo "Building container (${DOCKER_IMG_NAME})..."
# If enviornment variable DOCKER_CACHE_REPO is set, use an external Docker repo for build caching # If enviornment variable DOCKER_CACHE_REPO is set, use an external Docker repo for build caching
if [[ -n "${DOCKER_CACHE_REPO}" ]] if [[ -n "${DOCKER_CACHE_REPO}" ]]
then then
# Login for Docker registiry # Login for Docker registry
echo '$(python3 -m awscli ecr get-login --no-include-email --region us-west-2)' echo '$(python3 -m awscli ecr get-login --no-include-email --region us-west-2)'
$(python3 -m awscli ecr get-login --no-include-email --region us-west-2) $(python3 -m awscli ecr get-login --no-include-email --region us-west-2)
echo "docker pull ${DOCKER_CACHE_REPO}/${DOCKER_IMG_NAME}:${BRANCH_NAME} || true" # Pull pre-build container from Docker build cache,
docker pull "${DOCKER_CACHE_REPO}/${DOCKER_IMG_NAME}:${BRANCH_NAME}" || true # if one exists for the particular branch or pull request
CACHE_FROM_CMD="--cache-from ${DOCKER_CACHE_REPO}/${DOCKER_IMG_NAME}:${BRANCH_NAME}" echo "docker pull ${DOCKER_CACHE_REPO}/${DOCKER_IMG_NAME}:${BRANCH_NAME}"
if docker pull "${DOCKER_CACHE_REPO}/${DOCKER_IMG_NAME}:${BRANCH_NAME}"
then
CACHE_FROM_CMD="--cache-from ${DOCKER_CACHE_REPO}/${DOCKER_IMG_NAME}:${BRANCH_NAME}"
else
# If the build cache is empty of the particular branch or pull request,
# use the build cache associated with the master branch
echo "docker pull ${DOCKER_CACHE_REPO}/${DOCKER_IMG_NAME}:master"
docker pull "${DOCKER_CACHE_REPO}/${DOCKER_IMG_NAME}:master" || true
CACHE_FROM_CMD="--cache-from ${DOCKER_CACHE_REPO}/${DOCKER_IMG_NAME}:master"
fi
else else
CACHE_FROM_CMD='' CACHE_FROM_CMD=''
fi fi
@ -152,6 +162,8 @@ fi
# If enviornment variable DOCKER_CACHE_REPO is set, use an external Docker repo for build caching # If enviornment variable DOCKER_CACHE_REPO is set, use an external Docker repo for build caching
if [[ -n "${DOCKER_CACHE_REPO}" ]] if [[ -n "${DOCKER_CACHE_REPO}" ]]
then then
# Push the container we just built to the Docker build cache
# that is associated with the particular branch or pull request
echo "docker tag ${DOCKER_IMG_NAME} ${DOCKER_CACHE_REPO}/${DOCKER_IMG_NAME}:${BRANCH_NAME}" echo "docker tag ${DOCKER_IMG_NAME} ${DOCKER_CACHE_REPO}/${DOCKER_IMG_NAME}:${BRANCH_NAME}"
docker tag "${DOCKER_IMG_NAME}" "${DOCKER_CACHE_REPO}/${DOCKER_IMG_NAME}:${BRANCH_NAME}" docker tag "${DOCKER_IMG_NAME}" "${DOCKER_CACHE_REPO}/${DOCKER_IMG_NAME}:${BRANCH_NAME}"
echo "docker push ${DOCKER_CACHE_REPO}/${DOCKER_IMG_NAME}:${BRANCH_NAME}" echo "docker push ${DOCKER_CACHE_REPO}/${DOCKER_IMG_NAME}:${BRANCH_NAME}"