Skip to content
GitLab
Projects
Groups
Snippets
Help
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Open sidebar
ESCAPE2020
WP3
ZenodoCI
Commits
b8bc2ad7
Commit
b8bc2ad7
authored
May 25, 2021
by
Enrique Garcia
Browse files
Merge branch 'new_release' into 'master'
new_release See merge request
!21
parents
fb53d8cd
655be2fa
Pipeline
#120644
passed with stages
in 3 minutes and 23 seconds
Changes
3
Pipelines
2
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
46 additions
and
79 deletions
+46
-79
.gitlab-ci.yml
.gitlab-ci.yml
+39
-75
README.md
README.md
+4
-1
codemeta.json
codemeta.json
+3
-3
No files found.
.gitlab-ci.yml
View file @
b8bc2ad7
...
...
@@ -9,8 +9,8 @@ cache:
stages
:
-
install
-
test
-
build_docker
-
deploy
-
test_container
.virtualenv_template
:
&virtualenv_definition
|
python -V
...
...
@@ -52,88 +52,35 @@ test_py36:
only
:
-
branches
deploy_zenodo
:
stage
:
deploy
image
:
python:3.6-buster
#dependencies:
### Ideally to be used within a ci pipeline in where a container of the source code is build in a previous stage.
### You can have a look into https://gitlab.in2p3.fr/escape2020/wp3/template_project_escape/-/blob/master/.gitlab-ci.yml
# - build_image
build_docker
:
stage
:
build_docker
image
:
docker:19.03.12
services
:
-
docker:19.03.12-dind
before_script
:
### 1 - Install dependencies in the image and upload the files to Zenodo
-
apt-get -y update
### INFORMATION FOR THE USER; Python, pip and wget are already installed in the container
#- cat /etc/os-release # Debian GNU/Linux 10 (buster)
#- pip3 --version # pip 20.1.1
#- python3 --version # 3.6 as expected
-
pip install requests numpy
-
pip install .
# Install ZenodoCI
-
which parse_last_release_git.sh
### 2 - Test the communication to Zenodo - Uncomment if needed, or if already done it manually.
-
test_connection_zenodo --token $SANDBOX_ZENODO_TOKEN --sandbox True -p $CI_PROJECT_DIR
#- test_connection_zenodo -t $ZENODO_TOKEN -s False --project_dir $CI_PROJECT_DIR
-
cat /etc/os-release
# "Alpine Linux v3.12"
-
apk add git
-
export LAST_RELEASE=`git ls-remote --tags --refs --sort="v:refname" $CI_PROJECT_URL.git | tail -n1 | sed 's/.*\///'`
-
echo $LAST_RELEASE
script
:
### 3 - Get the last tag/release of the repository,
### 4 - Download the repository and move it to the build directory
### If no release is found/correctly parsed, the script will download the last commit pushed to the master branch
-
mkdir -p build
-
parse_last_release_git.sh $CI_PROJECT_NAME $CI_PROJECT_URL
-
ls ./build
### 5 - To deploy a NEW DEPOSIT to ZENODO SANDBOX
-
upload_new_deposit --token $SANDBOX_ZENODO_TOKEN --sandbox True --input-dir ./build
-
cd Docker
-
docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
-
docker build -t $CI_REGISTRY_IMAGE:$LAST_RELEASE .
-
docker push $CI_REGISTRY_IMAGE:$LAST_RELEASE
only
:
-
tags
### 5 - To deploy a NEW DEPOSIT to ZENODO
#- >
# upload_new_deposit
# --token $ZENODO_TOKEN
# --sandbox False
# --input-dir ./build
### 5 - To deploy a NEW VERSION to ZENODO: The deposit_id of the entry to be `new_versioned` MUST be provided.
-
>
upload_new_version_deposit
--token $SANDBOX_ZENODO_TOKEN
--sandbox True
--input-dir ./build
--deposit_id $ZENODO_PROJECT_ID
# --token $ZENODO_TOKEN
# --sandbox False
only
:
### Ideally this stage should be run only when a new release / tag of the source code is created, i.e., (- tags).
# The script is changed to check that the both `upload_new_deposit` and `upload_new_version_deposit` works nicely.
-
master
# For testing
#deploy_from_container:
# stage: deploy
# image: gitlab-registry.in2p3.fr/escape2020/wp3/zenodoci:v1.1
# before_script:
# - test_connection_zenodo --token $ZENODO_TOKEN --sandbox False -p $CI_PROJECT_DIR
# script:
# - mkdir -p build
# - parse_last_release_git.sh $CI_PROJECT_NAME $CI_PROJECT_URL
# - ls ./build
#
# - upload_new_deposit --token $ZENODO_TOKEN --sandbox False --input-dir ./build
# only:
# - tags
test_deploy_from_container
:
stage
:
test_container
deploy_sandbox
:
stage
:
deploy
image
:
gitlab-registry.in2p3.fr/escape2020/wp3/zenodoci:v1.1
before_script
:
-
which parse_last_release_git.sh
-
test_connection_zenodo --token $SANDBOX_ZENODO_TOKEN --sandbox True -p $CI_PROJECT_DIR
script
:
-
pwd
# testing
/debugging
-
ls
# testing
/debugging
-
env
# testing
/debugging
-
pwd
# testing
-
ls
# testing
-
env
# testing
-
upload_new_deposit -h
-
upload_new_version_deposit -h
...
...
@@ -141,9 +88,26 @@ test_deploy_from_container:
-
mkdir -p build
-
parse_last_release_git.sh $CI_PROJECT_NAME $CI_PROJECT_URL
-
if [[ -f ./codemeta.json ]]; then cp ./codemeta.json ./build; fi
-
ls ./build
-
upload_new_deposit --token $SANDBOX_ZENODO_TOKEN --sandbox True --input-dir ./build
-
upload_new_version_deposit -t $SANDBOX_ZENODO_TOKEN -s True -i ./build -id $ZENODO_PROJECT_ID
except
:
-
master
# run only in branches
deploy_zenodo
:
stage
:
deploy
image
:
gitlab-registry.in2p3.fr/escape2020/wp3/zenodoci:v1.2
before_script
:
-
test_connection_zenodo --token $ZENODO_TOKEN --sandbox False -p $CI_PROJECT_DIR
script
:
-
mkdir -p build
-
parse_last_release_git.sh $CI_PROJECT_NAME $CI_PROJECT_URL
-
if [[ -f ./codemeta.json ]]; then cp ./codemeta.json ./build; fi
-
ls ./build
-
upload_new_deposit --token $ZENODO_TOKEN --sandbox False --input-dir ./build
only
:
-
master
# testing/debugging
-
tags
README.md
View file @
b8bc2ad7
...
...
@@ -27,6 +27,8 @@ Then you would be able to run any of the following;
$
test_connection_zenodo
-t
$ZENODO_TOKEN
-s
False
--repo_dir
<PROJ_DIR>
```
-----------
Please note that all the code developed in this project, together with the
[
codemeta2zenodo crosswalk
](
https://gitlab.in2p3.fr/escape2020/wp3/codemeta2zenodo
)
are included and installed into a Docker container that can be
found at
[
the container registry
](
https://gitlab.in2p3.fr/escape2020/wp3/zenodoci/container_registry
)
of this project.
...
...
@@ -55,12 +57,13 @@ stages:
deploy_to_zenodo:
stage: deploy
image: gitlab-registry.in2p3.fr/escape2020/wp3/zenodoci:v1.
1
image: gitlab-registry.in2p3.fr/escape2020/wp3/zenodoci:v1.
2
before_script:
- test_connection_zenodo --token $ZENODO_TOKEN --sandbox False -p $CI_PROJECT_DIR
script:
- mkdir -p build
- parse_last_release_git.sh $CI_PROJECT_NAME $CI_PROJECT_URL
- if [[ -f ./codemeta.json ]]; then cp ./codemeta.json ./build; fi
- ls ./build
- upload_new_deposit --token $SANDBOX_ZENODO_TOKEN --sandbox True --input-dir ./build
...
...
codemeta.json
View file @
b8bc2ad7
...
...
@@ -3,9 +3,9 @@
"@type"
:
"SoftwareSourceCode"
,
"name"
:
"ZenodoCI"
,
"description"
:
"The library is intended to be part of a complete CI pipeline. This stage deploys to Zenodo the files found in the ./build directory, configured in the .gitlab-ci.yml file"
,
"keywords"
:
[],
"keywords"
:
[
"ESCAPE"
],
"license"
:
"https://spdx.org/licenses/MIT"
,
"softwareVersion"
:
"1.
0
"
,
"softwareVersion"
:
"1.
2
"
,
"developmentStatus"
:
"active"
,
"codeRepository"
:
"https://gitlab.in2p3.fr/escape2020/wp3/zenodoci"
,
"runtimePlatform"
:
"Python >3.6"
,
...
...
@@ -14,7 +14,7 @@
"releaseNotes"
:
"Deploy stage (into the Zenodo repository) for a CI pipeline"
,
"dateCreated"
:
"2020-06-30"
,
"datePublished"
:
"2020-06-30"
,
"dateModified"
:
"202
0-10
-2
4
"
,
"dateModified"
:
"202
1-05
-2
5
"
,
"isAccessibleForFree"
:
true
,
"isPartOf"
:
[
"https://gitlab.in2p3.fr/escape2020"
,
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment