Download the latest artifacts of failed gitlab pipeline

Nikita Meier

I want to automatically load test results of my gitlab pipeline (basically one xml file) into my project management application. No direct connection between these two is possible. However the gitlab API offers the following options to download artifacts of a pipeline:

  • all artifacts of the latest successful pipeline run (selected by job name)

GET /projects/:id/jobs/artifacts/:ref_name/download?job=name`

  • all artifacts of a specific job (selected by job id)

GET /projects/:id/jobs/:job_id/artifacts

  • a single artifact file (selected by job id)

GET /projects/:id/jobs/:job_id/artifacts/*artifact_path

My current situation is following:

I have test reports which are saved inside the job artifacts when running the pipeline. The artifacts are created on every run of the pipeline independent of its outcome

gitlab-ci.yaml

...
artifacts:

     when: always
...

The artifact I am trying to download has a dynamic name

./reports/junit/test-results-${CI_JOB_ID}.xml

If I now want to download the latest test results to a different server than the gitlab server, I have to realize that I don't know the latest job ID, which means:

  • I can't access the artifact directly because it has a dynamic name
  • I can't access the artifacts of a specific job
  • I can access the artifacts of the latest job, but only if it was successful

This leaves me with the situation, that i only get to download the latest test results, if nothing went wrong while testing. To put it mildly, this is suboptimal.

Is there some way to download the artifacts from the latest job run (without knowing the job ID), independent of its outcome?

Tolis Gerodimos

Is there some way to download the artifacts from the latest job run (without knowing the job ID), independent of its outcome?

In order to achieve this we will use the Gitlab API in combination with the jq package.

Let's break down this question into components.

Firstly, we need to find out the id of the last executed pipeline for this project. https://docs.gitlab.com/ee/api/pipelines.html#list-project-pipelines

GET /projects/:id/pipelines

For this call you will need your access token, if you don't have one already check https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html#create-a-personal-access-token https://docs.gitlab.com/ee/user/project/settings/project_access_tokens.html#create-a-project-access-token

You will also need the Project ID

enter image description here

LAST_PIPELINE_ID=$(curl -s --header "PRIVATE-TOKEN: <access_token>" https://gitlab.com/api/v4/projects/<project_id>/pipelines | jq '.[0].id')

Next we will retrieve the job id by providing the job name by using the following API https://docs.gitlab.com/ee/api/jobs.html#list-pipeline-jobs

GET /projects/:id/pipelines/:pipeline_id/jobs

In your case you need to change the following example with your job's name, in this example let's call it my_job

JOB_ID=$(curl -s --header "PRIVATE-TOKEN: <access_token>" https://gitlab.com/api/v4/projects/<project_id>/pipelines/$LAST_PIPELINE_ID/jobs | jq '.[] | select(.name=="my_job") | .id')

Now we are ready to actually retrieve the artifacts, with the following API

GET /projects/:id/jobs/:job_id/artifacts

https://docs.gitlab.com/ee/api/job_artifacts.html#get-job-artifacts

wget -U "Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.17 (KHTML,like Gecko) Ubuntu/11.04 Chromium/11.0.654.0 Chrome/11.0.654.0 Safari/534.17" --header "PRIVATE-TOKEN: <access_token>" "https://gitlab.com/api/v4/projects/<project_id>/jobs/$JOB_ID/artifacts" -O artifacts.zip

The artifacts are available as artifacts.zip in the folder you executed wget from

Combining them here for clarity

LAST_PIPELINE_ID=$(curl -s --header "PRIVATE-TOKEN: <access_token>" https://gitlab.com/api/v4/projects/<project_id>/pipelines | jq '.[0].id')
JOB_ID=$(curl -s --header "PRIVATE-TOKEN: <access_token>" https://gitlab.com/api/v4/projects/<project_id>/pipelines/$LAST_PIPELINE_ID/jobs | jq '.[] | select(.name=="my_job") | .id')
wget -U "Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.17 (KHTML,like Gecko) Ubuntu/11.04 Chromium/11.0.654.0 Chrome/11.0.654.0 Safari/534.17" --header "PRIVATE-TOKEN: <access_token>" "https://gitlab.com/api/v4/projects/<project_id>/jobs/$JOB_ID/artifacts" -O artifacts.zip

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

Artifacts between pipeline tasks in Gitlab

gitlab: how can i programatically download the artifacts issued at end of CI pipeline

"Download pipeline artifact" vs. "Download build artifacts"

How to construct urls for job names for downloading latest artifacts in GitLab CI?

Gitlab pipeline test stage to fail AND create artifacts anyway

How to download the latest build artifacts from Azure DevOps programmatically?

GitLab CI-Build Artifacts command line download

Download only artifacts I need in release pipeline and run tasks conditionnally

How to publish and download artifacts from multiple jobs into one location on pipeline?

Different artifacts when job succeeded and failed with GitLab CI

gitlab ci pipeline failed deploy ftp

Make GitLab pipeline failed using Jelastic API

Failed to execute goal in GitLab CI Pipeline

failed to connect to gitlab port 443 in pipeline

How to deploy latest nuget package (AzureDevOps Artifacts) from yml pipeline on a scheduled trigger

gitlab-runner download and store cache file in each job of pipeline

Download Build Artifacts in vsts not able to get latest build of a specific build definition

GitLab AutoDevops with Oracle artifacts

Gitlab: missing pages artifacts

Gitlab ci artifacts not found

Date Unit test passed locally but failed in Gitlab CD/CI pipeline

build failed in gitlab.com using default pipeline

Artifacts are not pulled in a child pipeline

Unable to build latest commit that pushed by pipeline during build using gitlab pipeline

Download some of the artifacts files

Gitlab pipeline failed : ERROR: Preparation failed: Error response from daemon: toomanyrequests

Gitlab CI Angular Artifacts not accessible

Get artifacts of included gitlab template

Upload Gitlab CI artifacts to a storage outside Gitlab