How to Bulk Download Azure Artifacts Packages?

AMoghrabi

My organization is moving our software packages (NuGet, NPM) to Azure Artifacts. We have approximately 50,000 NuGet packages (2GB) that we've pushed to Azure Artifacts.

Is there a way to to bulk-pull my packages from Azure Artifacts either to my machine or to Azure blob storage in case we are not satisfied with the product? I'm mainly concerned about pulling our NuGet packages because we have thousands of packages published.

I'm aware of Download all packages from private nuget feed but I'd like to know if there's a way to retrieve a zip file of all of our packages, or some other method to download all of our packages in bulk.

Thanks!

Jonathan Myers

We don't have a bulk download option, but it would be pretty easy to script something yourself.

First, you'll want to get a PAT. Make sure the PAT has the Packaging (read) scope. You will use the PAT as a password for authentication. The username you use doesn't matter.

Then, you'll need to get the list of all packages in the feed. You can use our Get Packages API for that.

Then, for NuGet packages, the protocol is as follows:

  1. Download the feed URL (from the Connect to Feed dialog, will end with .../v3/index.json)
  2. Read the returned JSON document, get the @id attribute from the object with @type = PackageBaseAddress/3.0.0. This is the PackageBaseAddress. Note that the actual value is considered an implementation detail. Always retrieve the address from index.json.
  3. If you do not already have the list of versions, download {PackageBaseAddress}/{PackageName (lowercased)}/index.json which will have the list as a JSON array (I suggest getting the versions this way since they will already be normalized)
  4. Download the packages from {PackageBaseAddress}/{PackageName (lowercase)}/{PackageVersion (normalized)}/{PackageName (lowercase)}.{PackageVersion (normalized)}.nupkg
    • e.g. {PackageBaseAddress}/restsharp/106.4.1/restsharp.106.4.1.nupkg
    • Be sure to follow redirects, as the service will actually send you to download from our backend storage

For npm packages:

  1. Get the Feed URL from the Connect to Feed dialog (it will end with .../npm/registry)
  2. Download {Feed URL}/{PackageName} (for scoped packages include the scope, e.g. {Feed URL}/@{Scope}/{PackageName})
  3. Download URLs for each version are listed in the JSON response as versions.*.dist.tarball
  4. Again, be sure to handle redirects when downloading the packages.

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

How to download an artifact in Azure DevOps artifacts

How to download the latest build artifacts from Azure DevOps programmatically?

Azure DevOps Artifacts not including all packages?

Maven project is not able to download artifacts from Azure Artifacts

Direct download URL for universal package in Azure artifacts

Azure Artifacts - Download a specific version of Universal artifact

How to get the list of all the packages with their versions from Artifacts/Nuget Feed in Azure Devops using C#?

How to archive a folder as tar.bz2 file and upload to universal packages from Azure Artifacts

how can we configure to pull npm packages from Jfrog artifacts for yaml based azure CI Pipeline?

How to download artifacts built with gulp script in azure-pipelines from dist folder

How to download .torrent files in bulk?

why azure artifacts include all reference 3rd packages

Do I have to use Azure Artifacts to access private NPM packages?

Publishing packages to Azure DeveOps Artifacts using Python-Poetry

Azure Artifacts is returning 404 for some of the public npm packages

Can I upload existing NuGet packages to an Azure DevOps artifacts feed?

Get list of artifacts / packages from azure-devops-extension-api

Azure Devops Artifacts: disable saving packages from upstream sources

Shell How install local bulk of packages

Azure Artifacts to download only specific files required for deployment on build server

Azure DevOps fails to download artifacts from its own repository

No option to copy artifacts download URL in new Azure DevOps Builds

How to manually publish a file to Azure Artifacts?

How build artifacts are managed in Azure DevOps pipeline?

How can I upload a directory full of artifacts to GitHub Packages?

How do I remove all upstream packages in DevOps artifacts?

Nexusartifactuploader bulk upload multiple artifacts

How can I download Maven artifacts within a plugin?

How to NOT download artifacts from previous stages for build configuration?

TOP Ranking

  1. 1

    Failed to listen on localhost:8000 (reason: Cannot assign requested address)

  2. 2

    Loopback Error: connect ECONNREFUSED 127.0.0.1:3306 (MAMP)

  3. 3

    How to import an asset in swift using Bundle.main.path() in a react-native native module

  4. 4

    pump.io port in URL

  5. 5

    Compiler error CS0246 (type or namespace not found) on using Ninject in ASP.NET vNext

  6. 6

    BigQuery - concatenate ignoring NULL

  7. 7

    ngClass error (Can't bind ngClass since it isn't a known property of div) in Angular 11.0.3

  8. 8

    ggplotly no applicable method for 'plotly_build' applied to an object of class "NULL" if statements

  9. 9

    Spring Boot JPA PostgreSQL Web App - Internal Authentication Error

  10. 10

    How to remove the extra space from right in a webview?

  11. 11

    java.lang.NullPointerException: Cannot read the array length because "<local3>" is null

  12. 12

    Jquery different data trapped from direct mousedown event and simulation via $(this).trigger('mousedown');

  13. 13

    flutter: dropdown item programmatically unselect problem

  14. 14

    How to use merge windows unallocated space into Ubuntu using GParted?

  15. 15

    Change dd-mm-yyyy date format of dataframe date column to yyyy-mm-dd

  16. 16

    Nuget add packages gives access denied errors

  17. 17

    Svchost high CPU from Microsoft.BingWeather app errors

  18. 18

    Can't pre-populate phone number and message body in SMS link on iPhones when SMS app is not running in the background

  19. 19

    12.04.3--- Dconf Editor won't show com>canonical>unity option

  20. 20

    Any way to remove trailing whitespace *FOR EDITED* lines in Eclipse [for Java]?

  21. 21

    maven-jaxb2-plugin cannot generate classes due to two declarations cause a collision in ObjectFactory class

HotTag

Archive