Reducing CMSIS Packs size

My problems:

• I build docker images with tools for developing for microcontrollers (which contain toolchains, debuggers, utilities, etc.). In this image I ship pyOCD as well, and as one requisite for these images is not connecting to the internet, I need to pre-download CMSIS packs for the targets that I want to use. CMSIS Packs are a lot of megabytes, thus the docker images grow of gigabytes in size with just a few packs!
• CMSIS Packs waste a lot of space even in my personal computer

Proposed solution

Looking at the code, it seems to me that of a CMSIS Pack only the following files are used:

• the .pdsc file
• the FlashAlgo files
• the .svd files
• more?

In any case the impression is that a lot of files in the pack are completely useless for pyocd.

What I'm proposing is that we find some way to store on the disk only the relevant parts of the pack, which is really small compared to the full size file. This would allow to save a very large amount of space.

Possible implementations

I have thought of two ways this could be implemented, which I'm giving just as a pointer of course:

• Download the zip file and then delete from the local copy the files that are not needed.
  • PROS:
    • Simple to implement, just delete files from zip after downloading
    • Could be done by pyOCD itself
  • CONS:
    • Might be forward incompatible, if in future versions of pyOCD some other files are needed, there may be errors and packs would need to be downloaded again.
    • The whole pack needs to be downloaded anyway
• Access only the needed part of the zip pack directly from the server via HTTP and extract on the fly the needed files (see stackoverflow or this library)
  • PROS:
    • No need to download the full pack
  • CONS:
    • Probably more tricky to implement (as it would be implemented in cmsis-pack-manager I suppose)
    • HTTP servers might not like GET requests for only a part of the file (only maybe, as it should be standard HTTP anyway)

Sorry for the long post, I just wanted to start a conversation and give a few ideas on this, given that in my opinion this is one of the few downsides of pyOCD compared for example to openOCD.

No specific files, tests, or entry points are named. Start by reviewing pyOCD's CMSIS Pack handling and the cmsis-pack-manager area mentioned in the proposal, then determine which pack contents are required. Done would mean reducing stored or downloaded pack data without breaking target support or offline use.