Firmware files appear to be compressed now. Need to add decompression support to our drivers.

Created attachment 280 [details]
0001-eal-unix-support-ZSTD-compression-for-firmwares.patch

Can you have a try with this patch?
I only checked compilation, so it might be broken, but I think you can figure out how to make it work.

Thanks.

Patch works for me with one small change - file suffix for the zstd-compressed files on ubuntu is ".zst", i.e. no "d". With that very minor change, patch works.

Build dpdk failed:
[75/2926] Compiling C object lib/librte_eal.a.p/eal_unix_eal_firmware.c.o
FAILED: lib/librte_eal.a.p/eal_unix_eal_firmware.c.o
gcc -Ilib/librte_eal.a.p -Ilib -I../lib -I. -I.. -Iconfig -I../config -Ilib/eal/include -I../lib/eal/include -Ilib/eal/linux/include -I../lib/eal/linux/include -Ilib/eal/x86/include -I../lib/eal/x86/include -Ilib/eal/common -I../lib/eal/common -Ilib/eal -I../lib/eal -Ilib/kvargs -I../lib/kvargs -Ilib/log -I../lib/log -Ilib/metrics -I../lib/metrics -Ilib/telemetry -I../lib/telemetry -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -std=c11 -O3 -include rte_config.h -Wcast-qual -Wdeprecated -Wformat -Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-address-of-packed-member -Wno-packed-not-aligned -Wno-missing-field-initializers -Wno-zero-length-bounds -D_GNU_SOURCE -fPIC -march=native -mrtm -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -Wno-format-truncation '-DABI_VERSION="24.2"' -DRTE_LOG_DEFAULT_LOGTYPE=lib.eal -MD -MQ lib/librte_eal.a.p/eal_unix_eal_firmware.c.o -MF lib/librte_eal.a.p/eal_unix_eal_firmware.c.o.d -o lib/librte_eal.a.p/eal_unix_eal_firmware.c.o -c ../lib/eal/unix/eal_firmware.c
In file included from ../lib/eal/unix/eal_firmware.c:13:
../lib/eal/unix/eal_firmware.c: In function ‘rte_firmware_read’:
../lib/eal/unix/eal_firmware.c:164:41: error: ‘compression_algorithms’ undeclared (first use in this function)
  164 |                 for (i = 0; i < RTE_DIM(compression_algorithms); i++) {
      |                                         ^~~~~~~~~~~~~~~~~~~~~~
../lib/eal/include/rte_common.h:719:34: note: in definition of macro ‘RTE_DIM’
  719 | #define RTE_DIM(a)      (sizeof (a) / sizeof ((a)[0]))
      |                                  ^
../lib/eal/unix/eal_firmware.c:164:41: note: each undeclared identifier is reported only once for each function it appears in
  164 |                 for (i = 0; i < RTE_DIM(compression_algorithms); i++) {
      |                                         ^~~~~~~~~~~~~~~~~~~~~~
../lib/eal/include/rte_common.h:719:34: note: in definition of macro ‘RTE_DIM’
  719 | #define RTE_DIM(a)      (sizeof (a) / sizeof ((a)[0]))
      |                                  ^
[156/2926] Compiling C object lib/librte_ethdev.a.p/ethdev_rte_ethdev.c.o

Hi Jiang,

patch may not have applied correctly for you. "compression_algorithms" is declared as a global at the top of the file in the patch. Patch compiles ok for me, and works with the one minor change made as I describe.

This is a problem in the patch when libarchive is not available, I'll fix it. For now make sure to install pkg-config and libarchive-dev.

Fixed in 6f80df8cb0f8 ("eal/unix: support ZSTD compression for firmware").