livecd-rootfs

Code review comment for lp://staging/~mwhudson/livecd-rootfs/live-server-gzip-initramfs

live-server-gzip-initramfs
Merge into trunk

Revision history for this message

Michael Hudson-Doyle (mwhudson) wrote on 2018-03-13:

> Tagging Balint for a review, to speak to / quantify the tradeoffs here between
> gzip and lzma (compressed size + time to read initramfs from bootloader, vs.
> time to decompress in kernel).

Thanks.

> There's no support for this yet in livecd-rootfs + live-build

Or mkinitramfs, given that you need to pass -l to lz4 when compressing to get the kernel to understand the initrd.lz4.

> , but this seems
> like an ideal use case for lz4 instead - compressed initrd created once
> server-side, and uncompressed many times
> on each boot of the live image. But we're still a way out from being able to
> make such a change and shouldn't block on it for bionic.

I did try lz4 in my playing around and it did not compress as well as gzip:

-rw-r--r-- 1 root root 148M Mar 12 12:59 initrd
-rw-r--r-- 1 root root 55M Mar 12 13:00 initrd.gz.3
-rw-r--r-- 1 root root 52M Mar 12 13:00 initrd.gz.6
-rw-r--r-- 1 root root 52M Mar 12 13:00 initrd.gz.9
-r--r--r-- 1 root root 33M Mar 11 20:36 initrd.lz
-rw-r--r-- 1 root root 61M Mar 12 13:01 initrd.lz4.3
-rw-r--r-- 1 root root 60M Mar 12 13:01 initrd.lz4.6
-rw-r--r-- 1 root root 60M Mar 12 13:00 initrd.lz4.9

It may have been faster to decompress but it was 0.7s for gzip, 0.4s for lz4 vs 5.2s for lzma sort of numbers.

Getting support for lz4 initrds would seem to involve changing a lot of things (e.g. initrd_suffix in debian-cd) and generally doesn't seem worth it to me at this point in the cycle.

Does the kernel support zstd initrds yet? :)

> Also, aligning with my previous feedback to @rcj, we should really not be
> shadowing the live-build default compression here, but instead ensure
> $INITRAMFS_COMPRESSION is unset for cases where live-build defaults (currently
> gzip; future, lz4?) DTRT.

So you'd prefer something along the lines of this:

=== modified file 'live-build/auto/config'
--- live-build/auto/config 2018-03-13 08:37:06 +0000
+++ live-build/auto/config 2018-03-13 20:31:46 +0000
@@ -818,7 +818,7 @@
  ${KERNEL_FLAVOURS:+--linux-flavours "$KERNEL_FLAVOURS"} \
  --initsystem none \
  --bootloader "$BOOTLOADER" \
- --initramfs-compression "${INITRAMFS_COMPRESSION:-lzma}" \
+ ${INITRAMFS_COMPRESSION:+--initramfs-compression "$INITRAMFS_COMPRESSION}" \
  --cache false \
  ${BOOTAPPEND_LIVE:+--bootappend-live "$BOOTAPPEND_LIVE"} \
  $OPTS \

(live-build appears to default to gzip by my reading)? That makes sense but is it safe to land at this point of the cycle?

> Tagging Balint for a review, to speak to / quantify the tradeoffs here between
> gzip and lzma (compressed size + time to read initramfs from bootloader, vs.
> time to decompress in kernel).

Thanks.

> There's no support for this yet in livecd-rootfs + live-build

Or mkinitramfs, given that you need to pass -l to lz4 when compressing to get the kernel to understand the initrd.lz4.

> , but this seems
> like an ideal use case for lz4 instead - compressed initrd created once
> server-side, and uncompressed many times
> on each boot of the live image.  But we're still a way out from being able to
> make such a change and shouldn't block on it for bionic.

I did try lz4 in my playing around and it did not compress as well as gzip:

-rw-r--r-- 1 root root 148M Mar 12 12:59 initrd
-rw-r--r-- 1 root root  55M Mar 12 13:00 initrd.gz.3
-rw-r--r-- 1 root root  52M Mar 12 13:00 initrd.gz.6
-rw-r--r-- 1 root root  52M Mar 12 13:00 initrd.gz.9
-r--r--r-- 1 root root  33M Mar 11 20:36 initrd.lz
-rw-r--r-- 1 root root  61M Mar 12 13:01 initrd.lz4.3
-rw-r--r-- 1 root root  60M Mar 12 13:01 initrd.lz4.6
-rw-r--r-- 1 root root  60M Mar 12 13:00 initrd.lz4.9

It may have been faster to decompress but it was 0.7s for gzip, 0.4s for lz4 vs 5.2s for lzma sort of numbers.

Getting support for lz4 initrds would seem to involve changing a lot of things (e.g. initrd_suffix in debian-cd) and generally doesn't seem worth it to me at this point in the cycle.

Does the kernel support zstd initrds yet? :)

So you'd prefer something along the lines of this:

=== modified file 'live-build/auto/config'
--- live-build/auto/config	2018-03-13 08:37:06 +0000
+++ live-build/auto/config	2018-03-13 20:31:46 +0000
@@ -818,7 +818,7 @@
 	${KERNEL_FLAVOURS:+--linux-flavours "$KERNEL_FLAVOURS"} \
 	--initsystem none \
 	--bootloader "$BOOTLOADER" \
-	--initramfs-compression "${INITRAMFS_COMPRESSION:-lzma}" \
+	${INITRAMFS_COMPRESSION:+--initramfs-compression "$INITRAMFS_COMPRESSION}" \
 	--cache false \
 	${BOOTAPPEND_LIVE:+--bootappend-live "$BOOTAPPEND_LIVE"} \
 	$OPTS \

(live-build appears to default to gzip by my reading)? That makes sense but is it safe to land at this point of the cycle?

« Back to merge proposal