lp:~ogayot/curtin

Author: Olivier Gayot
Author Date: 2023-09-20 09:28:05 UTC

block_meta: also add recovery key if zkey is used

When zkey is properly setup, we do not invoke cryptsetup luksFormat
ourselves. Instead we lean on zkey to invoke cryptsetup luksFormat for
us.

zkey seems to have no native support for invoking cryptsetup luksAddKey,
so we need to manually call it if we want to add a recovery key in a
second slot.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-09-07 10:55:58 UTC

apt_source: do not produce a deb822 source stanza with zero suite

When no suite is set for a given repository, we end up with a
deb822 styled source such as:

 Type: deb
 Suites:
 URIs: http://archive.ubuntu.com/ubuntu

This is invalid and apt/apt-get will raise the following error:

E: Malformed entry 2 in sources file /etc/apt/sources.list.d/ubuntu.sources (Suite)

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-08-10 14:09:56 UTC

block_meta: add luks recovery key if requested

If the storage configuration contains the key "recovery_keyfile", call
cryptsetup luksAddKey after cryptsetup luksFormat so that the key
specified is added as a recovery key (i.e., a normal key in the second
key slot).

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-07-06 14:35:05 UTC

curthooks: fix exception when passing 'kernel: null'

When we don't want curthooks to install a kernel, we can pass:

  kernel: null

However, currently, it makes the code raise an exception because we run
.get() on the kernel object even if it is None.

     config.merge_config(mapping, kernel_cfg.get('mapping', {}))
 AttributeError: 'NoneType' object has no attribute 'get'
 'NoneType' object has no attribute 'get'

Fixed by returning gracefully without installing any kernel upon
encountering this configuration.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-05-03 10:15:31 UTC

udev: fix crash when PARTNAME is invalid utf-8

(We already merged a similar fix in probert)

GPT partition names are basically free-text fields but the udev PARTNAME
field ends up butchered by the kernel when it contains characters
outside the ASCII range.

When using pyudev, the values are all expected to be decodable using the
system's default encoding.

In curtin, it can result in a crash when iterating over all the
properties of a block device, that are mapped by pyudev.

To work around the issue, we now stop iterating over all the values. We
iterate over the keys and only access the value if we need it.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-05-02 14:14:19 UTC

Add file a ( do not merge )

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-04-25 14:23:17 UTC

block: sleep and retry wiping part. if open(O_EXCL) returns EBUSY

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-04-24 10:28:43 UTC

block_meta_v2: don't solely expect flag='logical' for logical partitions

Most of the time, a logical partition has its flag set to 'local'.
However, it some cases, it can also have flag set to 'swap', 'boot' and
possibly other values.

To determine if a partition is actually logical on a DOS partition
table, we need to check the partition number.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-04-03 16:49:44 UTC

block-meta: fix failed disk lookup when WWN includes extension

curtin's storage config extracts the WWN of a disk from one of the
following udev variables:

 * ID_WWN_WITH_EXTENSION (preferred)
 * ID_WWN

However, when trying to look the disk up later in the block-meta code,
curtin only tries to match the WWN value against the ID_WWN variable
(and another variable related to DM multipath).

This means that the disk can not be be found just based on the wwn if
the ID_WWN and ID_WWN_WITH_EXTENSION variables don't hold the same
value.

In the past, that was often okay because other fields (such as disk path
or serial) would still make the disk lookup succeed.

However, the following patch introduced a restriction. In v2, all fields
specified must now match for the disk lookup to be successful:

8c5f87ed block_meta: all fields on a disk action must match with v2 config

Fixed by matching against the ID_WWN_WITH_EXTENSION (preferred) and then
ID_WWN.

(cherry picked from commit 73da3c4f47ea22d58f57371594a396c5434ab260)

Author: Olivier Gayot
Author Date: 2023-04-03 09:40:41 UTC

block-meta: fix failed disk lookup when WWN includes extension

curtin's storage config extracts the WWN of a disk from one of the
following udev variables:

 * ID_WWN_WITH_EXTENSION (preferred)
 * ID_WWN

However, when trying to look the disk up later in the block-meta code,
curtin only tries to match the WWN value against the ID_WWN variable
(and another variable related to DM multipath).

This means that the disk can not be be found just based on the wwn if
the ID_WWN and ID_WWN_WITH_EXTENSION variables don't hold the same
value.

In the past, that was often okay because other fields (such as disk path
or serial) would still make the disk lookup succeed.

However, the following patch introduced a restriction. In v2, all fields
specified must now match for the disk lookup to be successful:

8c5f87ed block_meta: all fields on a disk action must match with v2 config

Fixed by matching against the ID_WWN_WITH_EXTENSION (preferred) and then
ID_WWN.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-03-28 15:46:32 UTC

storage: filter out ventoy image when building storage config

When ventoy is used, the filesystem mounted to /cdrom is not a /dev/sr0
device (major number 11) but a dm device named "ventoy". Add a check to
exclude it as well as real cdrom devices.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-02-24 13:50:49 UTC

system-install: use --assume-downloaded instead of --no-download

Running $ apt-get install --no-download
fails if the packages were previously "downloaded" using the cdrom
rather than an online repository.

This means we can't reliably pass the option on the second apt-get
install invocation.
Passing the --no-download option to $ curtin system-install while not
passing it to apt-get feels wrong. Therefore, I've renamed the
system-install option from --no-download to --assume-downloaded.

For package managers that support the --no-download option properly,
they can use it on the second invocation. For apt, we will not pass the
option.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-02-28 09:56:59 UTC

apt-config: stop returning cfg object in translate_old_apt_features

The translate_old_apt_features function mutates its parameter, but also
used to return the translated object. This goes against the rules of
idiomatic Python. Returning a new configuration object without mutating
the original one would require leaning on copy.deepcopy, which is
expensive and not always needed.

Let's stop returning the resulting configuration so the interface is
clear.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Michael Hudson-Doyle
Author Date: 2022-03-10 22:14:00 UTC

have v2 get_path_to_storage_volume promote multipath members to multipath device more generally

Author: Olivier Gayot
Author Date: 2022-11-03 14:51:17 UTC

distro: fix typo s/packge/package/

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2022-08-08 07:47:15 UTC

install: add option to append to log file

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2022-07-26 09:36:10 UTC

curtin: do not reject rhel9 and derivatives

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2022-07-08 12:17:12 UTC

doc: remove duplicate section about extract subcommand

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2022-01-20 14:16:58 UTC

Make sure curthooks do not discard APT preferences

Although APT preferences applied properly when running curtin
apt-config, we observed during a full installation using Subiquity that
the preferences would not be honored when installing packages from the
"packages" autoinstall section.

This happened because the installation of packages occur after running
curthooks. Curtooks call handle_apt with an almost empty configuration.
Therefore, we would overwrite the preferences by removing the
etc/apt/preferences.d/90curtin.pref file.

It is to be noted that the same behavior occurs for the APT proxy.

Fixed by not removing etc/apt/preferences.d/90curtin.pref when the
configuration does not contain APT preferences.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2022-01-12 15:34:13 UTC

Remove leftover debug print statement

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2022-01-07 08:29:01 UTC

Fix format of examples/apt-source.yaml so it can be passed to apt-config

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2022-01-07 08:28:27 UTC

Add support for APT preferences in apt-config

apt-config now supports a list of APT preferences (i.e. pinning rules)
as in the following example:

apt:
  preferences:
    - {package: "python3-*", pin: "origin *ubuntu.com*", pin-priority: 200}
    - {package: "python-*", pin: "origin *ubuntu.com*", pin-priority: -1}

These preferences are deployed under
<target>/etc/apt/preferences.d/90-curtin.pref using the format specified
in apt_preferences(5).

  Package: python3-*
  Pin: origin *ubuntu.com*
  Pin-Priority: 200

  Package: python-*
  Pin: origin *ubuntu.com*
  Pin-Priority: -1

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Dan Bungert
Author Date: 2021-12-09 03:08:18 UTC

block: output partition device path

reconstructing this later is non-trivial for some devices

Author: Dan Bungert
Author Date: 2021-12-07 03:23:51 UTC

21.3 merge for Jammy

Author: Dan Bungert
Author Date: 2021-09-20 20:54:57 UTC

commands/apt: use python-apt for sources.list

Start using python{,3}-apt for sources.list handling.
generate_sources_list now operates in a pipeline-like model, where each
stage performs one specific transformation on the list of entries.

Author: Paride Legovini
Author Date: 2021-01-27 20:47:05 UTC

releasing curtin version 21.2-0ubuntu1~16.04.1

Author: Paride Legovini
Author Date: 2021-01-27 20:39:57 UTC

releasing curtin version 21.2-0ubuntu1~18.04.1

Author: Paride Legovini
Author Date: 2021-01-27 20:26:44 UTC

releasing curtin version 21.2-0ubuntu1~20.04.1

Author: Paride Legovini
Author Date: 2020-09-25 20:46:55 UTC

releasing curtin version 20.2-0ubuntu1

Author: Ryan Harper
Author Date: 2020-02-14 15:44:17 UTC

releasing curtin version 19.3-26-g82f23e3d-0ubuntu1~19.10.1

Author: Dan Watkins
Author Date: 2019-09-10 21:57:21 UTC

Revert "d/control: update Depends for new probert package names"

This reverts commit 43c0a1474c4d32cebf0e940e555a7ac6851dbe9c.

(I incorrectly didn't fix the Depends during the preceding merge, but
its existence is still sufficient for this revert to not cause merge
conflicts into master.)

Author: Ryan Harper
Author Date: 2019-06-12 20:55:08 UTC

update changelog

Author: Dan Watkins
Author Date: 2019-05-21 21:03:54 UTC

release 19.1

Author: Ryan Harper
Author Date: 2018-05-18 19:02:29 UTC

releasing curtin version 18.1-17-gae48e86f-0ubuntu1~17.10.1

Author: Scott Moser
Author Date: 2017-12-19 18:34:42 UTC

Import version 0.1.0~bzr532-0ubuntu1~17.04.1