lp:~raharper/cloud-init

Author: Ryan Harper
Author Date: 2020-02-20 17:08:10 UTC

cc_growpart: log growpart stdout to console and cloud-init.log (skip stderr duplication)

Author: Ryan Harper
Author Date: 2020-01-28 17:09:54 UTC

update changelog.

Author: Ryan Harper
Author Date: 2020-01-28 16:43:10 UTC

update changelog.

Author: Ryan Harper
Author Date: 2019-11-07 22:06:48 UTC

lp-to-git-users: adding raharper

Mapped from raharper

Author: Ryan Harper
Author Date: 2019-11-07 21:19:41 UTC

Tell users to fork from GitHub account first, fix clone url

Author: Ryan Harper
Author Date: 2019-11-06 21:35:02 UTC

Drop github pull-request template to prepare for migration to github as primary

Author: Ryan Harper
Author Date: 2019-10-23 20:48:12 UTC

Drop print debug lines

Author: Ryan Harper
Author Date: 2019-10-11 15:32:23 UTC

Add the sleep before next retry.

Author: Ryan Harper
Author Date: 2019-10-09 21:32:58 UTC

releasing cloud-init version 19.2-36-g059d049c-0ubuntu3

Author: Ryan Harper
Author Date: 2019-09-26 21:29:49 UTC

ovf: do not generate random instance-id for IMC customization path

Cloud-init will not operate properly if the instance-id value changes
on each boot. This is the source of a number of behavioral bugs filed
against cloud-init with OVF datasource. Instead, use a static instance-id
value, iid-vmware-imc, similar to iid-dsovf.

Author: Ryan Harper
Author Date: 2019-09-26 18:53:04 UTC

Clarify content variable names

Author: Ryan Harper
Author Date: 2019-09-26 18:44:39 UTC

Use only variant names in KNOWN_DISTROS

Author: Ryan Harper
Author Date: 2019-09-25 06:04:52 UTC

pylintrc: add 'enter_context' to generated-members list

On Bionic (python 3.6.8) we now see no-member errors on untouched code.
This does not reproduce on Xenial (3.5) nor on Eoan (3.7.4). Resolve this
by adding the member attribute to the generated member list in pylintrc.

Author: Ryan Harper
Author Date: 2019-09-19 07:50:38 UTC

Don't generate DSA keys by default anymore

Author: Ryan Harper
Author Date: 2019-09-18 14:22:34 UTC

Handle write_file_log_append failures

Author: Ryan Harper
Author Date: 2019-09-17 09:19:21 UTC

Better name is has_master since the interface has the 'master' sysfs attr file

Author: Ryan Harper
Author Date: 2019-09-13 20:59:44 UTC

Switch to json format for records, add unittests

Author: Ryan Harper
Author Date: 2019-09-11 16:18:07 UTC

cloud-init.service: on centos/fedora/redhat wait on NetworkManager.service

In Centos/RHEL 8, NetworkManager is installed as the networking service.
cloud-init.service needs to run After this service to allow the OS to
bring networking up first.

LP: #1843334

Author: Ryan Harper
Author Date: 2019-09-09 20:00:43 UTC

Drop extra line added after rebase

Author: Ryan Harper
Author Date: 2019-08-30 21:50:50 UTC

Break up replace expressions with -e

Author: Ryan Harper
Author Date: 2019-08-30 03:29:20 UTC

Fix pycodestyle

Author: Ryan Harper
Author Date: 2019-08-28 00:06:05 UTC

drop subnet and router from v1 config

Author: Ryan Harper
Author Date: 2019-08-24 04:35:10 UTC

Debug iterative _materialize

Author: Ryan Harper
Author Date: 2019-08-19 22:41:20 UTC

Allow for networking service to race cloud-init, try to wait a short time and if we fail, try ephemeral and restart networking if it was active before ephemeral

Author: Ryan Harper
Author Date: 2019-07-25 22:15:55 UTC

Enable generated mount checking by default, fix config entry

Author: Ryan Harper
Author Date: 2019-07-23 18:14:23 UTC

Workaround pylint issue with py2/py3 Queue import issues.

Author: Ryan Harper
Author Date: 2019-07-18 16:49:32 UTC

Fix extra newline

Author: Ryan Harper
Author Date: 2019-07-17 19:29:05 UTC

Fix leaked calls to net.device_driver, net.device_devid, add testing

Author: Ryan Harper
Author Date: 2019-07-17 16:11:27 UTC

Release 19.2

Bump the version in cloudinit/version.py to be 19.2 and update ChangeLog.

LP: #1836921

Author: Ryan Harper
Author Date: 2019-07-16 19:53:28 UTC

Fix pycodestyle

Author: Ryan Harper
Author Date: 2019-07-16 18:45:47 UTC

parse_static_routes: on parsing error return parsed static routes like dhclient

Author: Ryan Harper
Author Date: 2019-06-19 18:56:22 UTC

setup.py is awkward on arg parsing for --distro

Author: Ryan Harper
Author Date: 2019-06-17 15:12:56 UTC

Redhat-based systems use libexec for path to ds-identify

Author: Ryan Harper
Author Date: 2019-05-09 17:51:14 UTC

Update comment to reference original bug, note release, improve code per Chad.

Author: Ryan Harper
Author Date: 2019-04-22 21:36:53 UTC

Drop parens, unneeded.

Author: Ryan Harper
Author Date: 2019-03-31 16:05:52 UTC

cloud_tests: fix apt_pipelining test-cases

The apt_pipelining test-cases were broken but until cloud-init
changed it's default behavior to not disable, these silently passed
as both only ever checked if pipelinging was disabled.

First, the tests used the 'apt' namespace, which is not for configuring
pipelining, rather that requires 'apt_pipelining' as the namespace.

Second, the 'os' variant needs to check that cloud-init does not
write a configuration file; it was a copy-and-paste error from the
disable test-case.

This branch fixes the config and collection to validate both scenarios.

Author: Ryan Harper
Author Date: 2019-03-13 21:55:22 UTC

DataSource: use class property to allow subclasses to override update_events

Subclasses of DataSource, like DataSourceAzure would update the .update_events
attribute of the base-class; this value is propagated to any future subclass
that gets instantiated. Therefore cloud-init would *leak* these additional
update_events into subclasses which do not use them.

Convert the class attribute into a property and allow subclasses to
override the property to append/modify the attribute but only for
the subclass.

LP: #1819913

Author: Ryan Harper
Author Date: 2019-03-08 21:35:33 UTC

cloud-init-per: POSIX sh does not support string subst, use sed

cloud-init-per is run via /bin/sh which requires POSIX shell
compliance and does not implement string substitution like
bash. Replace these calls with use of sed.

LP: #1819222

Author: Ryan Harper
Author Date: 2019-03-04 15:52:31 UTC

tip-pylint: Fix assignment-from-return-none errors

pylint now complains about assignment of None from a
return of a function call. This does not account for subclassing
so we resolve this issue by removing the assignment in the unittest.

Author: Ryan Harper
Author Date: 2019-02-07 20:36:24 UTC

Use yaml.serializer anchor template to create regex for search

Author: Ryan Harper
Author Date: 2019-02-06 19:12:04 UTC

netinfo: add freebsd ifconfig output unittest

Author: Ryan Harper
Author Date: 2019-01-22 18:52:08 UTC

doc: add docs on built-in mergers and their options.

Author: Ryan Harper
Author Date: 2018-12-14 20:46:25 UTC

releasing cloud-init version 18.5-1-g5b065316-0ubuntu1

Author: Ryan Harper
Author Date: 2018-12-13 19:24:34 UTC

Fix Changelog LP: LP:

Author: Joshua Powers
Author Date: 2018-12-11 23:05:39 UTC

tests: add Disco release

Author: Ryan Harper
Author Date: 2018-12-05 20:45:14 UTC

net/sysconfig: fix routes with metric handling, add ipv6 scenarios, fix unittest

Author: Ryan Harper
Author Date: 2018-12-03 15:37:29 UTC

Replace tabs

Author: Ryan Harper
Author Date: 2018-10-12 19:58:03 UTC

docs: add documentation for events and hotplug support

Author: Ryan Harper
Author Date: 2018-09-14 21:13:39 UTC

releasing cloud-init version 18.3-46-gbb60f61b-0ubuntu1

Author: Ryan Harper
Author Date: 2018-09-14 19:06:50 UTC

releasing cloud-init version 18.3-44-g84bf2482-0ubuntu1

Author: Ryan Harper
Author Date: 2018-09-10 22:12:30 UTC

releasing cloud-init version 18.3-39-g757247f9-0ubuntu1

Author: Ryan Harper
Author Date: 2018-09-05 21:04:55 UTC

releasing cloud-init version 18.3-38-gd47d404e-0ubuntu1

Author: Scott Moser
Author Date: 2018-09-04 19:36:51 UTC

Additional cleanup of distro network rendering unittests

a.) update the freebsd class to use the reRooted()
    and then also the subsequent removal of no-longer-used ExitStack import.

b.) drop unneeded patch of glob.glob()
    the FilesystemMocking test case actually takes care of that via
    'listdir' replacements.

c.) drop of a mock of 'subp'.

Author: Ryan Harper
Author Date: 2018-08-31 21:17:40 UTC

Add a reporting.flush_events() call to allow main process to synchronize with daemon thread

Author: Ryan Harper
Author Date: 2018-04-10 15:49:59 UTC

Add reporting events and log_time around early source of blocking time

In looking at some boot time for Xenial, Artful and Bionic, we noticed
some long amounts of time that appeared to be part of the DataSource
but we related to resolving URLs. In Artful and Bionic, there was an
issue (bug: #1739672) that resulted in slow getaddrinfo() calls when
systemd-resolved was in use. This patch adds two events that track
time for ds.setup_datasource() and ds.activate_datasource().
Additionally use log_time() to wrapper util.is_resolvable_url() which
leaves info in cloud-init.log about how much time was spent.

Author: Ryan Harper
Author Date: 2018-04-27 16:31:05 UTC

releasing cloud-init version 18.2-27-g6ef92c98-0ubuntu1~18.04.1

Author: Ryan Harper
Author Date: 2018-04-26 21:34:06 UTC

releasing cloud-init version 18.2-27-g6ef92c98-0ubuntu1

Author: Ryan Harper
Author Date: 2018-04-26 19:15:39 UTC

unittest: add unittest for util.udevadm_settle

Author: Ryan Harper
Author Date: 2018-04-24 23:18:44 UTC

cloud-init-local: use systemd-udev-settle.service to ensure stable nic names

The cloud-init-local.service expects that any network device name changes have
already been completed by the kernel or udev daemon. While at least in Ubuntu
the systemd-udev-settle.service is enabled, nothing in the Ubuntu cloud images
by default has a Wants for this service so it does not run.

In some situations we've found that the renaming of interfaces from kernel names
(eth0, eth1, etc) to their persistent names (eno1, ens3, enp0s1, etc) may happen
after cloud-init-local has started where it reads values from sysfs about what
network devices are present, and which device to use as a fallback nic.

Subsequently, cloud-init-local would write out network configuration for a
kernel device name which would no longer be present by the time that networking
services start to bring up the devices. The result is that the instance does
not get networking configured. Prior to use of systemd-networkd, the Ubuntu
'networking.service' unit included a call to udevadm settle which is why this
race is not seen on a Xenial system.

LP: #1766287

Author: Ryan Harper
Author Date: 2018-04-04 17:49:40 UTC

cloud_tests: help pylint

pylint missed finding a typo in the lxd platform because it could not
determine that the variable was being used was a string. The variable
was set by loading a yaml file which pylint couldn't know that it
would be a string. In these cases, we can be more explicit.

Author: Ryan Harper
Author Date: 2018-04-17 14:16:44 UTC

Add base_params, remove positional params, combine dhclient,features case

Author: Ryan Harper
Author Date: 2018-04-12 18:26:11 UTC

Use /bin/sh where possible in cloud_tests for ntp

Author: Ryan Harper
Author Date: 2018-04-03 21:12:55 UTC

releasing cloud-init version 18.2-3-gb27f713a-0ubuntu1~17.10.1

Author: Ryan Harper
Author Date: 2018-04-03 21:10:43 UTC

releasing cloud-init version 18.2-3-gb27f713a-0ubuntu1~16.04.1

Author: Ryan Harper
Author Date: 2018-04-03 21:10:43 UTC

releasing cloud-init version 18.2-3-gb27f713a-0ubuntu1~16.04.1

Author: Ryan Harper
Author Date: 2018-04-03 20:43:59 UTC

releasing cloud-init version 18.2-3-gb27f713a-0ubuntu1

Author: Ryan Harper
Author Date: 2018-03-30 18:30:07 UTC

cc_resizefs, util: handle no /dev/zfs

The zfs/zpool commands will hang for 10 seconds if /dev/zfs is not
present (bug 1760173). This is a common occurance for containers
using zfs as rootfs. Additionally handle missing zpool command or
other errors that may occur while executing the zpool command.

Author: Ryan Harper
Author Date: 2018-03-27 14:52:39 UTC

Make tox happy

Author: Ryan Harper
Author Date: 2018-03-19 15:35:05 UTC

Handle global dns entries in netplan

In network config v1 format, there are dns values which are not bound
to a specific interface and do not map to the per-interface format in
netplan. To handle this case we render netplan configuration that duplicates
the DNS configuration on any interface that has a static network config.
We avoiding interfaces which have DHCP configuration which may provide
conflicting DNS values.

LP: #1750884

Author: Ryan Harper
Author Date: 2017-12-01 20:06:32 UTC

netplan: render bridge port-priority values

Update netplan renderer to write out bridge port-priority values
now that netplan supports the feature.

LP: #1735821

Author: Ryan Harper
Author Date: 2018-02-23 21:19:04 UTC

netplan: disable IPV6 RA mode unless configuring ipv6

On distros using netplan and systemd-networkd the default configuration
is to enable solicitation of of IPV6 Router Advertisments. This incurs
a significant delay (up to 10 seconds) during boot if the network
does not contain any IPV6 routers. If the network configuration that
cloud-init renders is not explicitly configuring IPV6, then disable IPV6 RA
on the interfaces in the configuration. Note, this does not affect IPV6
link-local addresses.

Author: Ryan Harper
Author Date: 2018-01-23 00:33:08 UTC

net: accept network-config in netplan format for renaming interfaces

net.apply_network_config_names currently only accepts network-config
in version 1 format. When users include a netplan format network-config the
rename code does not find any of the 'set-name' directives and does not rename
any of the interfaces. This causes some netplan configurations to fail.

This patch adds support for parsing netplan format and extracts the needed
information (macaddress and set-name values) to allow cloud-init to issue
interface rename commands. We know raise a RuntimeError if presented with
an unknown config format.

LP: #1709715

Author: Ryan Harper
Author Date: 2017-12-06 22:30:22 UTC

Add pylint v1.7.1 support to tox

- Assign class property accessors a result and use them
- Cast some instance objects with dict()
- Fix Cheetah test import
- Fix missing class member in CepkoResultTests
- Fix up cloud_tests use of platforms
- Handle python2.7 vs 3+ ConfigParser use of readfp (depricated)
- Modify .pylintrc generated-members ignore mocked object members, regex: m_.*
- Remove tools/hacking, it is replaced by 'hacking' in pep8 env
- Remove useless test-class calls to super class
- Replace "dangerous" params defaulting to {} with None
- Replace depricated assertRegexp -> assertRegex using
  workaround for python2.6 missing assertRegex attr
- Update use of assertEqual(<boolean>, value) to assert<Boolean>(value)

Author: Ryan Harper
Author Date: 2017-10-24 15:40:21 UTC

releasing package cloud-init version 17.1-25-g17a15f9e-0ubuntu1

Author: Ryan Harper
Author Date: 2017-10-10 12:28:05 UTC

Azure: don't generate network configuration for SRIOV devices

Azure kernel now configures the SRIOV devices itself so cloud-init
does not need to provide any SRIOV device configuration or udev
naming rules.

Author: Ryan Harper
Author Date: 2017-10-05 21:16:00 UTC

releasing package cloud-init version 17.1-17-g45d361cb-0ubuntu1

Author: Ryan Harper
Author Date: 2017-10-05 18:46:48 UTC

releasing package cloud-init version 17.1-16-g6eb4dc24-0ubuntu1

Author: Ryan Harper
Author Date: 2017-10-05 18:18:53 UTC

releasing package cloud-init version 17.1-15-g57e2e01-0ubuntu1

Author: Ryan Harper
Author Date: 2017-10-03 23:56:52 UTC

network: bridge_stp value not always correct

Update network_state to store the bridge_stp value as a boolean.
The various renderers then can map the boolean value to the correct
output as needed; eni uses 'on/off', sysconfig uses 'yes/no' and
netplan will use the boolean directly.

Update unittest values for sysconfig and netplan. Both contained
the network_state string value which resulted in not correctly enable/disable
STP in the target system.

Update network_state comment (fd -> forward-delay, add stp as boolean) on
bridge commands to match the expected format of a netplan bridge command.

LP: #1721157

Author: Ryan Harper
Author Date: 2017-10-03 16:00:09 UTC

releasing package cloud-init version 17.1-13-g7fd0425-0ubuntu1

Author: Ryan Harper
Author Date: 2017-10-03 15:42:25 UTC

releasing package cloud-init version 17.1-13-g7fd0425-0ubuntu1

Author: Ryan Harper
Author Date: 2017-10-03 15:22:32 UTC

releasing package cloud-init version 17.1-13-g7fd0425-0ubuntu1

Author: Ryan Harper
Author Date: 2017-09-19 16:10:09 UTC

DataSourceOVF: use util.find_devs_with(TYPE=iso9660)

DataSourceOVF attempts to find iso files via walking os.listdir('/dev/')
which is far too wide. This approach is too invasive and can sometimes
race with systemd attempting to fsck and mount devices.

Instead, utilize cloudinit.util.find_devs_with to filter devices by
criteria (which uses blkid under the covers). This results in fewer
attempts to mount block devices which do not contain iso filesystems.

Unittest changes include:
- cloudinit.tests.helpers; introduce add_patch() helper
- Add unittest coverage for DataSourceOVF use of transport_iso9660

Author: Ryan Harper
Author Date: 2017-08-16 21:50:07 UTC

distro: allow distro to specify a default locale

Currently the cloud-init default locale (en_US.UTF-8) is set by
the base datasource class. This patch allows a distro to overide
the fallback value with one that's available in the distro but continues
to respect an image which has preconfigured a locale.

- Distro object now has a get_locale method which will return a preconfigure
  locale setting by checking the distros locale system configuration file.
  If not set or not present, return the default locale of en_US.UTF-8 which
  retains behavior of all previous cloud-init releases
- Apply locale now handles regenerating locales or system configuraion files
  as needed.
- Adjust apply_locale logic to skip locale-regen if the specified LANG value is
  C.UTF-8,C, or POSIX; they do not require regeneration.
- Further add unittests to exercise the default paths for Ubuntu and
  non-ubuntu paths to validate they get the LANG expected.

Author: Ryan Harper
Author Date: 2017-08-07 18:38:56 UTC

Always use UTC for timevalues in logging

Author: Ryan Harper
Author Date: 2017-08-21 20:09:49 UTC

releasing package cloud-init version 0.7.9-243-ge74d775-0ubuntu1

Author: Ryan Harper
Author Date: 2017-08-21 20:09:49 UTC

releasing package cloud-init version 0.7.9-243-ge74d775-0ubuntu1

Author: Ryan Harper
Author Date: 2017-08-21 18:17:36 UTC

releasing package cloud-init version 0.7.9-242-gdc2bd79-0ubuntu1

Author: Ryan Harper
Author Date: 2017-08-09 19:44:20 UTC

network: add v2 passthrough and fix parsing v2 config with bonds/bridge params

If the network-config sent to cloud-init is in version: 2 format then
when rendering netplan, we can pass the content through and avoid
consuming network_state elements. This removes the need for trying to
map many v2 features onto network state where other renderers won't be
able to use anyhow (for example match parameters for multi-interface
configuration and wifi configuration support).

Additionally ensure we retain bond/bridge v2 configuration in network
state so when rendering to eni or sysconfig we don't lose the configuration

- Drop the NotImplemented wifi exception, log a warning that it works for
  netplan only
- Adjust unittests to new code path and output
- Fix issue with v2 macaddress values getting dropped
- Add unittests for consuming/validating v2 configurations

LP: #1709180

Author: Ryan Harper
Author Date: 2017-08-14 16:40:54 UTC

doc: update capabilities with features available, link doc reference, cli example

Author: Ryan Harper
Author Date: 2017-08-01 23:00:00 UTC

cc_ntp: fallback on timesyncd configuration if ntp is not installable

Some systems like Ubuntu-Core do not provide an ntp package for
installation but do include systemd-timesyncd (an ntp client).
On such systems cloud-init will generate a timesyncd configuration
using the 'servers' and 'pools' values as ntp hosts for timesyncd to use.

LP: #1686485

Author: Ryan Harper
Author Date: 2017-07-31 18:35:07 UTC

sysconfig: Dont repeat header when rendering resolv.conf

The sysconfig renderer duplicates the cloud-init header string
when rendering resolv.conf file. This leads to resolv.conf file
growing with every reboot of a system. Fix this by checking for
the header when loading content from existing file.

Update one of the sysconfig unittests with multiple render calls
to simulate the reboot to check that we don't repeat the header.

LP: #1701420

Author: Ryan Harper
Author Date: 2017-07-20 19:46:30 UTC

sysconfig: use MACADDR on bonds/bridges to configure mac_address

Previously, sysconfig rendered HWADDR for all interface types, but
that value is only used to identify physical devices. Instead use
MACADDR to configure the MAC on virtual devices, like bonds and
bridges.

- Sort bond slave list to ensure consistent ordering in sysconfig
  rendered files.
- Add unittests for sysconfig rendering of bonds/bridges with
  mac_address

LP: #1701417

Author: Ryan Harper
Author Date: 2017-06-28 21:55:49 UTC

Update network unittest to validate ipv6 gateway route

Author: Ryan Harper
Author Date: 2017-06-26 19:46:42 UTC

Fix spelling errors in comment block.

Author: Ryan Harper
Author Date: 2017-06-13 14:06:41 UTC

Templatize systemd service files

Distro specfic changes are needed in the unit file to ensure that
cloud-init behaves the same in different systemd environment.

Author: Ryan Harper
Author Date: 2017-06-09 18:04:29 UTC

Merge branch 'redhat-fix-spec-systemd' of git.launchpad.net:~raharper/cloud-init into redhat-fix-spec-systemd

Author: Ryan Harper
Author Date: 2017-06-08 20:42:12 UTC

Check before attempting to regenerate locales

Avoid unnecessary IO incurred when generating locales by
testing if the system has already configured a LANG setting.
If a user supplied locale or the default locale defined in
in cloud-init differs from what is presently configured
then cloud-init will regenerate locales based on the supplied
value.

Author: Ryan Harper
Author Date: 2017-04-27 20:34:28 UTC

Allow restorecon to be non-fatal

On some systems with python-libselinux a bug[1] related to recursive
restorecon fails but the distro release does not yet include
an update. This change will accept the error and log a warning.

1. https://bugzilla.redhat.com/show_bug.cgi?id=1406520

Author: Ryan Harper
Author Date: 2017-05-26 19:29:24 UTC

Fix eni rendering for bridge parameters which require repeated key for values

There are a few bridge parameters which require repeating the key with each
value in the list when rendering eni. Extend the network unittests to cover
all of the known bridge parameters and check we render eni and netplan
correctly.