Merge lp://staging/~sinzui/juju-ci-tools/azure-arm into lp://staging/juju-ci-tools

Proposed by Curtis Hovey
Status: Merged
Merged at revision: 1404
Proposed branch: lp://staging/~sinzui/juju-ci-tools/azure-arm
Merge into: lp://staging/juju-ci-tools
Diff against target: 562 lines (+553/-0)
2 files modified
tests/test_winazurearm.py (+250/-0)
winazurearm.py (+303/-0)
To merge this branch: bzr merge lp://staging/~sinzui/juju-ci-tools/azure-arm
Reviewer Review Type Date Requested Status
Martin Packman (community) Approve
Review via email: mp+294181@code.staging.launchpad.net

Description of the change

Add support to delete Azure ARM resource_groups.

This branch adds a script that unifies non-final Azure ARM python support library and allows us to delete resource groups. This branch is large. Im sorry. The spike grew out of proportion when I needed to refactor the implementation to allow for testing.

I think the code and tests are well documented. There are a few things that may not be obvious about the upstream lib that influenced this implementation.

1. Azure's ARM python lib is not final. The preview release 3 is missing support for several kinds of objects that are defined in models. but it is not possible to request them. The library cannot be installed with the Azure SMS version, not can it be upgraded. Setup is not ideal. I expect we will only want this lib installed on hosts that needed.

2. The Azure lib is composed of many loosely coupled micro services. The lib defines many model which are light objects, sometimes just namedtuples. The operations modules are the factories that use the msrest lib to request the data to make the objects.

3. As this Library is not stable, I do not want to assume we will installed on all slaves. The import of Azure lib is isolated to a helper method. The script can be on hosts that do not have the library. The test suite will also pass. It uses mocks and fakes in the same manner of the azure tests do to avoid defining a strict contract with the objects.

4. The function use print instead of log because the azure lib logs at lot of information at the info level. At the debug level, I cannot see our script messages.

This branch provides the bazooka needed to delete many old resource groups. My next branch will be a sniper rifle that allows us to delete specific instances in a resource group.

I learned a few things about azure writing this branch.

1. An industrial test run on azure takes about 4.5 hours. I choose to define old to be 6 hours instead of 8.

2. We commonly see resource groups with only a network. this is because they can take a long time to delete. I assume these resource groups are old and should be deleted. The other case might be the 429 rate-limit issue where resources are not provided quickly, Juju aborts, but i cannot delete the network because it is being created.

3. Most deletes of resource groups happen within 20 minutes. We should set the kill-controller timeout to 20 minutes.

To post a comment you must log in.
Revision history for this message
Martin Packman (gz) wrote :

Thanks Curtis!

Have a bunch of nitpicky comments inline, nothing that actually needs fixing to land.

We also need to update the Makefile/requirements.txt for both pytz and azure libraries at some point? For now I guess you've manually installed on the machines we need?

review: Approve
Revision history for this message
Curtis Hovey (sinzui) wrote :

Thank you. I sorted out a few issues.

1. Logging works. We cannot use our common logging setup because azure's lib will spam our logs.
2. tests.TestCase.test_environ now exists so that tests can easily setup a starting environ
3. dry_run became read_only in the AzureARMClient to be clear that the client is not allowed to make changes.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
The diff is not available at this time. You can reload the page or download it.

Subscribers

People subscribed via source and target branches