Merge lp://staging/~bcsaller/charm-tools/composer into lp://staging/charm-tools/1.6

Proposed by Benjamin Saller
Status: Merged
Merged at revision: 359
Proposed branch: lp://staging/~bcsaller/charm-tools/composer
Merge into: lp://staging/charm-tools/1.6
Diff against target: 12668 lines (+11533/-368)
117 files modified
.bzrignore (+3/-1)
MANIFEST.in (+1/-0)
Makefile (+14/-37)
charmtools/compose/__init__.py (+466/-0)
charmtools/compose/config.py (+113/-0)
charmtools/compose/diff_match_patch.py (+1919/-0)
charmtools/compose/fetchers.py (+117/-0)
charmtools/compose/inspector.py (+101/-0)
charmtools/compose/tactics.py (+449/-0)
charmtools/utils.py (+518/-0)
doc/source/compose-intro.md (+18/-0)
doc/source/composer.md (+123/-0)
ez_setup.py (+0/-272)
helpers/python/charmhelpers/tests/test_charmhelpers.py (+1/-1)
requirements.txt (+18/-13)
scripts/packages.sh (+19/-0)
scripts/test (+0/-7)
setup.cfg (+8/-0)
setup.py (+16/-16)
tests/interfaces/mysql/interface.yaml (+1/-0)
tests/interfaces/mysql/provides.py (+1/-0)
tests/interfaces/mysql/requires.py (+1/-0)
tests/test_charm_generate.py (+0/-7)
tests/test_compose.py (+239/-0)
tests/test_config.py (+29/-0)
tests/test_juju_test.py (+0/-9)
tests/test_utils.py (+43/-0)
tests/trusty/a/README.md (+1/-0)
tests/trusty/a/a (+1/-0)
tests/trusty/b/README.md (+1/-0)
tests/trusty/b/composer.yaml (+1/-0)
tests/trusty/b/metadata.yaml (+11/-0)
tests/trusty/c-reactive/README.md (+1/-0)
tests/trusty/c-reactive/composer.yaml (+1/-0)
tests/trusty/c-reactive/hooks/reactive/main.py (+6/-0)
tests/trusty/c/README.md (+1/-0)
tests/trusty/c/composer.yaml (+1/-0)
tests/trusty/c/metadata.yaml (+11/-0)
tests/trusty/chlayer/hooks/charmhelpers.pypi (+1/-0)
tests/trusty/mysql/.bzrignore (+2/-0)
tests/trusty/mysql/Makefile (+24/-0)
tests/trusty/mysql/README.md (+133/-0)
tests/trusty/mysql/charm-helpers.yaml (+9/-0)
tests/trusty/mysql/config.yaml (+141/-0)
tests/trusty/mysql/copyright (+17/-0)
tests/trusty/mysql/hooks/charmhelpers/__init__.py (+38/-0)
tests/trusty/mysql/hooks/charmhelpers/contrib/__init__.py (+15/-0)
tests/trusty/mysql/hooks/charmhelpers/contrib/charmsupport/nrpe.py (+219/-0)
tests/trusty/mysql/hooks/charmhelpers/contrib/charmsupport/volumes.py (+156/-0)
tests/trusty/mysql/hooks/charmhelpers/contrib/database/mysql.py (+385/-0)
tests/trusty/mysql/hooks/charmhelpers/contrib/network/__init__.py (+15/-0)
tests/trusty/mysql/hooks/charmhelpers/contrib/network/ip.py (+450/-0)
tests/trusty/mysql/hooks/charmhelpers/contrib/peerstorage/__init__.py (+148/-0)
tests/trusty/mysql/hooks/charmhelpers/core/__init__.py (+15/-0)
tests/trusty/mysql/hooks/charmhelpers/core/decorators.py (+41/-0)
tests/trusty/mysql/hooks/charmhelpers/core/fstab.py (+134/-0)
tests/trusty/mysql/hooks/charmhelpers/core/hookenv.py (+568/-0)
tests/trusty/mysql/hooks/charmhelpers/core/host.py (+446/-0)
tests/trusty/mysql/hooks/charmhelpers/core/services/__init__.py (+18/-0)
tests/trusty/mysql/hooks/charmhelpers/core/services/base.py (+329/-0)
tests/trusty/mysql/hooks/charmhelpers/core/services/helpers.py (+267/-0)
tests/trusty/mysql/hooks/charmhelpers/core/strutils.py (+42/-0)
tests/trusty/mysql/hooks/charmhelpers/core/sysctl.py (+56/-0)
tests/trusty/mysql/hooks/charmhelpers/core/templating.py (+69/-0)
tests/trusty/mysql/hooks/charmhelpers/core/unitdata.py (+477/-0)
tests/trusty/mysql/hooks/charmhelpers/fetch/__init__.py (+439/-0)
tests/trusty/mysql/hooks/charmhelpers/fetch/archiveurl.py (+161/-0)
tests/trusty/mysql/hooks/charmhelpers/fetch/bzrurl.py (+78/-0)
tests/trusty/mysql/hooks/charmhelpers/fetch/giturl.py (+71/-0)
tests/trusty/mysql/hooks/common.py (+109/-0)
tests/trusty/mysql/hooks/config-changed (+414/-0)
tests/trusty/mysql/hooks/data-relation.py (+31/-0)
tests/trusty/mysql/hooks/db-relation-broken (+21/-0)
tests/trusty/mysql/hooks/db-relation-joined (+87/-0)
tests/trusty/mysql/hooks/ha_relations.py (+163/-0)
tests/trusty/mysql/hooks/install (+49/-0)
tests/trusty/mysql/hooks/master-relation-changed (+95/-0)
tests/trusty/mysql/hooks/monitors-relation-broken (+8/-0)
tests/trusty/mysql/hooks/monitors-relation-departed (+3/-0)
tests/trusty/mysql/hooks/monitors-relation-joined (+9/-0)
tests/trusty/mysql/hooks/monitors.common.bash (+8/-0)
tests/trusty/mysql/hooks/munin-relation-changed (+26/-0)
tests/trusty/mysql/hooks/munin-relation-joined (+6/-0)
tests/trusty/mysql/hooks/nrpe_relations.py (+91/-0)
tests/trusty/mysql/hooks/shared_db_relations.py (+153/-0)
tests/trusty/mysql/hooks/slave-relation-broken (+11/-0)
tests/trusty/mysql/hooks/slave-relation-changed (+89/-0)
tests/trusty/mysql/hooks/slave-relation-joined (+2/-0)
tests/trusty/mysql/hooks/start (+5/-0)
tests/trusty/mysql/hooks/stop (+3/-0)
tests/trusty/mysql/hooks/upgrade-charm (+27/-0)
tests/trusty/mysql/icon.svg (+335/-0)
tests/trusty/mysql/keys/repo.percona.com (+30/-0)
tests/trusty/mysql/metadata.yaml (+43/-0)
tests/trusty/mysql/monitors.yaml (+13/-0)
tests/trusty/mysql/revision (+1/-0)
tests/trusty/mysql/scripts/add_to_cluster (+13/-0)
tests/trusty/mysql/scripts/charm_helpers_sync.py (+225/-0)
tests/trusty/mysql/scripts/mysql_backup.sh (+30/-0)
tests/trusty/mysql/scripts/remove_from_cluster (+4/-0)
tests/trusty/mysql/templates/apparmor.j2 (+15/-0)
tests/trusty/mysql/templates/mysql_backup.j2 (+12/-0)
tests/trusty/mysql/tests/00-setup (+12/-0)
tests/trusty/mysql/tests/15-configs (+77/-0)
tests/trusty/mysql/unit_tests/test_mysql_common.py (+18/-0)
tests/trusty/tester/README.md (+1/-0)
tests/trusty/tester/composer.yaml (+8/-0)
tests/trusty/tester/generate/custom.py (+17/-0)
tests/trusty/tester/hooks/start (+1/-0)
tests/trusty/tester/metadata.yaml (+14/-0)
tests/trusty/use-layers/README.md (+1/-0)
tests/trusty/use-layers/composer.yaml (+1/-0)
tests/trusty/use-layers/hooks/reactive/main.py (+6/-0)
tests_functional/add/test.sh (+2/-2)
tests_functional/create/test.sh (+4/-2)
tests_functional/proof/record.sh (+1/-1)
tox.ini (+21/-0)
To merge this branch: bzr merge lp://staging/~bcsaller/charm-tools/composer
Reviewer Review Type Date Requested Status
Tim Van Steenburgh (community) Approve
Cory Johns (community) Needs Fixing
Marco Ceppi Pending
Review via email: mp+266281@code.staging.launchpad.net

Description of the change

This adds the composer stuff
and ports the probject to use tox

To post a comment you must log in.
Revision history for this message
Cory Johns (johnsca) wrote :

Need to add blessings, ruamel.yaml, pathspec, and bundletester to the install_requires in setup.py

Revision history for this message
Cory Johns (johnsca) :
review: Needs Fixing
Revision history for this message
Adam Israel (aisrael) wrote :

Hi Ben,

Per our earlier conversation, I'd also like to see some documentation on how a charm author would consider and use composer. I'm very excited to see that myself, and give composer a spin.

358. By Benjamin Saller

merge lp:~johnsca/charm-tools/compose

359. By Benjamin Saller

various composer fixes

360. By Benjamin Saller

without the pdb

361. By Benjamin Saller

show paths and so on with -l DEBUG, tests do less actual remote work

362. By Benjamin Saller

compose cli help a little better

363. By Benjamin Saller

fix bzrignore

364. By Benjamin Saller

fix typo

Revision history for this message
Charles Butler (lazypower) wrote :

Sorry this took me so long to circle back, but I've tried this branch of charm tools and it appears the manifest is not fetching all the dependencies.

What i did:

bzr branch lp:~bcsaller/charm-tools/composer/
virtualenv .venv
source .venv/bin/activate

pip install ./

charm compose -h
ImportError: No module named path

pip install path.py
charm compose -h
ImportError: No module named otherstuf

pip install otherstuf
charm compose -h

OSError: [Errno 2] No such file or directory: '/home/charles/projects/work/composer/.venv/local/lib/python2.7/site-packages/charmtools/compose/../../doc/source/compose-intro.md'

Once I ran through that dependency hoop, compose appears to be sorted and available.

Revision history for this message
Cory Johns (johnsca) wrote :

Fix for the missing deps and help error here: https://code.launchpad.net/~johnsca/charm-tools/compose/+merge/268164

Revision history for this message
Cory Johns (johnsca) wrote :

Updated my most recent MP to fix the default value for --name when composing current dir (e.g., "charm compose ." or just "charm compose")

365. By Benjamin Saller

fix deps in setup, better seaching for charm.name

366. By Benjamin Saller

change default interface address to public one

367. By Benjamin Saller

update to work with real DNS and service

368. By Benjamin Saller

remove find name call

369. By Benjamin Saller

various fixes around naming and patch to install to ignore existing deps, also depend on modern pip

370. By Benjamin Saller

repair tests

371. By Benjamin Saller

fix/remove broken tests w/updated bundletester

372. By Benjamin Saller

patch for inspect to work with more varied naming

373. By Benjamin Saller

force key order on metadata.yaml merges

374. By Benjamin Saller

fix tests (to reflect remote changes to basic layer) and ordered metadata rendering

375. By Benjamin Saller

update notes on workflow

376. By Benjamin Saller

various fixes and cleanups. Change installer to do better signing

377. By Benjamin Saller

merge trunk

Revision history for this message
Tim Van Steenburgh (tvansteenburgh) wrote :
Download full text (9.0 KiB)

I'm eager to merge this but can't get the tests to run. I always run `make check` on charm-tools since that includes the integration tests as well. According to the Makefile that should still work, but I get:

tvansteenburgh@trusty-vm:/tmp/charm-tools> make clean
find . -name '*.py[co]' -delete
find . -type f -name '*~' -delete
find . -name '*.bak' -delete
rm -rf bin include lib local man dependencies
tvansteenburgh@trusty-vm:/tmp/charm-tools> make check
bzr checkout lp:~juju-jitsu/charm-tools/dependencies
tox --develop
py27 create: /tmp/charm-tools/.tox/py27
py27 installdeps: -r/tmp/charm-tools/requirements.txt
ERROR: invocation failed, logfile: /tmp/charm-tools/.tox/py27/log/py27-1.log
ERROR: actionid=py27
msg=getenv
cmdargs=[local('/tmp/charm-tools/.tox/py27/bin/pip'), 'install', '--no-index', '-f', 'dependencies/python', '-r/tmp/charm-tools/requirements.txt']
env={'BYOBU_TTY': '/dev/pts/1', 'UPSTART_EVENTS': 'started starting', 'SHELL': '/bin/bash', 'XDG_DATA_DIRS': '/usr/share/ubuntu:/usr/share/gnome:/usr/local/share/:/usr/share/', 'MANDATORY_PATH': '/usr/share/gconf/ubuntu.mandatory.path', 'CLUTTER_IM_MODULE': 'xim', 'BYOBU_RUN_DIR': '/dev/shm/byobu-tvansteenburgh-KRSOmrmO', 'UPSTART_INSTANCE': '', 'JOB': 'gnome-session', 'TEXTDOMAIN': 'im-config', 'XMODIFIERS': '@im=ibus', 'MFLAGS': '', 'SELINUX_INIT': 'YES', 'BYOBU_SED': 'sed', 'BYOBU_LIGHT': '#EEEEEE', 'DESKTOP_SESSION': 'ubuntu', 'BYOBU_DATE': '%Y-%m-%d ', 'XDG_SESSION_ID': 'c1', 'DBUS_SESSION_BUS_ADDRESS': 'unix:abstract=/tmp/dbus-m7XWOMt928', 'DEFAULTS_PATH': '/usr/share/gconf/ubuntu.default.path', 'LESS_TERMCAP_ue': '\x1b[0m', 'GTK_MODULES': 'overlay-scrollbar:unity-gtk-module', 'INSTANCE': 'Unity', 'LESS_TERMCAP_us': '\x1b[04;38;5;139m', 'LS_COLORS': 'rs=0:di=38;5;5:ln=4;5;37:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=3;28:ow=34;42:st=37;44:ex=38;5;202:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=...

Read more...

review: Needs Fixing
Revision history for this message
Tim Van Steenburgh (tvansteenburgh) wrote :
Download full text (13.6 KiB)

I got integration tests to run by changing this in the Makefile:

-build: deps develop
+build: deps

But then I got a bunch of legit failures from the tests:

2 tvansteenburgh@trusty-vm:/tmp/charm-tools> make check
tests_functional/helpers/helpers.sh || sh -x tests_functional/helpers/helpers.sh timeout
Fri Aug 28 11:04:24 EDT 2015: Testing ch_apparmor_load...PASS
Fri Aug 28 11:04:24 EDT 2015: Testing ch_type_hash...PASS
Fri Aug 28 11:04:24 EDT 2015: Testing ch_is_url...PASS
Fri Aug 28 11:04:24 EDT 2015: Testing Testing ch_is_ip...PASS
Fri Aug 28 11:04:24 EDT 2015: Testing Testing ch_get_ip...PASS
Fri Aug 28 11:04:24 EDT 2015: Starting SimpleHTTPServer in /tmp/charm-helper-srv.BlmbCS on port 8999 to test fetching files.
Fri Aug 28 11:04:24 EDT 2015: Looping wget until webserver responds...
Fri Aug 28 11:04:25 EDT 2015: Attempt 1 succeeded.
Fri Aug 28 11:04:25 EDT 2015: Creating temp data file
Fri Aug 28 11:04:25 EDT 2015: creating gzipped test data
Fri Aug 28 11:04:25 EDT 2015: Testing ch_get_file...PASS
Fri Aug 28 11:04:25 EDT 2015: Shutting down webserver...DONE
Fri Aug 28 11:04:25 EDT 2015: Printing server log
127.0.0.1 - - [28/Aug/2015 11:04:25] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [28/Aug/2015 11:04:25] "GET /testdata.txt HTTP/1.1" 200 -
127.0.0.1 - - [28/Aug/2015 11:04:25] "GET /testdata.txt HTTP/1.1" 200 -
127.0.0.1 - - [28/Aug/2015 11:04:25] "GET /testdata.txt HTTP/1.1" 200 -
127.0.0.1 - - [28/Aug/2015 11:04:25] "GET /testdata.txt.gz HTTP/1.1" 200 -
127.0.0.1 - - [28/Aug/2015 11:04:25] "GET /testdata.txt.gz HTTP/1.1" 200 -
127.0.0.1 - - [28/Aug/2015 11:04:25] "GET /testdata.txt.gz HTTP/1.1" 200 -
127.0.0.1 - - [28/Aug/2015 11:04:25] "GET /testdata.txt HTTP/1.1" 200 -
Fri Aug 28 11:04:25 EDT 2015: Testing ch_unit_name...PASS
Fri Aug 28 11:04:25 EDT 2015: Testing ch_unit_id...PASS
Fri Aug 28 11:04:25 EDT 2015: Testing ch_my_unit_id...PASS
Test shell helpers with bash
bash tests_functional/helpers/helpers.sh \
            || bash -x tests_functional/helpers/helpers.sh timeout
Fri Aug 28 11:04:25 EDT 2015: Testing ch_apparmor_load...PASS
Fri Aug 28 11:04:25 EDT 2015: Testing ch_type_hash...PASS
Fri Aug 28 11:04:25 EDT 2015: Testing ch_is_url...PASS
Fri Aug 28 11:04:25 EDT 2015: Testing Testing ch_is_ip...PASS
Fri Aug 28 11:04:25 EDT 2015: Testing Testing ch_get_ip...PASS
Fri Aug 28 11:04:25 EDT 2015: Starting SimpleHTTPServer in /tmp/charm-helper-srv.TOq187 on port 8999 to test fetching files.
Fri Aug 28 11:04:25 EDT 2015: Looping wget until webserver responds...
Fri Aug 28 11:04:26 EDT 2015: Attempt 1 succeeded.
Fri Aug 28 11:04:26 EDT 2015: Creating temp data file
Fri Aug 28 11:04:26 EDT 2015: creating gzipped test data
Fri Aug 28 11:04:26 EDT 2015: Testing ch_get_file...PASS
Fri Aug 28 11:04:26 EDT 2015: Shutting down webserver...DONE
Fri Aug 28 11:04:26 EDT 2015: Printing server log
127.0.0.1 - - [28/Aug/2015 11:04:26] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [28/Aug/2015 11:04:26] "GET /testdata.txt HTTP/1.1" 200 -
127.0.0.1 - - [28/Aug/2015 11:04:26] "GET /testdata.txt HTTP/1.1" 200 -
127.0.0.1 - - [28/Aug/2015 11:04:26] "GET /testdata.txt HTTP/1.1" 200 -
127.0.0.1 - - [28/Aug/2015 11:04:26] "GET /testdata.txt.gz HTTP/1.1" 200 -
127.0.0...

review: Needs Fixing
378. By Benjamin Saller

merge trunk

379. By Benjamin Saller

path fixes on integration tests

Revision history for this message
Benjamin Saller (bcsaller) wrote :

should be fixed, very minor pathing issues in the tests to point to the tox virtualenv

380. By Benjamin Saller

notest on develop

381. By Benjamin Saller

pass term env

382. By Benjamin Saller

move passenv

Revision history for this message
Tim Van Steenburgh (tvansteenburgh) wrote :

Tests pass after upgrading tox to 2.1.1 (1.6.0 failed), LGTM.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
The diff is not available at this time. You can reload the page or download it.

Subscribers

People subscribed via source and target branches