Juju Charms Collection
ceph-osd package

Merge lp://staging/~xfactor973/charms/trusty/ceph-osd/sysctl-perf into lp://staging/~openstack-charmers-archive/charms/trusty/ceph-osd/next

  1. Trusty Tahr (14.04)
  2. sysctl-perf
  3. Merge into next
Proposed by James Page
Status: Needs review
Proposed branch: lp://staging/~xfactor973/charms/trusty/ceph-osd/sysctl-perf
Merge into: lp://staging/~openstack-charmers-archive/charms/trusty/ceph-osd/next
Diff against target: 2895 lines (+2241/-11) (has conflicts)
36 files modified
.bzrignore (+3/-0)
.coveragerc (+7/-0)
.testr.conf (+8/-0)
Makefile (+3/-0)
charm-helpers-hooks.yaml (+1/-1)
charm-helpers-tests.yaml (+1/-1)
config.yaml (+10/-1)
hooks/ceph.py (+241/-3)
hooks/ceph_hooks.py (+279/-0)
hooks/charmhelpers/cli/__init__.py (+191/-0)
hooks/charmhelpers/cli/benchmark.py (+36/-0)
hooks/charmhelpers/cli/commands.py (+32/-0)
hooks/charmhelpers/cli/hookenv.py (+23/-0)
hooks/charmhelpers/cli/host.py (+31/-0)
hooks/charmhelpers/cli/unitdata.py (+39/-0)
hooks/charmhelpers/core/files.py (+45/-0)
hooks/charmhelpers/core/hookenv.py (+238/-0)
hooks/charmhelpers/core/host.py (+92/-0)
hooks/charmhelpers/core/hugepage.py (+62/-0)
hooks/charmhelpers/core/services/helpers.py (+26/-0)
hooks/charmhelpers/fetch/__init__.py (+29/-0)
hooks/charmhelpers/fetch/giturl.py (+17/-0)
hooks/install (+23/-0)
requirements.txt (+11/-0)
setup.cfg (+5/-0)
templates/ceph.conf (+5/-0)
test-requirements.txt (+9/-0)
tests/README (+25/-0)
tests/basic_deployment.py (+15/-5)
tests/charmhelpers/contrib/amulet/utils.py (+194/-0)
tests/charmhelpers/contrib/openstack/amulet/deployment.py (+41/-0)
tests/charmhelpers/contrib/openstack/amulet/utils.py (+273/-0)
tests/tests.yaml (+20/-0)
tox.ini (+29/-0)
unit_tests/test_status.py (+56/-0)
unit_tests/test_utils.py (+121/-0)
Conflict adding file .coveragerc.  Moved existing file to .coveragerc.moved.
Conflict adding file .testr.conf.  Moved existing file to .testr.conf.moved.
Text conflict in Makefile
Text conflict in hooks/ceph.py
Path conflict: hooks/ceph_hooks.py / <deleted>
Conflict adding file hooks/ceph_hooks.py.  Moved existing file to hooks/ceph_hooks.py.moved.
Conflict adding file hooks/charmhelpers/cli.  Moved existing file to hooks/charmhelpers/cli.moved.
Conflict adding file hooks/charmhelpers/core/files.py.  Moved existing file to hooks/charmhelpers/core/files.py.moved.
Text conflict in hooks/charmhelpers/core/hookenv.py
Text conflict in hooks/charmhelpers/core/host.py
Conflict adding file hooks/charmhelpers/core/hugepage.py.  Moved existing file to hooks/charmhelpers/core/hugepage.py.moved.
Text conflict in hooks/charmhelpers/core/services/helpers.py
Text conflict in hooks/charmhelpers/fetch/__init__.py
Text conflict in hooks/charmhelpers/fetch/giturl.py
Text conflict in hooks/install
Conflict adding file hooks/install.real.  Moved existing file to hooks/install.real.moved.
Conflict adding file hooks/update-status.  Moved existing file to hooks/update-status.moved.
Conflict adding file requirements.txt.  Moved existing file to requirements.txt.moved.
Conflict adding file setup.cfg.  Moved existing file to setup.cfg.moved.
Conflict adding file test-requirements.txt.  Moved existing file to test-requirements.txt.moved.
Text conflict in tests/README
Text conflict in tests/basic_deployment.py
Text conflict in tests/charmhelpers/contrib/amulet/utils.py
Text conflict in tests/charmhelpers/contrib/openstack/amulet/deployment.py
Text conflict in tests/charmhelpers/contrib/openstack/amulet/utils.py
Conflict adding file tests/tests.yaml.  Moved existing file to tests/tests.yaml.moved.
Conflict adding file tox.ini.  Moved existing file to tox.ini.moved.
Conflict adding file unit_tests/test_status.py.  Moved existing file to unit_tests/test_status.py.moved.
Conflict adding file unit_tests/test_utils.py.  Moved existing file to unit_tests/test_utils.py.moved.
To merge this branch: bzr merge lp://staging/~xfactor973/charms/trusty/ceph-osd/sysctl-perf
Reviewer Review Type Date Requested Status
James Page Pending
Review via email: mp+286492@code.staging.launchpad.net

This proposal supersedes a proposal from 2016-01-26.

Description of the change

This branch adds some HDD and network adapter sysctl tuning. Lots of potential performance has been left on the table with our out of the box ceph setup. This aims to recapture some of that.

For the HDD portion: Linux has set very conservative settings when it comes to HDDs. While this is great for desktop responsiveness it isn't so kind to storage servers. I have exposed a configuration option if an administrator knows they have a RAID card with a lot of cache onboard. In a future patch set I will add some SSD tuning to take advantage of its unique characteristics.

Testing this has proven challenging. While it doesn't break anything on AWS that doesn't really prove anything either. If anyone has access to physical hardware to test this I would really appreciate it.

To post a comment you must log in.
Revision history for this message
Chris MacNaughton (chris.macnaughton) wrote : Posted in a previous version of this proposal

See comment on network interfaces. I'd also like to see some validation on varied hardware (SSD / HDD) that this doesn't break things either way.

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_lint_check #18194 ceph-osd for xfactor973 mp284033
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
make: *** [lint] Error 1
ERROR:root:Make target returned non-zero.

Full lint test output: http://paste.ubuntu.com/14678289/
Build: http://10.245.162.77:8080/job/charm_lint_check/18194/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_unit_test #16953 ceph-osd for xfactor973 mp284033
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/16953/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_amulet_test #9068 ceph-osd for xfactor973 mp284033
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/14678328/
Build: http://10.245.162.77:8080/job/charm_amulet_test/9068/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_unit_test #16956 ceph-osd for xfactor973 mp284033
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/16956/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_lint_check #18198 ceph-osd for xfactor973 mp284033
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/18198/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_amulet_test #9071 ceph-osd for xfactor973 mp284033
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/14680501/
Build: http://10.245.162.77:8080/job/charm_amulet_test/9071/

Revision history for this message
Chris Holcombe (xfactor973) wrote : Posted in a previous version of this proposal

@chris.macnaughton Ask and ye shall receive ;). This keys off of link speed now

Revision history for this message
Chris Holcombe (xfactor973) wrote : Posted in a previous version of this proposal

Ignore the stupid line nits. That's from me using auto format in pycharms IDE

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_unit_test #113 ceph-osd for xfactor973 mp284033
    UNIT FAIL: unit-test failed

UNIT Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full unit test output: http://paste.ubuntu.com/15002338/
Build: http://10.245.162.36:8080/job/charm_unit_test/113/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_lint_check #130 ceph-osd for xfactor973 mp284033
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
make: *** [lint] Error 1
ERROR:root:Make target returned non-zero.

Full lint test output: http://paste.ubuntu.com/15002365/
Build: http://10.245.162.36:8080/job/charm_lint_check/130/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_amulet_test #12 ceph-osd for xfactor973 mp284033
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/15002518/
Build: http://10.245.162.36:8080/job/charm_amulet_test/12/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_unit_test #132 ceph-osd for xfactor973 mp284033
    UNIT FAIL: unit-test failed

UNIT Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full unit test output: http://paste.ubuntu.com/15002739/
Build: http://10.245.162.36:8080/job/charm_unit_test/132/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_lint_check #140 ceph-osd for xfactor973 mp284033
    LINT OK: passed

Build: http://10.245.162.36:8080/job/charm_lint_check/140/

Revision history for this message
Ryan Beisner (1chb1n) wrote : Posted in a previous version of this proposal

@xfactor973 The tests/* dir affects only Amulet test runs, and the 00-setup file is batch-maintained (may be overwritten).

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_amulet_test #27 ceph-osd for xfactor973 mp284033
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/15004320/
Build: http://10.245.162.36:8080/job/charm_amulet_test/27/

Revision history for this message
James Page (james-page) wrote : Posted in a previous version of this proposal

Amulet test failure is genuine; install hook fails on 12.04:

2016-02-09 19:55:55 INFO install After this operation, 1467 kB of additional disk space will be used.
2016-02-09 19:55:55 INFO install Get:1 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise/main python-setuptools all 0.6.24-1ubuntu1 [441 kB]
2016-02-09 19:55:55 INFO install Get:2 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise/universe python-pip all 1.0-1build1 [95.1 kB]
2016-02-09 19:55:55 INFO install Fetched 536 kB in 0s (25.1 MB/s)
2016-02-09 19:55:55 INFO install Selecting previously unselected package python-setuptools.
2016-02-09 19:55:55 INFO install (Reading database ... 47935 files and directories currently installed.)
2016-02-09 19:55:55 INFO install Unpacking python-setuptools (from .../python-setuptools_0.6.24-1ubuntu1_all.deb) ...
2016-02-09 19:55:55 INFO install Selecting previously unselected package python-pip.
2016-02-09 19:55:55 INFO install Unpacking python-pip (from .../python-pip_1.0-1build1_all.deb) ...
2016-02-09 19:55:55 INFO install Processing triggers for man-db ...
2016-02-09 19:55:55 INFO install Setting up python-setuptools (0.6.24-1ubuntu1) ...
2016-02-09 19:55:55 INFO install Setting up python-pip (1.0-1build1) ...
2016-02-09 19:55:56 INFO install Package `python-enum34' is not installed and no info is available.
2016-02-09 19:55:56 INFO install Use dpkg --info (= dpkg-deb --info) to examine archive files,
2016-02-09 19:55:56 INFO install and dpkg --contents (= dpkg-deb --contents) to list their contents.
2016-02-09 19:55:56 INFO install Reading package lists...
2016-02-09 19:55:56 INFO install Building dependency tree...
2016-02-09 19:55:56 INFO install Reading state information...
2016-02-09 19:55:57 INFO install E: Unable to locate package python-enum34
2016-02-09 19:55:57 INFO install Traceback (most recent call last):
2016-02-09 19:55:57 INFO install File "/var/lib/juju/agents/unit-ceph-osd-0/charm/hooks/install.real", line 16, in <module>
2016-02-09 19:55:57 INFO install import ceph
2016-02-09 19:55:57 INFO install File "/var/lib/juju/agents/unit-ceph-osd-0/charm/hooks/ceph.py", line 9, in <module>
2016-02-09 19:55:57 INFO install from enum import Enum
2016-02-09 19:55:57 INFO install ImportError: No module named enum

review: Needs Fixing
Revision history for this message
James Page (james-page) wrote : Posted in a previous version of this proposal

I'll have to defer to your knowledge on nic and block device tuning as I can't directly validate that - one question - are the changes make to interface and block device parameters persistent? I don't think that they are, so a server reboot will hit reset on tuned params.

review: Needs Information
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_unit_test #315 ceph-osd for xfactor973 mp284033
    UNIT FAIL: unit-test failed

UNIT Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full unit test output: http://paste.ubuntu.com/15031007/
Build: http://10.245.162.36:8080/job/charm_unit_test/315/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_lint_check #395 ceph-osd for xfactor973 mp284033
    LINT OK: passed

Build: http://10.245.162.36:8080/job/charm_lint_check/395/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_amulet_test #166 ceph-osd for xfactor973 mp284033
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/15031371/
Build: http://10.245.162.36:8080/job/charm_amulet_test/166/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_unit_test #373 ceph-osd for xfactor973 mp284033
    UNIT FAIL: unit-test failed

UNIT Results (max last 2 lines):
make: *** [test] Error 1
ERROR:root:Make target returned non-zero.

Full unit test output: http://paste.ubuntu.com/15032005/
Build: http://10.245.162.36:8080/job/charm_unit_test/373/

Revision history for this message
Chris Holcombe (xfactor973) wrote : Posted in a previous version of this proposal

I dumped the enum dependency :)

On 02/11/2016 05:46 AM, James Page wrote:
> Review: Needs Fixing
>
> Amulet test failure is genuine; install hook fails on 12.04:
>
> 2016-02-09 19:55:55 INFO install After this operation, 1467 kB of additional disk space will be used.
> 2016-02-09 19:55:55 INFO install Get:1 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise/main python-setuptools all 0.6.24-1ubuntu1 [441 kB]
> 2016-02-09 19:55:55 INFO install Get:2 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise/universe python-pip all 1.0-1build1 [95.1 kB]
> 2016-02-09 19:55:55 INFO install Fetched 536 kB in 0s (25.1 MB/s)
> 2016-02-09 19:55:55 INFO install Selecting previously unselected package python-setuptools.
> 2016-02-09 19:55:55 INFO install (Reading database ... 47935 files and directories currently installed.)
> 2016-02-09 19:55:55 INFO install Unpacking python-setuptools (from .../python-setuptools_0.6.24-1ubuntu1_all.deb) ...
> 2016-02-09 19:55:55 INFO install Selecting previously unselected package python-pip.
> 2016-02-09 19:55:55 INFO install Unpacking python-pip (from .../python-pip_1.0-1build1_all.deb) ...
> 2016-02-09 19:55:55 INFO install Processing triggers for man-db ...
> 2016-02-09 19:55:55 INFO install Setting up python-setuptools (0.6.24-1ubuntu1) ...
> 2016-02-09 19:55:55 INFO install Setting up python-pip (1.0-1build1) ...
> 2016-02-09 19:55:56 INFO install Package `python-enum34' is not installed and no info is available.
> 2016-02-09 19:55:56 INFO install Use dpkg --info (= dpkg-deb --info) to examine archive files,
> 2016-02-09 19:55:56 INFO install and dpkg --contents (= dpkg-deb --contents) to list their contents.
> 2016-02-09 19:55:56 INFO install Reading package lists...
> 2016-02-09 19:55:56 INFO install Building dependency tree...
> 2016-02-09 19:55:56 INFO install Reading state information...
> 2016-02-09 19:55:57 INFO install E: Unable to locate package python-enum34
> 2016-02-09 19:55:57 INFO install Traceback (most recent call last):
> 2016-02-09 19:55:57 INFO install File "/var/lib/juju/agents/unit-ceph-osd-0/charm/hooks/install.real", line 16, in <module>
> 2016-02-09 19:55:57 INFO install import ceph
> 2016-02-09 19:55:57 INFO install File "/var/lib/juju/agents/unit-ceph-osd-0/charm/hooks/ceph.py", line 9, in <module>
> 2016-02-09 19:55:57 INFO install from enum import Enum
> 2016-02-09 19:55:57 INFO install ImportError: No module named enum
>
>

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_lint_check #669 ceph-osd for xfactor973 mp284033
    LINT OK: passed

Build: http://10.245.162.36:8080/job/charm_lint_check/669/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_unit_test #579 ceph-osd for xfactor973 mp284033
    UNIT OK: passed

Build: http://10.245.162.36:8080/job/charm_unit_test/579/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote : Posted in a previous version of this proposal

charm_amulet_test #277 ceph-osd for xfactor973 mp284033
    AMULET OK: passed

Build: http://10.245.162.36:8080/job/charm_amulet_test/277/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #730 ceph-osd-next for james-page mp286492
    UNIT OK: passed

Build: http://10.245.162.36:8080/job/charm_unit_test/730/

Reply
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #829 ceph-osd-next for james-page mp286492
    LINT OK: passed

Build: http://10.245.162.36:8080/job/charm_lint_check/829/

Reply
Revision history for this message
James Page (james-page) wrote : Posted in a previous version of this proposal

Some suggested refactoring to use charmhelpers.core.sysctl to manage the nic settings.

Also I'm not sure that max_sectors_kb is reboot persistent yet

review: Needs Fixing
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #329 ceph-osd-next for james-page mp286492
    AMULET OK: passed

Build: http://10.245.162.36:8080/job/charm_amulet_test/329/

Reply

Unmerged revisions

56. By Chris Holcombe

Removed enum dependency

55. By Chris Holcombe

Pull in the tox files from next

54. By Chris Holcombe

Persist the changes for reboots. Also catch IOError in all the posix operations

53. By Chris Holcombe

Address the lint issues and try to solve the python-enum34 package missing with the unit tests

52. By Chris Holcombe

Change over to using network link speed as the key to sysctl tuning instead of the driver name. This is a more generic approach because most 10Gb links need roughly the same set of sysctls

51. By Chris Holcombe

Lint spacing

50. By Chris Holcombe

Add a status notification to the network adapter tuning section

49. By Chris Holcombe

Added possible network adapter sysctl tuning

48. By Chris Holcombe

Add a config flag and a function to tune the read and write sectors for each osd block device

47. By Chris Holcombe

Sysctl and performance tweaks for ceph

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
The diff is not available at this time. You can reload the page or download it.

Subscribers

People subscribed via source and target branches