We still need the 100G disk for Jenkins because we can have many different projects and builds and each one requires a LXC instance, which takes space, so in theory and with my bad calculation we could have about 75 containers. I think there's an user story about me adding storage support for /var/lib/lxc somewhere, this would solve the problem and we could reduce the disk size to, say, 10G. On Fri, Mar 13, 2015 at 1:26 PM, Daniel Manrique wrote: > Some diff comments, see below. > > Diff comments: > >> === added file 'docs/doc-requirements.txt' >> --- docs/doc-requirements.txt 1970-01-01 00:00:00 +0000 >> +++ docs/doc-requirements.txt 2015-03-12 17:53:21 +0000 >> @@ -0,0 +1,2 @@ >> +sphinx >> +sphinxcontrib-blockdiag >> >> === added file 'docs/source/apps.dot' >> --- docs/source/apps.dot 1970-01-01 00:00:00 +0000 >> +++ docs/source/apps.dot 2015-03-12 17:53:21 +0000 >> @@ -0,0 +1,285 @@ >> +digraph model_graph { >> + // Dotfile by Django-Extensions graph_models >> + // Created: 2015-03-11 19:21 >> + // Cli Options: --pygraphviz -d -g credentials jenkins archives projects >> + >> + fontname = "Helvetica" >> + fontsize = 8 >> + splines = true >> + >> + node [ >> + fontname = "Helvetica" >> + fontsize = 8 >> + shape = "plaintext" >> + ] >> + >> + edge [ >> + fontname = "Helvetica" >> + fontsize = 8 >> + ] >> + >> + // Labels >> + subgraph cluster_credentials_models { >> + label=< >> + >> + >> +

>> + >> + credentials >> + >> +

>> + > >> + color=olivedrab4 >> + style="rounded" >> + >> + credentials_models_SshKeyPair [label=< >> + >> + >> + >> +

>> + >> + SshKeyPair >> +

>> + >] >> + >> + } >> + subgraph cluster_jenkins_models { >> + label=< >> + >> + >> +

>> + >> + jenkins >> + >> +

>> + > >> + color=olivedrab4 >> + style="rounded" >> + >> + jenkins_models_JenkinsServer [label=< >> + >> + >> + >> +

>> + >> + JenkinsServer >> +

>> + >] >> + >> + jenkins_models_JobType [label=< >> + >> + >> + >> +

>> + >> + JobType >> +

>> + >] >> + >> + jenkins_models_Job [label=< >> + >> + >> + >> +

>> + >> + Job >> +

>> + >] >> + >> + jenkins_models_Build [label=< >> + >> + >> + >> +

>> + >> + Build >> +

>> + >] >> + >> + jenkins_models_Artifact [label=< >> + >> + >> + >> +

>> + >> + Artifact >> +

>> + >] >> + >> + } >> + subgraph cluster_archives_models { >> + label=< >> + >> + >> +

>> + >> + archives >> + >> +

>> + > >> + color=olivedrab4 >> + style="rounded" >> + >> + archives_models_Archive [label=< >> + >> + >> + >> +

>> + >> + Archive >> +

>> + >] >> + >> + archives_models_ArchiveArtifact [label=< >> + >> + >> + >> +

>> + >> + ArchiveArtifact >> +

>> + >] >> + >> + } >> + subgraph cluster_projects_models { >> + label=< >> + >> + >> +

>> + >> + projects >> + >> +

>> + > >> + color=olivedrab4 >> + style="rounded" >> + >> + projects_models_Dependency [label=< >> + >> + >> + >> +

>> + >> + Dependency >> +

>> + >] >> + >> + projects_models_ProjectDependency [label=< >> + >> + >> + >> +

>> + >> + ProjectDependency >> +

>> + >] >> + >> + projects_models_Project [label=< >> + >> + >> + >> +

>> + >> + Project >> +

>> + >] >> + >> + projects_models_ProjectBuildDependency [label=< >> + >> + >> + >> +

>> + >> + ProjectBuildDependency >> +

>> + >] >> + >> + projects_models_ProjectBuild [label=< >> + >> + >> + >> +

>> + >> + ProjectBuild >> +

>> + >] >> + >> + } >> + >> + >> + // Relations >> + >> + jenkins_models_Job -> jenkins_models_JenkinsServer >> + [label="server (job)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + jenkins_models_Job -> jenkins_models_JobType >> + [label="jobtype (job)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + jenkins_models_Build -> jenkins_models_Job >> + [label="job (build)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + django_contrib_auth_models_User [label=< >> + >> + >> +

>> + User >> +

>> + >] >> + jenkins_models_Build -> django_contrib_auth_models_User >> + [label="requested_by (build)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + jenkins_models_Artifact -> jenkins_models_Build >> + [label="build (artifact)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + archives_models_Archive -> credentials_models_SshKeyPair >> + [label="ssh_credentials (archive)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + archives_models_ArchiveArtifact -> archives_models_Archive >> + [label="archive (items)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + archives_models_ArchiveArtifact -> jenkins_models_Artifact >> + [label="artifact (archiveartifact)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + archives_models_ArchiveArtifact -> jenkins_models_Build >> + [label="build (archiveartifact)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + archives_models_ArchiveArtifact -> projects_models_ProjectBuildDependency >> + [label="projectbuild_dependency (archiveartifact)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + archives_models_ArchiveArtifact -> projects_models_Dependency >> + [label="dependency (archiveartifact)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + projects_models_Dependency -> jenkins_models_Job >> + [label="job (dependency)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + projects_models_ProjectDependency -> projects_models_Dependency >> + [label="dependency (projectdependency)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + projects_models_ProjectDependency -> projects_models_Project >> + [label="project (projectdependency)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + projects_models_ProjectDependency -> jenkins_models_Build >> + [label="current_build (projectdependency)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + projects_models_ProjectBuildDependency -> projects_models_ProjectBuild >> + [label="projectbuild (dependencies)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + projects_models_ProjectBuildDependency -> jenkins_models_Build >> + [label="build (projectbuild_dependencies)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + projects_models_ProjectBuildDependency -> projects_models_Dependency >> + [label="dependency (projectbuilddependency)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + projects_models_ProjectBuild -> projects_models_Project >> + [label="project (projectbuild)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + django_contrib_auth_models_User [label=< >> + >> + >> +

>> + User >> +

>> + >> + archives >> + >> +

>> + > >> + color=olivedrab4 >> + style="rounded" >> + >> + archives_models_Archive [label=< >> + >> + >> + >> +

>> + >> + Archive >> +

>> + >] >> + >> + archives_models_ArchiveArtifact [label=< >> + >> + >> + >> +

>> + >> + ArchiveArtifact >> +

>> + >] >> + >> + } >> + >> + >> + // Relations >> + credentials_models_SshKeyPair [label=< >> + >> + >> +

>> + SshKeyPair >> +

>> + >] >> + archives_models_Archive -> credentials_models_SshKeyPair >> + [label="ssh_credentials (archive)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + archives_models_ArchiveArtifact -> archives_models_Archive >> + [label="archive (items)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + jenkins_models_Artifact [label=< >> + >> + >> +

>> + Artifact >> +

>> + >] >> + archives_models_ArchiveArtifact -> jenkins_models_Artifact >> + [label="artifact (archiveartifact)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + jenkins_models_Build [label=< >> + >> + >> +

>> + Build >> +

>> + >] >> + archives_models_ArchiveArtifact -> jenkins_models_Build >> + [label="build (archiveartifact)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + projects_models_ProjectBuildDependency [label=< >> + >> + >> +

>> + ProjectBuildDependency >> +

>> + >] >> + archives_models_ArchiveArtifact -> projects_models_ProjectBuildDependency >> + [label="projectbuild_dependency (archiveartifact)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + projects_models_Dependency [label=< >> + >> + >> +

>> + Dependency >> +

>> + >] >> + archives_models_ArchiveArtifact -> projects_models_Dependency >> + [label="dependency (archiveartifact)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> +} >> + >> >> === modified file 'docs/source/archives.rst' >> --- docs/source/archives.rst 2014-10-27 12:40:23 +0000 >> +++ docs/source/archives.rst 2015-03-12 17:53:21 +0000 >> @@ -50,4 +50,9 @@ >> >> ErrorLog ${APACHE_LOG_DIR}/error.log >> CustomLog ${APACHE_LOG_DIR}/access.log combined >> - >> \ No newline at end of file >> + >> + >> +Archives Data Model >> +------------------- >> + >> +.. graphviz:: archives.dot >> >> === modified file 'docs/source/conf.py' >> --- docs/source/conf.py 2014-10-27 12:40:23 +0000 >> +++ docs/source/conf.py 2015-03-12 17:53:21 +0000 >> @@ -31,6 +31,9 @@ >> extensions = [ >> 'sphinx.ext.autodoc', >> 'sphinx.ext.todo', >> + 'sphinx.ext.graphviz', >> + 'sphinxcontrib.blockdiag', >> + 'alabaster', >> ] >> >> # Add any paths that contain templates here, relative to this directory. >> @@ -101,12 +104,19 @@ >> >> # The theme to use for HTML and HTML Help pages. See the documentation for >> # a list of builtin themes. >> -html_theme = 'default' >> +html_theme = 'alabaster' >> >> # Theme options are theme-specific and customize the look and feel of a theme >> # further. For a list of options available for each theme, see the >> # documentation. >> -#html_theme_options = {} >> +html_theme_options = { >> + 'description': "Jenkin's best friend", > > Should be "Jenkins' " I think. > >> + 'github_button': False, >> + 'extra_nav_links': { >> + 'email': 'mailto:', >> + 'lp:capomastro': 'https://launchpad.net/capomastro', >> + } >> +} >> >> # Add any paths that contain custom themes here, relative to this directory. >> #html_theme_path = [] >> @@ -146,7 +156,13 @@ >> #html_use_smartypants = True >> >> # Custom sidebar templates, maps document names to template names. >> -#html_sidebars = {} >> +html_sidebars = { >> + '**': [ >> + 'about.html', >> + 'navigation.html', >> + 'searchbox.html', >> + ], >> +} >> >> # Additional templates that should be rendered to pages, maps page names to >> # template names. >> >> === modified file 'docs/source/dependency.rst' >> --- docs/source/dependency.rst 2014-10-27 12:40:23 +0000 >> +++ docs/source/dependency.rst 2015-03-12 17:53:21 +0000 >> @@ -2,15 +2,26 @@ >> Dependencies >> ============ >> >> -The following dependencies are the ones documented in the first trials of Capomastro and are used mainly by the Phablet team. While they can be almost considered default, they're not shipped with Capomastro and must be set up manually after its installation. They'll be used as dependencies for projects later on. >> - >> -Please note some dependencies change the default value of build parameters. We don't have much knowledge on how they are used, but their different parameters are documented below. In the UI, they need to be specified as key value pairs, separated by an equal sign, e.g. ``KEY=VALUE``. >> - >> -Dependencies can be tracked automatically or not. >> - >> -If you have auto-track on, then anything that builds a dependency (automatically triggered by a VCS, manually built using Jenkins, or a project build using the same dependency) will result in the version of that dependency being used by the project being updated to use the newer version. >> - >> -Dependencies control the building of Jobs, when you create a dependency, you specify a job type and a server to build that job on. Delegating the builds to a Jenkins server means that Jenkins will handle building on the slaves associated with that Jenkins server. Parameters can be specified for this dependency, and will be passed to the Jenkins job on build on every build. >> +When you create a dependency, you specify a job type and a server to build that >> +job on. Delegating the builds to a Jenkins server means that Jenkins will >> +handle building on the slaves associated with that Jenkins server. Parameters >> +can be specified for this dependency, and will be passed to the Jenkins job on >> +every build. >> + >> +Creating a dependency should result in a job being pushed to your chosen >> +Jenkins server, and in your Celery screen or log you should see it happening. >> + >> +Projects can track dependencies automatically by using the *auto-track* option. If >> +you use *auto-track*, anything that triggers a dependency build (e.g. automatically >> +triggered by a VCS, manually built using Jenkins, or a project build using the >> +same dependency) will update all of the projects that auto-track that dependency >> +to the most recent successful build. >> + > > If I want a build to be "frozen", meaning that I access the build's page in Capomastro and download the artifacts and expect them to always be the same, that'd be a use case for not auto-tracking by default. Otherwise, if I download the artifacts today, then tomorrow one of them is rebuilt as part of another project, and I download the artifacts for my projet again on Sunday, they'll be different from today's. > >> +.. Note:: >> + Some dependencies change the default value of build parameters. At this time, we >> + don't have much knowledge on how they are used, but the different parameters >> + are documented below. In the UI, they need to be specified as key value pairs, >> + separated by an equal sign, e.g. ``KEY=VALUE``. >> >> The parameter entry box can be filled in like the following:: >> >> @@ -19,10 +30,28 @@ >> >> This will result in two parameters being passed to every build. >> >> -Creating a dependency should result in a job being pushed to your chosen Jenkins server, and in your Celery screen or log you should see it happening. >> +Capomastro Data Model >> +--------------------- >> + >> +This diagram shows how dependencies are modeled in Capomastro and how they >> +relate to all of the Capomastro apps. The models representing dependencies and >> +their relationships are labeled in black text. >> + >> +.. graphviz:: apps.dot >> + >> +Examples >> +-------- >> + >> +The following dependencies are the ones documented in the first trials of >> +Capomastro and are used mainly by the Phablet team. While they can be almost >> +considered default, they're not shipped with Capomastro and must be set up >> +manually after its installation. They'll be used as dependencies for projects >> +later on. >> + >> + >> >> Stock RootFS >> ------------- >> +++++++++++++ >> >> The following settings are default, for documentation purposes only. >> >> @@ -32,7 +61,7 @@ >> ``trusty`` >> >> Stock Android >> -------------- >> ++++++++++++++ >> >> The following settings are default, for documentation purposes only. >> >> @@ -44,19 +73,22 @@ >> ``trusty`` >> >> Barajas Android >> ---------------- >> ++++++++++++++++ >> + >> * ``JOB_BRANCH`` >> ``lp:bygmester`` >> >> Barajas RootFS >> --------------- >> +++++++++++++++ >> + >> * ``SERIES`` >> ``utopic`` >> * ``SOURCE_BRANCH`` >> ``lp:~barajas-team/barajas/barajas-config`` >> >> Barajas RootFS (pulseaudio) >> ---------------------------- >> ++++++++++++++++++++++++++++ >> + >> * ``SERIES`` >> ``utopic`` >> * ``ADDITIONAL_SOURCES`` >> @@ -65,11 +97,13 @@ >> ``lp:~barajas-team/barajas/barajas-config`` >> >> Barajas Customization Tarball >> ------------------------------ >> ++++++++++++++++++++++++++++++ >> + >> * ``BRANCH`` >> ``lp:~barajas-team/barajas/new-custom`` >> >> Upstream RootFS >> ---------------- >> ++++++++++++++++ >> + >> * ``URL_TO_ARCHIVE`` >> - ``http://cdimage.ubuntu.com/ubuntu-touch/daily-preinstalled/current/utopic-preinstalled-touch-armhf.tar.gz`` >> \ No newline at end of file >> + ``http://cdimage.ubuntu.com/ubuntu-touch/daily-preinstalled/current/utopic-preinstalled-touch-armhf.tar.gz`` >> >> === added file 'docs/source/development.rst' >> --- docs/source/development.rst 1970-01-01 00:00:00 +0000 >> +++ docs/source/development.rst 2015-03-12 17:53:21 +0000 >> @@ -0,0 +1,14 @@ >> +Development >> +=========== >> + >> +.. Note:: >> + TODO: document deployment of Capomastro with mojo. > > This would probably require heavy reworking of the spec because some of the components are also private (the IS-pinned Jenkins and some of the charms are private, right?). This intermingles with the work to be done on the local stage, which we *could* conceivably make to depend only on public stuff. But at the time I'd say there's no point on pointing external people to the spec because it'll be inaccessible to them :/ > > The plan is for our internal docs to point to Mojo but not sure about Capomastro's docs which may be public at some point. > >> + >> +.. toctree:: >> + :maxdepth: 2 >> + >> + setup >> + testing >> + docker >> + >> + >> >> === modified file 'docs/source/index.rst' >> --- docs/source/index.rst 2014-11-28 16:52:36 +0000 >> +++ docs/source/index.rst 2015-03-12 17:53:21 +0000 >> @@ -1,18 +1,10 @@ >> .. Capomastro documentation master file, created by >> sphinx-quickstart on Mon Oct 20 16:11:51 2014. >> -.. _setup: setup.html >> >> Capomastro's documentation >> ========================== >> >> -Contents: >> - >> -.. toctree:: >> - :maxdepth: 2 >> - :glob: >> - >> - ** >> - >> + >> Introduction >> ------------ >> >> @@ -22,11 +14,6 @@ >> >> Orinally, Capomastro was developed on Github as an open source side project of Canonical employees, but this is the forked version of it hosted on Launchpad to better reflect the roadmap we have for it. >> >> -Setting up >> ----------- >> - >> -Please refer to the setup_ section for instructions. >> - >> Contact >> ------- >> >> @@ -34,6 +21,22 @@ >> >> You can also send us an e-mail and we'll get back to you as soon as possible, just shoot it one for the PES Certification & Tools team at . You'll also find many of us in our IRC channel ``#cert-infra``. >> >> +Contents: >> + >> +.. toctree:: >> + :maxdepth: 2 >> + >> + project >> + dependency >> + jenkins >> + mta >> + sites >> + sshkeypair >> + stats >> + development >> + >> + >> + >> >> Indices and tables >> ================== >> >> === added file 'docs/source/jenkins.dot' >> --- docs/source/jenkins.dot 1970-01-01 00:00:00 +0000 >> +++ docs/source/jenkins.dot 2015-03-12 17:53:21 +0000 >> @@ -0,0 +1,112 @@ >> +digraph model_graph { >> + // Dotfile by Django-Extensions graph_models >> + // Created: 2015-03-11 16:20 >> + // Cli Options: --pygraphviz -d -g jenkins >> + >> + fontname = "Helvetica" >> + fontsize = 8 >> + splines = true >> + >> + node [ >> + fontname = "Helvetica" >> + fontsize = 8 >> + shape = "plaintext" >> + ] >> + >> + edge [ >> + fontname = "Helvetica" >> + fontsize = 8 >> + ] >> + >> + // Labels >> + subgraph cluster_jenkins_models { >> + label=< >> + >> + >> +

>> + >> + jenkins >> + >> +

>> + > >> + color=olivedrab4 >> + style="rounded" >> + >> + jenkins_models_JenkinsServer [label=< >> + >> + >> + >> +

>> + >> + JenkinsServer >> +

>> + >] >> + >> + jenkins_models_JobType [label=< >> + >> + >> + >> +

>> + >> + JobType >> +

>> + >] >> + >> + jenkins_models_Job [label=< >> + >> + >> + >> +

>> + >> + Job >> +

>> + >] >> + >> + jenkins_models_Build [label=< >> + >> + >> + >> +

>> + >> + Build >> +

>> + >] >> + >> + jenkins_models_Artifact [label=< >> + >> + >> + >> +

>> + >> + Artifact >> +

>> + User >> +

>> + >] >> + jenkins_models_Build -> django_contrib_auth_models_User >> + [label="requested_by (build)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + jenkins_models_Artifact -> jenkins_models_Build >> + [label="build (artifact)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> +} >> + >> >> === modified file 'docs/source/jenkins.rst' >> --- docs/source/jenkins.rst 2014-10-27 12:44:28 +0000 >> +++ docs/source/jenkins.rst 2015-03-12 17:53:21 +0000 >> @@ -4,6 +4,12 @@ >> >> The Jenkins server which actually builds the artifacts cannot be a simple VM, it needs some extra resources not available to all VMs in Canonistack for instance. As of October 2014, the minimal configuration for a Jenkins server is way over ``M1.LARGE``. You'll need a custom VM with 100G of root disk (though the artifacts can be archived in a network mounted ``/mnt`` path), with at least 10G of RAM and 10 CPU cores for ideal testing performance. > > This is more of a question for Caio, does the use of ceph volumes remove the need for such a big disk? if so, maybe we can change this paragraph to be less scary and avoid future clashes with IS over our compute instances consuming all disk space in the hosts :) (This is only about documenting btw, it doesn't affect what we're actually doing in production). > >> >> +Jenkins Data Model >> +------------------ >> + >> +.. graphviz:: jenkins.dot >> + >> + >> Bot >> --- >> >> >> === added file 'docs/source/project.dot' >> --- docs/source/project.dot 1970-01-01 00:00:00 +0000 >> +++ docs/source/project.dot 2015-03-12 17:53:21 +0000 >> @@ -0,0 +1,142 @@ >> +digraph model_graph { >> + // Dotfile by Django-Extensions graph_models >> + // Created: 2015-03-11 16:20 >> + // Cli Options: --pygraphviz -d -g projects >> + >> + fontname = "Helvetica" >> + fontsize = 8 >> + splines = true >> + >> + node [ >> + fontname = "Helvetica" >> + fontsize = 8 >> + shape = "plaintext" >> + ] >> + >> + edge [ >> + fontname = "Helvetica" >> + fontsize = 8 >> + ] >> + >> + // Labels >> + subgraph cluster_projects_models { >> + label=< >> + >> + >> +

>> + >> + projects >> + >> +

>> + > >> + color=olivedrab4 >> + style="rounded" >> + >> + projects_models_Dependency [label=< >> + >> + >> + >> +

>> + >> + Dependency >> +

>> + >] >> + >> + projects_models_ProjectDependency [label=< >> + >> + >> + >> +

>> + >> + ProjectDependency >> +

>> + >] >> + >> + projects_models_Project [label=< >> + >> + >> + >> +

>> + >> + Project >> +

>> + >] >> + >> + projects_models_ProjectBuildDependency [label=< >> + >> + >> + >> +

>> + >> + ProjectBuildDependency >> +

>> + >] >> + >> + projects_models_ProjectBuild [label=< >> + >> + >> + >> +

>> + >> + ProjectBuild >> +

>> + >] >> + >> + } >> + >> + >> + // Relations >> + jenkins_models_Job [label=< >> + >> + >> +

>> + Job >> +

>> + >] >> + projects_models_Dependency -> jenkins_models_Job >> + [label="job (dependency)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + projects_models_ProjectDependency -> projects_models_Dependency >> + [label="dependency (projectdependency)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + projects_models_ProjectDependency -> projects_models_Project >> + [label="project (projectdependency)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + jenkins_models_Build [label=< >> + >> + >> +

>> + Build >> +

>> + >] >> + projects_models_ProjectDependency -> jenkins_models_Build >> + [label="current_build (projectdependency)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + projects_models_ProjectBuildDependency -> projects_models_ProjectBuild >> + [label="projectbuild (dependencies)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + jenkins_models_Build [label=< >> + >> + >> +

>> + Build >> +

>> + >] >> + projects_models_ProjectBuildDependency -> jenkins_models_Build >> + [label="build (projectbuild_dependencies)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + projects_models_ProjectBuildDependency -> projects_models_Dependency >> + [label="dependency (projectbuilddependency)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> + projects_models_ProjectBuild -> projects_models_Project >> + [label="project (projectbuild)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + django_contrib_auth_models_User [label=< >> + >> + >> +

>> + User >> +

>> + >] >> + projects_models_ProjectBuild -> django_contrib_auth_models_User >> + [label="requested_by (projectbuild)"] [arrowhead=none, arrowtail=dot, dir=both]; >> + >> +} >> + >> >> === modified file 'docs/source/project.rst' >> --- docs/source/project.rst 2014-10-27 12:40:23 +0000 >> +++ docs/source/project.rst 2015-03-12 17:53:21 +0000 >> @@ -1,12 +1,34 @@ >> +.. _dependencies: dependency.html >> + >> ======== >> Projects >> ======== >> >> -Similar to the dependencies setup, Capomastro will need them to be linked to actual projects, so they can be built and their artifacts will finally be available for download from the web interface. Always prefer to build images through a project and its respective dependencies. Avoid building dependencies directly. >> - >> -You can create a "project build" this will be given a build-id, which is sent to Jenkins in the `BUILD_ID` variable, all builds of dependencies will be given the same `BUILD_ID`, this means it's possible to determine when a project's dependencies have all built, and get all artifacts from all dependencies. >> - >> -Please refer to the dependencies documentation if you're unsure on how to set them up. These are simply the default projects used internally by the Phablet team at Canonical, they're not necessarily required for all Capomastro setups. >> +Capomastro links dependencies and projects, so that projects can be built and >> +their artifacts made available for download from the web interface. Always >> +prefer to build images through a project and its respective dependencies over >> +building dependencies directly. >> + >> +You can create a "project build". This will be given a build-id, which is sent >> +to Jenkins in the `BUILD_ID` variable. All builds of dependencies will be given >> +the same `BUILD_ID`. This means it's possible to determine when a project's >> +dependencies have all built and get all artifacts from all dependencies. >> + >> +Please refer to the dependencies_ documentation for information on how to set >> +up dependencies and on the auto-tracking feature for Projects. >> + >> + >> +Projects Data Model >> +------------------- >> + >> +.. graphviz:: project.dot >> + >> + >> +Default Projects >> +---------------- >> + >> +This section documents the default projects used internally by the Phablet >> +team at Canonical. They're not required for all Capomastro setups. >> >> Stock >> ----- >> >> === modified file 'docs/source/setup.rst' >> --- docs/source/setup.rst 2014-12-15 12:20:17 +0000 >> +++ docs/source/setup.rst 2015-03-12 17:53:21 +0000 >> @@ -2,9 +2,14 @@ >> Setup >> ===== >> >> -This file was created based on the old historic README file from Capomastro, if anything seems to be missing here please refer to the whole documentation files in the ``docs/`` directory. >> +This file was created based on the old historic README file from Capomastro, if >> +anything seems to be missing here please refer to the whole documentation files >> +in the ``docs/`` directory. >> >> -If you're an old school Django guy, you can get started with local testing and development by following the next sections. You'll need the following services in order to bring the application up anyway, no matter if using the packaged code or not: >> +If you're an old school Django guy, you can get started with local testing and >> +development by following the next sections. You'll need the following services >> +in order to bring the application up anyway, no matter if using the packaged >> +code or not: >> >> * PostgreSQL, default database >> * Apache with mod WSGI, to expose the UI >> @@ -14,11 +19,18 @@ >> Environment >> ----------- >> >> -Create your environment either with ``mkvirtualenv ``, if you have virtualenvwrapper installed, or simply ``tox -e devenv`` since it does some extra steps for you. It's recommended to install ``tox`` and go with the latter option. >> - >> -If you decide for using ``mkvirtualenv`` you'll also need to manually install the Python dependencies with ``pip install -r requirements.txt`` or even ``pip install -r dev-requirements.txt`` if you're going to run the code tests. >> - >> -Then either activate the virtual environment manually with ``source devenv/bin/activate`` or explicitly call Python from the virtual environment with ``devenv/bin/python``. >> +Create your environment either with ``mkvirtualenv ``, if you have >> +virtualenvwrapper installed, or simply ``tox -e devenv`` since it does some >> +extra steps for you. It's recommended to install ``tox`` and go with the latter >> +option. >> + >> +If you decide for using ``mkvirtualenv`` you'll also need to manually install >> +the Python dependencies with ``pip install -r requirements.txt`` or even ``pip >> +install -r dev-requirements.txt`` if you're going to run the code tests. >> + >> +Then either activate the virtual environment manually with ``source >> +devenv/bin/activate`` or explicitly call Python from the virtual environment >> +with ``devenv/bin/python``. >> >> Database >> -------- >> @@ -30,50 +42,80 @@ >> .. include:: ../../testing/requirements/capomastro.sql >> :literal: >> >> -If needed, fixup the database entry in ``capomastro/db_settings.py``, but most likely the default values will work okay unless you're testing database changes. >> +If needed, fixup the database entry in ``capomastro/db_settings.py``, but most >> +likely the default values will work okay unless you're testing database >> +changes. >> >> -You'll need to prepare the database with ``./manage.py syncdb`` (and you'll be prompted to create a superuser at this time). >> +You'll need to prepare the database with ``./manage.py syncdb`` (and you'll be >> +prompted to create a superuser at this time). >> >> Web server >> ---------- >> >> -You can manually start Capomastro from its source directory by executing ``./manage.py runserver`` otherwise. However, if you want to set it up with WSGI, take a look at the file ``wsgi.conf``. >> - >> -All the remaining setup steps are through the ``/admin/`` interface of Capomastro, so go to http://localhost:8000/admin/ then login with the username and password created before and continue. >> - >> -If you prefer to test things with Gunicorn you'll need a Celery worker running already, as well as RabbitMQ. >> - >> -Just run ``gunicorn -b 0.0.0.0:8000 capomastro.wsgi:application`` and start the Celery worker that will talk to the Rabbit queue with ``celery -A capomastro worker -l debug``. >> - >> -Django >> ------- >> - >> -All that remains now is the first the dependencies setup, then the projects setup, in that particular order. Creating of projects before its dependencies are set is not currently supported. >> - >> -In the ``/admin/`` interface of Capomastro you'll need an initial ``jenkins.JenkinsServer`` object, with the correct credentials and ``REMOTE_ADDR`` set up correctly, so that it can receive callbacks. >> - >> -Create a ``DependencyType`` and ``Dependency`` for that ``DependencyType``. The ``Dependency`` should be hooked up to your ``Job`` model. >> +You can manually start Capomastro from its source directory by executing >> +``./manage.py runserver`` otherwise. However, if you want to set it up with >> +WSGI, take a look at the file ``wsgi.conf``. >> + >> +All the remaining setup steps are through the ``/admin/`` interface of >> +Capomastro, so go to http://localhost:8000/admin/ then login with the username >> +and password created before and continue. >> + >> +If you prefer to test things with Gunicorn you'll need a Celery worker running >> +already, as well as RabbitMQ. >> + >> +Just run ``gunicorn -b 0.0.0.0:8000 capomastro.wsgi:application`` and start the >> +Celery worker that will talk to the Rabbit queue with ``celery -A capomastro >> +worker -l debug``. >> + >> +Django Admin Setup >> +------------------ >> + >> +All that remains now is the first the dependencies setup, then the projects >> +setup, in that particular order. Creating of projects before its dependencies >> +are set is not currently supported. >> + >> +In the ``/admin/`` interface of Capomastro you'll need to create a >> +``jenkins.JenkinsServer`` object, with the correct credentials and >> +``REMOTE_ADDR`` set up correctly, so that it can receive callbacks. >> + >> +Create a ``JobType`` and a ``Job`` for that ``JobType``. Then >> +create a ``Dependency`` for that ``Job`` model. >> >> You should have something like this: >> :: >> >> - Dependency - DependencyType >> + Dependency >> | >> - Job >> + Job - JobType >> | >> JenkinsServer >> >> -You can create several dependencies, with different jobs. Your Jenkins server requires the ``notifications`` plugin installed too, so make sure it's available otherwise you won't get status back from the builds. >> - >> -The jobs that are created must have a parameter ``BUILD_ID`` and they must have a notification setup, type ``http/json`` and with a callback address of ``http://hostname/jenkins/notifications/``. >> - >> -Now you can create a ``Project``, associated with your dependencies, at ``http://localhost:8000/projects/create/``. The option "auto track" means that the project will use the latest version of any dependencies automatically. >> - >> -Now you can build your project. This will create a project build, and trigger the tasks to build your project on the Jenkins server. >> +You can create several dependencies, with different jobs. Your Jenkins server >> +requires the ``notifications`` plugin installed, so make sure it's >> +available otherwise you won't get status back from the builds. >> + >> +The jobs that are created must have a parameter ``BUILD_ID`` and they must have >> +a notification setup, type ``http/json`` and with a callback address of >> +``http://hostname/jenkins/notifications/``. >> + >> +Now you can create a ``Project``, associated with your dependencies, at >> +``http://localhost:8000/projects/create/``. The option "auto track" means that >> +the project will use the latest version of any dependencies automatically. >> + >> +Now you can build your project. This will create a project build, and trigger >> +the tasks to build your project on the Jenkins server. >> >> Packaged setup >> -------------- >> >> -All the configurations of Capomastro (messaging, DB, the service etc) are stored via symlinks inside of ``/etc/capomastro/`` if you're using the Debian package of the application. This directory also contains the file ``wsgi.conf`` which is symlinked to the Apache configuration so the site is enabled by default. The default Django superuser can be used to access the service for the first time, the username and password should be both ``capomastro`` and it's created by the package automatically. >> +All the configurations of Capomastro (messaging, DB, the service etc) are >> +stored via symlinks inside of ``/etc/capomastro/`` if you're using the Debian >> +package of the application. This directory also contains the file ``wsgi.conf`` >> +which is symlinked to the Apache configuration so the site is enabled by >> +default. The default Django superuser can be used to access the service for the >> +first time, the username and password should be both ``capomastro`` and it's >> +created by the package automatically. >> >> -All code is actually validated through the application package, so although you can test and develop things really locally, only code from packages is considered deployable. >> +All code is actually validated through the application package, so although you >> +can test and develop things really locally, only code from packages is >> +considered deployable. >> >> === modified file 'projects/models.py' >> --- projects/models.py 2014-12-11 19:09:37 +0000 >> +++ projects/models.py 2015-03-12 17:53:21 +0000 >> @@ -83,7 +83,7 @@ >> """ >> dependency = models.ForeignKey(Dependency) >> project = models.ForeignKey("Project") >> - auto_track = models.BooleanField(default=True) >> + auto_track = models.BooleanField(default=True, help_text="Always use the latest build.") >> current_build = models.ForeignKey(Build, null=True, editable=False) >> >> class Meta: >> > > > -- > https://code.launchpad.net/~codersquid/capomastro/autotracking-docs/+merge/252795 > You are reviewing the proposed merge of lp:~codersquid/capomastro/autotracking-docs into lp:capomastro.