Launchpad itself

Merge lp://staging/~al-maisan/launchpad/bq-platform-columns-505725 into lp://staging/launchpad/db-devel

  1. bq-platform-columns-505725
  2. Merge into db-devel
Proposed by Muharem Hrnjadovic
Status: Merged
Approved by: Jonathan Lange
Approved revision: not available
Merged at revision: not available
Proposed branch: lp://staging/~al-maisan/launchpad/bq-platform-columns-505725
Merge into: lp://staging/launchpad/db-devel
Diff against target: 198 lines (+96/-10)
8 files modified
database/sampledata/current-dev.sql (+2/-2)
database/sampledata/current.sql (+2/-2)
database/schema/comments.sql (+2/-0)
database/schema/patch-2207-24-0.sql (+20/-0)
lib/lp/soyuz/interfaces/buildqueue.py (+18/-5)
lib/lp/soyuz/model/build.py (+2/-1)
lib/lp/soyuz/model/buildqueue.py (+3/-0)
lib/lp/soyuz/tests/test_buildqueue.py (+47/-0)
To merge this branch: bzr merge lp://staging/~al-maisan/launchpad/bq-platform-columns-505725
Reviewer Review Type Date Requested Status
Paul Hummer (community) code Approve
Jonathan Lange (community) Approve
Stuart Bishop (community) db Approve
Review via email: mp+17125@code.staging.launchpad.net
To post a comment you must log in.
Revision history for this message
Muharem Hrnjadovic (al-maisan) wrote :

Hello there!

In order to be able to formulate reasonably straightforward queries that find
candidate jobs for idle builders we'd like to add NULLable 'processor' and
'virtualized' columns to BuildQueue.

In case of binary builds these will replicate the respective Build.processor
and Build.is_virtualized values. For processor-independent jobs they will
remain NULL.

Please note also that these columns will only be initialed once, when the
BuildQueue record is inserted.

Pre-implementation chat with jml. Bug #505725.

Revision history for this message
Stuart Bishop (stub) wrote :

Looks fine. patch-2207-24-0.sql

review: Approve (db)
Revision history for this message
Jonathan Lange (jml) wrote :

Fine by me.

review: Approve
Revision history for this message
Paul Hummer (rockstar) wrote :

Please add whitespace around the test's "layer =..." line. Other than that, it looks good.

review: Approve (code)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'database/sampledata/current-dev.sql'
2--- database/sampledata/current-dev.sql 2009-12-14 13:49:03 +0000
3+++ database/sampledata/current-dev.sql 2010-01-12 02:26:13 +0000
4@@ -1715,8 +1715,8 @@
5
6 ALTER TABLE buildqueue DISABLE TRIGGER ALL;
7
8-INSERT INTO buildqueue (id, builder, logtail, lastscore, manual, job, job_type, estimated_duration) VALUES (1, 1, 'Dummy sampledata entry, not processing', 1, false, 1, 1, '00:00:00');
9-INSERT INTO buildqueue (id, builder, logtail, lastscore, manual, job, job_type, estimated_duration) VALUES (2, NULL, NULL, 10, false, 2, 1, '00:01:00');
10+INSERT INTO buildqueue (id, builder, logtail, lastscore, manual, job, job_type, estimated_duration, processor, virtualized) VALUES (1, 1, 'Dummy sampledata entry, not processing', 1, false, 1, 1, '00:00:00', 1, FALSE);
11+INSERT INTO buildqueue (id, builder, logtail, lastscore, manual, job, job_type, estimated_duration, processor, virtualized) VALUES (2, NULL, NULL, 10, false, 2, 1, '00:01:00', 1, FALSE);
12
13
14 ALTER TABLE buildqueue ENABLE TRIGGER ALL;
15
16=== modified file 'database/sampledata/current.sql'
17--- database/sampledata/current.sql 2009-12-14 13:49:03 +0000
18+++ database/sampledata/current.sql 2010-01-12 02:26:13 +0000
19@@ -1697,8 +1697,8 @@
20
21 ALTER TABLE buildqueue DISABLE TRIGGER ALL;
22
23-INSERT INTO buildqueue (id, builder, logtail, lastscore, manual, job, job_type, estimated_duration) VALUES (1, 1, 'Dummy sampledata entry, not processing', 1, false, 1, 1, '00:00:00');
24-INSERT INTO buildqueue (id, builder, logtail, lastscore, manual, job, job_type, estimated_duration) VALUES (2, NULL, NULL, 10, false, 2, 1, '00:01:00');
25+INSERT INTO buildqueue (id, builder, logtail, lastscore, manual, job, job_type, estimated_duration, processor, virtualized) VALUES (1, 1, 'Dummy sampledata entry, not processing', 1, false, 1, 1, '00:00:00', 1, FALSE);
26+INSERT INTO buildqueue (id, builder, logtail, lastscore, manual, job, job_type, estimated_duration, processor, virtualized) VALUES (2, NULL, NULL, 10, false, 2, 1, '00:01:00', 1, FALSE);
27
28
29 ALTER TABLE buildqueue ENABLE TRIGGER ALL;
30
31=== modified file 'database/schema/comments.sql'
32--- database/schema/comments.sql 2009-12-01 13:45:58 +0000
33+++ database/schema/comments.sql 2010-01-12 02:26:13 +0000
34@@ -1537,6 +1537,8 @@
35 COMMENT ON COLUMN BuildQueue.job IS 'Foreign key to the `Job` table row with the generic job data.';
36 COMMENT ON COLUMN BuildQueue.job_type IS 'Type of job (enumeration value), enables us to find/query the correct table with the data specific to this type of job.';
37 COMMENT ON COLUMN BuildQueue.estimated_duration IS 'Estimated job duration, based on previous running times of comparable jobs.';
38+COMMENT ON COLUMN BuildQueue.processor IS 'The processor required by the associated build farm job.';
39+COMMENT ON COLUMN BuildQueue.virtualized IS 'The virtualization setting required by the associated build farm job.';
40
41 -- Mirrors
42
43
44=== added file 'database/schema/patch-2207-24-0.sql'
45--- database/schema/patch-2207-24-0.sql 1970-01-01 00:00:00 +0000
46+++ database/schema/patch-2207-24-0.sql 2010-01-12 02:26:13 +0000
47@@ -0,0 +1,20 @@
48+-- Copyright 2009 Canonical Ltd. This software is licensed under the
49+-- GNU Affero General Public License version 3 (see the file LICENSE).
50+
51+SET client_min_messages=ERROR;
52+
53+-- Another schema patch required for the Soyuz buildd generalisation, see
54+-- https://dev.launchpad.net/Soyuz/Specs/BuilddGeneralisation for details.
55+-- Bug #505725.
56+
57+-- Changes needed to the `BuildQueue` table.
58+
59+-- The 'processor' and the 'virtualized' columns will enable us to formulate
60+-- more straightforward queries for finding candidate jobs when builders
61+-- become idle.
62+ALTER TABLE ONLY buildqueue ADD COLUMN processor integer;
63+ALTER TABLE ONLY buildqueue ADD COLUMN virtualized boolean;
64+
65+CREATE INDEX buildqueue__processor__virtualized__idx ON buildqueue USING btree (processor, virtualized) WHERE (processor IS NOT NULL);
66+
67+INSERT INTO LaunchpadDatabaseRevision VALUES (2207, 24, 0);
68
69=== modified file 'lib/lp/soyuz/interfaces/buildqueue.py'
70--- lib/lp/soyuz/interfaces/buildqueue.py 2010-01-07 04:07:09 +0000
71+++ lib/lp/soyuz/interfaces/buildqueue.py 2010-01-12 02:26:13 +0000
72@@ -13,7 +13,7 @@
73 ]
74
75 from zope.interface import Interface, Attribute
76-from zope.schema import Choice, Datetime, Field, Timedelta
77+from zope.schema import Bool, Choice, Datetime, Field, Int, Text, Timedelta
78
79 from lazr.restful.fields import Reference
80
81@@ -21,6 +21,8 @@
82 from lp.buildmaster.interfaces.buildfarmjob import (
83 IBuildFarmJob, BuildFarmJobType)
84 from lp.services.job.interfaces.job import IJob
85+from lp.soyuz.interfaces.builder import IBuilder
86+from lp.soyuz.interfaces.processor import IProcessor
87
88
89 class IBuildQueue(Interface):
90@@ -38,10 +40,21 @@
91 """
92
93 id = Attribute("Job identifier")
94- builder = Attribute("The IBuilder instance processing this job")
95- logtail = Attribute("The current tail of the log of the build")
96- lastscore = Attribute("Last score to be computed for this job")
97- manual = Attribute("Whether or not the job was manually scored")
98+ builder = Reference(
99+ IBuilder, title=_("Builder"), required=True, readonly=True,
100+ description=_("The IBuilder instance processing this job"))
101+ logtail = Text(
102+ description=_("The current tail of the log of the job"))
103+ lastscore = Int(description=_("This job's score."))
104+ manual = Bool(
105+ description=_("Whether or not the job was manually scored."))
106+ processor = Reference(
107+ IProcessor, title=_("Processor"), required=False, readonly=True,
108+ description=_("The processor required by this build farm job."))
109+ virtualized = Bool(
110+ required=False,
111+ description=_(
112+ "The virtualization setting required by this build farm job."))
113
114 job = Reference(
115 IJob, title=_("Job"), required=True, readonly=True,
116
117=== modified file 'lib/lp/soyuz/model/build.py'
118--- lib/lp/soyuz/model/build.py 2009-12-10 16:51:18 +0000
119+++ lib/lp/soyuz/model/build.py 2010-01-12 02:26:13 +0000
120@@ -682,7 +682,8 @@
121 queue_entry = BuildQueue(
122 estimated_duration=duration_estimate,
123 job_type=BuildFarmJobType.PACKAGEBUILD,
124- job=job.id)
125+ job=job.id, processor=self.processor,
126+ virtualized=self.is_virtualized)
127 store.add(queue_entry)
128 return queue_entry
129
130
131=== modified file 'lib/lp/soyuz/model/buildqueue.py'
132--- lib/lp/soyuz/model/buildqueue.py 2010-01-08 08:55:10 +0000
133+++ lib/lp/soyuz/model/buildqueue.py 2010-01-12 02:26:13 +0000
134@@ -52,6 +52,9 @@
135 lastscore = IntCol(dbName='lastscore', default=0)
136 manual = BoolCol(dbName='manual', default=False)
137 estimated_duration = IntervalCol()
138+ processor = ForeignKey(
139+ dbName='processor', foreignKey='Processor', notNull=True)
140+ virtualized = BoolCol(dbName='virtualized')
141
142 @property
143 def required_build_behavior(self):
144
145=== modified file 'lib/lp/soyuz/tests/test_buildqueue.py'
146--- lib/lp/soyuz/tests/test_buildqueue.py 2010-01-07 20:12:52 +0000
147+++ lib/lp/soyuz/tests/test_buildqueue.py 2010-01-12 02:26:13 +0000
148@@ -740,3 +740,50 @@
149 self.assertEqual(
150 bq.specific_job_classes[BuildFarmJobType.BRANCHBUILD],
151 FakeBranchBuild)
152+
153+
154+class TestPlatformData(TestCaseWithFactory):
155+ """Tests covering the processor/virtualized properties."""
156+
157+ layer = LaunchpadZopelessLayer
158+
159+ def setUp(self):
160+ """Set up a native x86 build for the test archive."""
161+ super(TestPlatformData, self).setUp()
162+
163+ self.publisher = SoyuzTestPublisher()
164+ self.publisher.prepareBreezyAutotest()
165+
166+ # First mark all builds in the sample data as already built.
167+ store = getUtility(IStoreSelector).get(MAIN_STORE, DEFAULT_FLAVOR)
168+ sample_data = store.find(Build)
169+ for build in sample_data:
170+ build.buildstate = BuildStatus.FULLYBUILT
171+ store.flush()
172+
173+ # We test builds that target a primary archive.
174+ self.non_ppa = self.factory.makeArchive(
175+ name="primary", purpose=ArchivePurpose.PRIMARY)
176+ self.non_ppa.require_virtualized = False
177+
178+ self.builds = []
179+ self.builds.extend(
180+ self.publisher.getPubSource(
181+ sourcename="gedit", status=PackagePublishingStatus.PUBLISHED,
182+ archive=self.non_ppa).createMissingBuilds())
183+
184+ def test_JobPlatformSettings(self):
185+ """The `BuildQueue` instance shares the processor/virtualized
186+ properties with the associated `Build`."""
187+ build, bq = find_job(self, 'gedit')
188+
189+ # Make sure the 'processor' properties are the same.
190+ self.assertEqual(
191+ bq.processor, build.processor,
192+ "The 'processor' property deviates.")
193+
194+ # Make sure the 'virtualized' properties are the same.
195+ self.assertEqual(
196+ bq.virtualized, build.is_virtualized,
197+ "The 'virtualized' property deviates.")
198+

Subscribers

People subscribed via source and target branches

to status/vote changes: