Bazaar

Merge lp://staging/~mbp/bzr/doc-old into lp://staging/~bzr/bzr/trunk-old

  1. doc-old
  2. Merge into trunk-old
Proposed by Martin Pool
Status: Merged
Approved by: Martin Pool
Approved revision: no longer in the source branch.
Merged at revision: not available
Proposed branch: lp://staging/~mbp/bzr/doc-old
Merge into: lp://staging/~bzr/bzr/trunk-old
Diff against target: 794 lines (has conflicts) 
Text conflict in bzrlib/tests/__init__.py
Text conflict in bzrlib/tests/blackbox/test_selftest.py
Text conflict in bzrlib/tests/test_selftest.py
To merge this branch: bzr merge lp://staging/~mbp/bzr/doc-old
Reviewer Review Type Date Requested Status
Ian Clatworthy Pending
Review via email: mp+10638@code.staging.launchpad.net
To post a comment you must log in.
Revision history for this message
Martin Pool (mbp) wrote :

Some documentation on how content filtering is supposed to work, gleaned from the code, email, people's answers.

Revision history for this message
Ian Clatworthy (ian-clatworthy) wrote :

Martin Pool wrote:

> Some documentation on how content filtering is supposed to work, gleaned from the code, email, people's answers.

Thanks.

> +Dirstate interactions
> +*********************
> +
> +The dirstate file stores, in the column for the working copy, the cached

Maybe this should read "should store".

> +User interface
> +**************
> +
> +Most commands that deal with the text of files should present the
> +convenient form, but also have an option to get the canonical form.
> +

To my knowledge, it's the other way around, at least for diff, cat and
export. It's true that some commands must operate on the convenient
form, e.g. shelve, merge.

> +* Content filtering at the moment is a bit specific to on-disk trees (eg
> + in the definition of ``SHA1Provider``) but that seems unnecessary.
> +

You probably need to clarify what you mean by that. RevisionTrees don't
have a convenience form, though it's possible to grab their content and
apply the *current* configuration of filters. (Rules aren't stored
historically in any way yet.) The export and cat commands do precisely that.

> +Preconditions
> +-------------
> +
> +#. Download the pqm plugin and install it into your ``~/.bazaar/plugins``::
> +
> + bzr branch lp:bzr-pqm ~/.bazaar/plugins/pqm
> +
> Starting the release phase
> --------------------------
>
> @@ -34,10 +41,6 @@
>
> bzr branch trunk prepare-1.14
>
> -#. Download the pqm plugin and install it into your ``~/.bazaar/plugins``::
> -
> - bzr branch lp:bzr-pqm ~/.bazaar/plugins/pqm
> -
> #. Configure pqm-submit for this branch, with a section like this in
> ``~/.bazaar/locations.conf``::
>
> @@ -63,9 +66,8 @@
> ./tools/check-newsbugs.py NEWS
>
> (But note there can be some false positives, and this script may be
> - flaky
> - <https://bugs.edge.launchpad.net/bzr/+bug/354985>. Don't let this
> - slow you down too much.)
> + flaky <https://bugs.edge.launchpad.net/bzr/+bug/354985>. Don't let
> + this slow you down too much.)
>

This bit seems like an unrelated doc patch. Perhaps that's been sent to
pqm but not merged yet?

Ian C.

Revision history for this message
Martin Pool (mbp) wrote :

> Martin Pool wrote:
>
> > Some documentation on how content filtering is supposed to work, gleaned
> from the code, email, people's answers.
>
> Thanks.

Thanks for the speedy review.

> Maybe this should read "should store".

Yes and I filed bug 418439.

> To my knowledge, it's the other way around, at least for diff, cat and
> export. It's true that some commands must operate on the convenient
> form, e.g. shelve, merge.

Corrected.

> > +* Content filtering at the moment is a bit specific to on-disk trees (eg
> > + in the definition of ``SHA1Provider``) but that seems unnecessary.
>
> You probably need to clarify what you mean by that. RevisionTrees don't
> have a convenience form, though it's possible to grab their content and
> apply the *current* configuration of filters. (Rules aren't stored
> historically in any way yet.) The export and cat commands do precisely that.

What I meant is that at the moment export for example has it's own code in each exporter that does filtering, and that seems a bit less clean than just letting export get a tree that looks filtered and then exporting that.

>
> > +Preconditions
> > +-------------
> > +
> > +#. Download the pqm plugin and install it into your ``~/.bazaar/plugins``::
> > +
> > + bzr branch lp:bzr-pqm ~/.bazaar/plugins/pqm
> > +
> > Starting the release phase
> > --------------------------
> >
> > @@ -34,10 +41,6 @@
> >
> > bzr branch trunk prepare-1.14
> >
> > -#. Download the pqm plugin and install it into your ``~/.bazaar/plugins``::
> > -
> > - bzr branch lp:bzr-pqm ~/.bazaar/plugins/pqm
> > -
> > #. Configure pqm-submit for this branch, with a section like this in
> > ``~/.bazaar/locations.conf``::
> >
> > @@ -63,9 +66,8 @@
> > ./tools/check-newsbugs.py NEWS
> >
> > (But note there can be some false positives, and this script may be
> > - flaky
> > - <https://bugs.edge.launchpad.net/bzr/+bug/354985>. Don't let this
> > - slow you down too much.)
> > + flaky <https://bugs.edge.launchpad.net/bzr/+bug/354985>. Don't let
> > + this slow you down too much.)
> >
>
> This bit seems like an unrelated doc patch. Perhaps that's been sent to
> pqm but not merged yet?
>
>
> Ian C.

Revision history for this message
Martin Pool (mbp) wrote :

> This bit seems like an unrelated doc patch. Perhaps that's been sent to
> pqm but not merged yet?

yes.

Revision history for this message
Aaron Bentley (abentley) wrote :

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ian Clatworthy wrote:
> It's true that some commands must operate on the convenient
> form, e.g. shelve, merge.

But even those commands should calculate the result using the canonical
form, and the write out the convenience form, right?

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkqUKAsACgkQ0F+nu1YWqI0eeQCfTsz6IJMRZvQ7/v5N2Ca0IV/m
zyEAnjI34gDQLbydpbenfVgYD7CAPfQm
=WSYO
-----END PGP SIGNATURE-----

Revision history for this message
Ian Clatworthy (ian-clatworthy) wrote :

Aaron Bentley wrote:
> Ian Clatworthy wrote:
>> It's true that some commands must operate on the convenient
>> form, e.g. shelve, merge.
>
> But even those commands should calculate the result using the canonical
> form, and the write out the convenience form, right?

Thinking about it more, I suspect you're right. We certainly don't want
things like expanded keywords showing as conflicts or interesting stuff
to shelve.

Ian C.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'NEWS'
2--- NEWS 2009-08-24 22:32:53 +0000
3+++ NEWS 2009-08-25 06:35:28 +0000
4@@ -61,6 +61,9 @@
5 Documentation
6 *************
7
8+* New developer documentation for content filtering.
9+ (Martin Pool)
10+
11 API Changes
12 ***********
13
14
15=== modified file 'bzrlib/builtins.py'
16=== modified file 'bzrlib/tests/__init__.py'
17--- bzrlib/tests/__init__.py 2009-08-25 05:04:05 +0000
18+++ bzrlib/tests/__init__.py 2009-08-25 06:35:28 +0000
19@@ -3263,8 +3263,12 @@
20 starting_with=None,
21 runner_class=None,
22 suite_decorators=None,
23+<<<<<<< TREE
24 stream=None,
25 lsprof_tests=False,
26+=======
27+ stream=None,
28+>>>>>>> MERGE-SOURCE
29 ):
30 """Run the whole test suite under the enhanced runner"""
31 # XXX: Very ugly way to do this...
32@@ -3296,12 +3300,18 @@
33 suite = test_suite(keep_only, starting_with)
34 else:
35 suite = test_suite_factory()
36+<<<<<<< TREE
37 if starting_with:
38 # But always filter as requested.
39 suite = filter_suite_by_id_startswith(suite, starting_with)
40 result_decorators = []
41 if lsprof_tests:
42 result_decorators.append(ProfileResult)
43+=======
44+ if starting_with:
45+ # But always filter as requested.
46+ suite = filter_suite_by_id_startswith(suite, starting_with)
47+>>>>>>> MERGE-SOURCE
48 return run_suite(suite, 'testbzr', verbose=verbose, pattern=pattern,
49 stop_on_failure=stop_on_failure,
50 transport=transport,
51@@ -3314,8 +3324,12 @@
52 strict=strict,
53 runner_class=runner_class,
54 suite_decorators=suite_decorators,
55+<<<<<<< TREE
56 stream=stream,
57 result_decorators=result_decorators,
58+=======
59+ stream=stream,
60+>>>>>>> MERGE-SOURCE
61 )
62 finally:
63 default_transport = old_transport
64
65=== modified file 'bzrlib/tests/blackbox/test_selftest.py'
66--- bzrlib/tests/blackbox/test_selftest.py 2009-08-24 22:32:53 +0000
67+++ bzrlib/tests/blackbox/test_selftest.py 2009-08-25 06:35:28 +0000
68@@ -149,6 +149,7 @@
69 return (header,body,footer)
70
71 def test_list_only(self):
72+<<<<<<< TREE
73 # check that bzr selftest --list-only outputs no ui noise
74 def selftest(*args, **kwargs):
75 """Capture the arguments selftest was run with."""
76@@ -176,3 +177,28 @@
77 def test_lsprof_tests(self):
78 params = self.get_params_passed_to_core('selftest --lsprof-tests')
79 self.assertEqual(True, params[1]["lsprof_tests"])
80+=======
81+ # check that bzr selftest --list-only outputs no ui noise
82+ def selftest(*args, **kwargs):
83+ """Capture the arguments selftest was run with."""
84+ return True
85+ def outputs_nothing(cmdline):
86+ out,err = self.run_bzr(cmdline)
87+ (header,body,footer) = self._parse_test_list(out.splitlines())
88+ num_tests = len(body)
89+ self.assertLength(0, header)
90+ self.assertLength(0, footer)
91+ self.assertEqual('', err)
92+ # Yes this prevents using threads to run the test suite in parallel,
93+ # however we don't have a clean dependency injector for commands,
94+ # and even if we did - we'd still be testing that the glue is wired
95+ # up correctly. XXX: TODO: Solve this testing problem.
96+ original_selftest = tests.selftest
97+ tests.selftest = selftest
98+ try:
99+ outputs_nothing('selftest --list-only')
100+ outputs_nothing('selftest --list-only selftest')
101+ outputs_nothing(['selftest', '--list-only', '--exclude', 'selftest'])
102+ finally:
103+ tests.selftest = original_selftest
104+>>>>>>> MERGE-SOURCE
105
106=== modified file 'bzrlib/tests/test_selftest.py'
107--- bzrlib/tests/test_selftest.py 2009-08-25 03:48:15 +0000
108+++ bzrlib/tests/test_selftest.py 2009-08-25 06:35:28 +0000
109@@ -1820,6 +1820,7 @@
110 test_suite_factory=factory)
111 self.assertEqual([True], factory_called)
112
113+<<<<<<< TREE
114 def factory(self):
115 """A test suite factory."""
116 class Test(tests.TestCase):
117@@ -2339,6 +2340,513 @@
118 self.assertEndsWith(result[0], 'one\n')
119 self.assertEqual('', result[1])
120
121+=======
122+ def factory(self):
123+ """A test suite factory."""
124+ class Test(tests.TestCase):
125+ def a(self):
126+ pass
127+ def b(self):
128+ pass
129+ def c(self):
130+ pass
131+ return TestUtil.TestSuite([Test("a"), Test("b"), Test("c")])
132+
133+ def test_list_only(self):
134+ output = self.run_selftest(test_suite_factory=self.factory,
135+ list_only=True)
136+ self.assertEqual(3, len(output.readlines()))
137+
138+ def test_list_only_filtered(self):
139+ output = self.run_selftest(test_suite_factory=self.factory,
140+ list_only=True, pattern="Test.b")
141+ self.assertEndsWith(output.getvalue(), "Test.b\n")
142+ self.assertLength(1, output.readlines())
143+
144+ def test_list_only_excludes(self):
145+ output = self.run_selftest(test_suite_factory=self.factory,
146+ list_only=True, exclude_pattern="Test.b")
147+ self.assertNotContainsRe("Test.b", output.getvalue())
148+ self.assertLength(2, output.readlines())
149+
150+ def test_random(self):
151+ # test randomising by listing a number of tests.
152+ output_123 = self.run_selftest(test_suite_factory=self.factory,
153+ list_only=True, random_seed="123")
154+ output_234 = self.run_selftest(test_suite_factory=self.factory,
155+ list_only=True, random_seed="234")
156+ self.assertNotEqual(output_123, output_234)
157+ # "Randominzing test order..\n\n
158+ self.assertLength(5, output_123.readlines())
159+ self.assertLength(5, output_234.readlines())
160+
161+ def test_random_reuse_is_same_order(self):
162+ # test randomising by listing a number of tests.
163+ expected = self.run_selftest(test_suite_factory=self.factory,
164+ list_only=True, random_seed="123")
165+ repeated = self.run_selftest(test_suite_factory=self.factory,
166+ list_only=True, random_seed="123")
167+ self.assertEqual(expected.getvalue(), repeated.getvalue())
168+
169+ def test_runner_class(self):
170+ self.requireFeature(SubUnitFeature)
171+ from subunit import ProtocolTestCase
172+ stream = self.run_selftest(runner_class=tests.SubUnitBzrRunner,
173+ test_suite_factory=self.factory)
174+ test = ProtocolTestCase(stream)
175+ result = unittest.TestResult()
176+ test.run(result)
177+ self.assertEqual(3, result.testsRun)
178+
179+ def test_starting_with_single_argument(self):
180+ output = self.run_selftest(test_suite_factory=self.factory,
181+ starting_with=['bzrlib.tests.test_selftest.Test.a'],
182+ list_only=True)
183+ self.assertEqual('bzrlib.tests.test_selftest.Test.a\n',
184+ output.getvalue())
185+
186+ def test_starting_with_multiple_argument(self):
187+ output = self.run_selftest(test_suite_factory=self.factory,
188+ starting_with=['bzrlib.tests.test_selftest.Test.a',
189+ 'bzrlib.tests.test_selftest.Test.b'],
190+ list_only=True)
191+ self.assertEqual('bzrlib.tests.test_selftest.Test.a\n'
192+ 'bzrlib.tests.test_selftest.Test.b\n',
193+ output.getvalue())
194+
195+ def check_transport_set(self, transport_server):
196+ captured_transport = []
197+ def seen_transport(a_transport):
198+ captured_transport.append(a_transport)
199+ class Capture(tests.TestCase):
200+ def a(self):
201+ seen_transport(bzrlib.tests.default_transport)
202+ def factory():
203+ return TestUtil.TestSuite([Capture("a")])
204+ self.run_selftest(transport=transport_server, test_suite_factory=factory)
205+ self.assertEqual(transport_server, captured_transport[0])
206+
207+ def test_transport_sftp(self):
208+ try:
209+ import bzrlib.transport.sftp
210+ except ParamikoNotPresent:
211+ raise TestSkipped("Paramiko not present")
212+ self.check_transport_set(bzrlib.transport.sftp.SFTPAbsoluteServer)
213+
214+ def test_transport_memory(self):
215+ self.check_transport_set(bzrlib.transport.memory.MemoryServer)
216+
217+
218+class TestSelftestWithIdList(tests.TestCaseInTempDir, SelfTestHelper):
219+ # Does IO: reads test.list
220+
221+ def test_load_list(self):
222+ # Provide a list with one test - this test.
223+ test_id_line = '%s\n' % self.id()
224+ self.build_tree_contents([('test.list', test_id_line)])
225+ # And generate a list of the tests in the suite.
226+ stream = self.run_selftest(load_list='test.list', list_only=True)
227+ self.assertEqual(test_id_line, stream.getvalue())
228+
229+ def test_load_unknown(self):
230+ # Provide a list with one test - this test.
231+ # And generate a list of the tests in the suite.
232+ err = self.assertRaises(errors.NoSuchFile, self.run_selftest,
233+ load_list='missing file name', list_only=True)
234+
235+
236+class TestRunBzr(tests.TestCase):
237+
238+ out = ''
239+ err = ''
240+
241+ def _run_bzr_core(self, argv, retcode=0, encoding=None, stdin=None,
242+ working_dir=None):
243+ """Override _run_bzr_core to test how it is invoked by run_bzr.
244+
245+ Attempts to run bzr from inside this class don't actually run it.
246+
247+ We test how run_bzr actually invokes bzr in another location.
248+ Here we only need to test that it is run_bzr passes the right
249+ parameters to run_bzr.
250+ """
251+ self.argv = list(argv)
252+ self.retcode = retcode
253+ self.encoding = encoding
254+ self.stdin = stdin
255+ self.working_dir = working_dir
256+ return self.out, self.err
257+
258+ def test_run_bzr_error(self):
259+ self.out = "It sure does!\n"
260+ out, err = self.run_bzr_error(['^$'], ['rocks'], retcode=34)
261+ self.assertEqual(['rocks'], self.argv)
262+ self.assertEqual(34, self.retcode)
263+ self.assertEqual(out, 'It sure does!\n')
264+
265+ def test_run_bzr_error_regexes(self):
266+ self.out = ''
267+ self.err = "bzr: ERROR: foobarbaz is not versioned"
268+ out, err = self.run_bzr_error(
269+ ["bzr: ERROR: foobarbaz is not versioned"],
270+ ['file-id', 'foobarbaz'])
271+
272+ def test_encoding(self):
273+ """Test that run_bzr passes encoding to _run_bzr_core"""
274+ self.run_bzr('foo bar')
275+ self.assertEqual(None, self.encoding)
276+ self.assertEqual(['foo', 'bar'], self.argv)
277+
278+ self.run_bzr('foo bar', encoding='baz')
279+ self.assertEqual('baz', self.encoding)
280+ self.assertEqual(['foo', 'bar'], self.argv)
281+
282+ def test_retcode(self):
283+ """Test that run_bzr passes retcode to _run_bzr_core"""
284+ # Default is retcode == 0
285+ self.run_bzr('foo bar')
286+ self.assertEqual(0, self.retcode)
287+ self.assertEqual(['foo', 'bar'], self.argv)
288+
289+ self.run_bzr('foo bar', retcode=1)
290+ self.assertEqual(1, self.retcode)
291+ self.assertEqual(['foo', 'bar'], self.argv)
292+
293+ self.run_bzr('foo bar', retcode=None)
294+ self.assertEqual(None, self.retcode)
295+ self.assertEqual(['foo', 'bar'], self.argv)
296+
297+ self.run_bzr(['foo', 'bar'], retcode=3)
298+ self.assertEqual(3, self.retcode)
299+ self.assertEqual(['foo', 'bar'], self.argv)
300+
301+ def test_stdin(self):
302+ # test that the stdin keyword to run_bzr is passed through to
303+ # _run_bzr_core as-is. We do this by overriding
304+ # _run_bzr_core in this class, and then calling run_bzr,
305+ # which is a convenience function for _run_bzr_core, so
306+ # should invoke it.
307+ self.run_bzr('foo bar', stdin='gam')
308+ self.assertEqual('gam', self.stdin)
309+ self.assertEqual(['foo', 'bar'], self.argv)
310+
311+ self.run_bzr('foo bar', stdin='zippy')
312+ self.assertEqual('zippy', self.stdin)
313+ self.assertEqual(['foo', 'bar'], self.argv)
314+
315+ def test_working_dir(self):
316+ """Test that run_bzr passes working_dir to _run_bzr_core"""
317+ self.run_bzr('foo bar')
318+ self.assertEqual(None, self.working_dir)
319+ self.assertEqual(['foo', 'bar'], self.argv)
320+
321+ self.run_bzr('foo bar', working_dir='baz')
322+ self.assertEqual('baz', self.working_dir)
323+ self.assertEqual(['foo', 'bar'], self.argv)
324+
325+ def test_reject_extra_keyword_arguments(self):
326+ self.assertRaises(TypeError, self.run_bzr, "foo bar",
327+ error_regex=['error message'])
328+
329+
330+class TestRunBzrCaptured(tests.TestCaseWithTransport):
331+ # Does IO when testing the working_dir parameter.
332+
333+ def apply_redirected(self, stdin=None, stdout=None, stderr=None,
334+ a_callable=None, *args, **kwargs):
335+ self.stdin = stdin
336+ self.factory_stdin = getattr(bzrlib.ui.ui_factory, "stdin", None)
337+ self.factory = bzrlib.ui.ui_factory
338+ self.working_dir = osutils.getcwd()
339+ stdout.write('foo\n')
340+ stderr.write('bar\n')
341+ return 0
342+
343+ def test_stdin(self):
344+ # test that the stdin keyword to _run_bzr_core is passed through to
345+ # apply_redirected as a StringIO. We do this by overriding
346+ # apply_redirected in this class, and then calling _run_bzr_core,
347+ # which calls apply_redirected.
348+ self.run_bzr(['foo', 'bar'], stdin='gam')
349+ self.assertEqual('gam', self.stdin.read())
350+ self.assertTrue(self.stdin is self.factory_stdin)
351+ self.run_bzr(['foo', 'bar'], stdin='zippy')
352+ self.assertEqual('zippy', self.stdin.read())
353+ self.assertTrue(self.stdin is self.factory_stdin)
354+
355+ def test_ui_factory(self):
356+ # each invocation of self.run_bzr should get its
357+ # own UI factory, which is an instance of TestUIFactory,
358+ # with stdin, stdout and stderr attached to the stdin,
359+ # stdout and stderr of the invoked run_bzr
360+ current_factory = bzrlib.ui.ui_factory
361+ self.run_bzr(['foo'])
362+ self.failIf(current_factory is self.factory)
363+ self.assertNotEqual(sys.stdout, self.factory.stdout)
364+ self.assertNotEqual(sys.stderr, self.factory.stderr)
365+ self.assertEqual('foo\n', self.factory.stdout.getvalue())
366+ self.assertEqual('bar\n', self.factory.stderr.getvalue())
367+ self.assertIsInstance(self.factory, tests.TestUIFactory)
368+
369+ def test_working_dir(self):
370+ self.build_tree(['one/', 'two/'])
371+ cwd = osutils.getcwd()
372+
373+ # Default is to work in the current directory
374+ self.run_bzr(['foo', 'bar'])
375+ self.assertEqual(cwd, self.working_dir)
376+
377+ self.run_bzr(['foo', 'bar'], working_dir=None)
378+ self.assertEqual(cwd, self.working_dir)
379+
380+ # The function should be run in the alternative directory
381+ # but afterwards the current working dir shouldn't be changed
382+ self.run_bzr(['foo', 'bar'], working_dir='one')
383+ self.assertNotEqual(cwd, self.working_dir)
384+ self.assertEndsWith(self.working_dir, 'one')
385+ self.assertEqual(cwd, osutils.getcwd())
386+
387+ self.run_bzr(['foo', 'bar'], working_dir='two')
388+ self.assertNotEqual(cwd, self.working_dir)
389+ self.assertEndsWith(self.working_dir, 'two')
390+ self.assertEqual(cwd, osutils.getcwd())
391+
392+
393+class StubProcess(object):
394+ """A stub process for testing run_bzr_subprocess."""
395+
396+ def __init__(self, out="", err="", retcode=0):
397+ self.out = out
398+ self.err = err
399+ self.returncode = retcode
400+
401+ def communicate(self):
402+ return self.out, self.err
403+
404+
405+class TestRunBzrSubprocess(tests.TestCaseWithTransport):
406+
407+ def setUp(self):
408+ tests.TestCaseWithTransport.setUp(self)
409+ self.subprocess_calls = []
410+
411+ def start_bzr_subprocess(self, process_args, env_changes=None,
412+ skip_if_plan_to_signal=False,
413+ working_dir=None,
414+ allow_plugins=False):
415+ """capture what run_bzr_subprocess tries to do."""
416+ self.subprocess_calls.append({'process_args':process_args,
417+ 'env_changes':env_changes,
418+ 'skip_if_plan_to_signal':skip_if_plan_to_signal,
419+ 'working_dir':working_dir, 'allow_plugins':allow_plugins})
420+ return self.next_subprocess
421+
422+ def assertRunBzrSubprocess(self, expected_args, process, *args, **kwargs):
423+ """Run run_bzr_subprocess with args and kwargs using a stubbed process.
424+
425+ Inside TestRunBzrSubprocessCommands we use a stub start_bzr_subprocess
426+ that will return static results. This assertion method populates those
427+ results and also checks the arguments run_bzr_subprocess generates.
428+ """
429+ self.next_subprocess = process
430+ try:
431+ result = self.run_bzr_subprocess(*args, **kwargs)
432+ except:
433+ self.next_subprocess = None
434+ for key, expected in expected_args.iteritems():
435+ self.assertEqual(expected, self.subprocess_calls[-1][key])
436+ raise
437+ else:
438+ self.next_subprocess = None
439+ for key, expected in expected_args.iteritems():
440+ self.assertEqual(expected, self.subprocess_calls[-1][key])
441+ return result
442+
443+ def test_run_bzr_subprocess(self):
444+ """The run_bzr_helper_external command behaves nicely."""
445+ self.assertRunBzrSubprocess({'process_args':['--version']},
446+ StubProcess(), '--version')
447+ self.assertRunBzrSubprocess({'process_args':['--version']},
448+ StubProcess(), ['--version'])
449+ # retcode=None disables retcode checking
450+ result = self.assertRunBzrSubprocess({},
451+ StubProcess(retcode=3), '--version', retcode=None)
452+ result = self.assertRunBzrSubprocess({},
453+ StubProcess(out="is free software"), '--version')
454+ self.assertContainsRe(result[0], 'is free software')
455+ # Running a subcommand that is missing errors
456+ self.assertRaises(AssertionError, self.assertRunBzrSubprocess,
457+ {'process_args':['--versionn']}, StubProcess(retcode=3),
458+ '--versionn')
459+ # Unless it is told to expect the error from the subprocess
460+ result = self.assertRunBzrSubprocess({},
461+ StubProcess(retcode=3), '--versionn', retcode=3)
462+ # Or to ignore retcode checking
463+ result = self.assertRunBzrSubprocess({},
464+ StubProcess(err="unknown command", retcode=3), '--versionn',
465+ retcode=None)
466+ self.assertContainsRe(result[1], 'unknown command')
467+
468+ def test_env_change_passes_through(self):
469+ self.assertRunBzrSubprocess(
470+ {'env_changes':{'new':'value', 'changed':'newvalue', 'deleted':None}},
471+ StubProcess(), '',
472+ env_changes={'new':'value', 'changed':'newvalue', 'deleted':None})
473+
474+ def test_no_working_dir_passed_as_None(self):
475+ self.assertRunBzrSubprocess({'working_dir': None}, StubProcess(), '')
476+
477+ def test_no_working_dir_passed_through(self):
478+ self.assertRunBzrSubprocess({'working_dir': 'dir'}, StubProcess(), '',
479+ working_dir='dir')
480+
481+ def test_run_bzr_subprocess_no_plugins(self):
482+ self.assertRunBzrSubprocess({'allow_plugins': False},
483+ StubProcess(), '')
484+
485+ def test_allow_plugins(self):
486+ self.assertRunBzrSubprocess({'allow_plugins': True},
487+ StubProcess(), '', allow_plugins=True)
488+
489+
490+class _DontSpawnProcess(Exception):
491+ """A simple exception which just allows us to skip unnecessary steps"""
492+
493+
494+class TestStartBzrSubProcess(tests.TestCase):
495+
496+ def check_popen_state(self):
497+ """Replace to make assertions when popen is called."""
498+
499+ def _popen(self, *args, **kwargs):
500+ """Record the command that is run, so that we can ensure it is correct"""
501+ self.check_popen_state()
502+ self._popen_args = args
503+ self._popen_kwargs = kwargs
504+ raise _DontSpawnProcess()
505+
506+ def test_run_bzr_subprocess_no_plugins(self):
507+ self.assertRaises(_DontSpawnProcess, self.start_bzr_subprocess, [])
508+ command = self._popen_args[0]
509+ self.assertEqual(sys.executable, command[0])
510+ self.assertEqual(self.get_bzr_path(), command[1])
511+ self.assertEqual(['--no-plugins'], command[2:])
512+
513+ def test_allow_plugins(self):
514+ self.assertRaises(_DontSpawnProcess, self.start_bzr_subprocess, [],
515+ allow_plugins=True)
516+ command = self._popen_args[0]
517+ self.assertEqual([], command[2:])
518+
519+ def test_set_env(self):
520+ self.failIf('EXISTANT_ENV_VAR' in os.environ)
521+ # set in the child
522+ def check_environment():
523+ self.assertEqual('set variable', os.environ['EXISTANT_ENV_VAR'])
524+ self.check_popen_state = check_environment
525+ self.assertRaises(_DontSpawnProcess, self.start_bzr_subprocess, [],
526+ env_changes={'EXISTANT_ENV_VAR':'set variable'})
527+ # not set in theparent
528+ self.assertFalse('EXISTANT_ENV_VAR' in os.environ)
529+
530+ def test_run_bzr_subprocess_env_del(self):
531+ """run_bzr_subprocess can remove environment variables too."""
532+ self.failIf('EXISTANT_ENV_VAR' in os.environ)
533+ def check_environment():
534+ self.assertFalse('EXISTANT_ENV_VAR' in os.environ)
535+ os.environ['EXISTANT_ENV_VAR'] = 'set variable'
536+ self.check_popen_state = check_environment
537+ self.assertRaises(_DontSpawnProcess, self.start_bzr_subprocess, [],
538+ env_changes={'EXISTANT_ENV_VAR':None})
539+ # Still set in parent
540+ self.assertEqual('set variable', os.environ['EXISTANT_ENV_VAR'])
541+ del os.environ['EXISTANT_ENV_VAR']
542+
543+ def test_env_del_missing(self):
544+ self.failIf('NON_EXISTANT_ENV_VAR' in os.environ)
545+ def check_environment():
546+ self.assertFalse('NON_EXISTANT_ENV_VAR' in os.environ)
547+ self.check_popen_state = check_environment
548+ self.assertRaises(_DontSpawnProcess, self.start_bzr_subprocess, [],
549+ env_changes={'NON_EXISTANT_ENV_VAR':None})
550+
551+ def test_working_dir(self):
552+ """Test that we can specify the working dir for the child"""
553+ orig_getcwd = osutils.getcwd
554+ orig_chdir = os.chdir
555+ chdirs = []
556+ def chdir(path):
557+ chdirs.append(path)
558+ os.chdir = chdir
559+ try:
560+ def getcwd():
561+ return 'current'
562+ osutils.getcwd = getcwd
563+ try:
564+ self.assertRaises(_DontSpawnProcess, self.start_bzr_subprocess, [],
565+ working_dir='foo')
566+ finally:
567+ osutils.getcwd = orig_getcwd
568+ finally:
569+ os.chdir = orig_chdir
570+ self.assertEqual(['foo', 'current'], chdirs)
571+
572+
573+class TestBzrSubprocess(tests.TestCaseWithTransport):
574+
575+ def test_start_and_stop_bzr_subprocess(self):
576+ """We can start and perform other test actions while that process is
577+ still alive.
578+ """
579+ process = self.start_bzr_subprocess(['--version'])
580+ result = self.finish_bzr_subprocess(process)
581+ self.assertContainsRe(result[0], 'is free software')
582+ self.assertEqual('', result[1])
583+
584+ def test_start_and_stop_bzr_subprocess_with_error(self):
585+ """finish_bzr_subprocess allows specification of the desired exit code.
586+ """
587+ process = self.start_bzr_subprocess(['--versionn'])
588+ result = self.finish_bzr_subprocess(process, retcode=3)
589+ self.assertEqual('', result[0])
590+ self.assertContainsRe(result[1], 'unknown command')
591+
592+ def test_start_and_stop_bzr_subprocess_ignoring_retcode(self):
593+ """finish_bzr_subprocess allows the exit code to be ignored."""
594+ process = self.start_bzr_subprocess(['--versionn'])
595+ result = self.finish_bzr_subprocess(process, retcode=None)
596+ self.assertEqual('', result[0])
597+ self.assertContainsRe(result[1], 'unknown command')
598+
599+ def test_start_and_stop_bzr_subprocess_with_unexpected_retcode(self):
600+ """finish_bzr_subprocess raises self.failureException if the retcode is
601+ not the expected one.
602+ """
603+ process = self.start_bzr_subprocess(['--versionn'])
604+ self.assertRaises(self.failureException, self.finish_bzr_subprocess,
605+ process)
606+
607+ def test_start_and_stop_bzr_subprocess_send_signal(self):
608+ """finish_bzr_subprocess raises self.failureException if the retcode is
609+ not the expected one.
610+ """
611+ process = self.start_bzr_subprocess(['wait-until-signalled'],
612+ skip_if_plan_to_signal=True)
613+ self.assertEqual('running\n', process.stdout.readline())
614+ result = self.finish_bzr_subprocess(process, send_signal=signal.SIGINT,
615+ retcode=3)
616+ self.assertEqual('', result[0])
617+ self.assertEqual('bzr: interrupted\n', result[1])
618+
619+ def test_start_and_stop_working_dir(self):
620+ cwd = osutils.getcwd()
621+ self.make_branch_and_tree('one')
622+ process = self.start_bzr_subprocess(['root'], working_dir='one')
623+ result = self.finish_bzr_subprocess(process, universal_newlines=True)
624+ self.assertEndsWith(result[0], 'one\n')
625+ self.assertEqual('', result[1])
626+
627+>>>>>>> MERGE-SOURCE
628
629 class TestKnownFailure(tests.TestCase):
630
631
632=== added file 'doc/developers/content-filtering.txt'
633--- doc/developers/content-filtering.txt 1970-01-01 00:00:00 +0000
634+++ doc/developers/content-filtering.txt 2009-08-25 06:35:28 +0000
635@@ -0,0 +1,147 @@
636+*****************
637+Content Filtering
638+*****************
639+
640+Content filtering is the feature by which Bazaar can do line-ending
641+conversion or keyword expansion so that the files that appear in the
642+working tree are not precisely the same as the files stored in the
643+repository.
644+
645+This document describes the implementation; see the user guide for how to
646+use it.
647+
648+
649+We distinguish between the *canonical form* which is stored in the
650+repository and the *convenient form* which is stored in the working tree.
651+The convenient form will for example use OS-local newline conventions or
652+have keywords expanded, and the canonical form will not. We use these
653+names rather than eg "filtered" and "unfiltered" because filters are
654+applied when both reading and writing so those names might cause
655+confusion.
656+
657+Content filtering is only active on working trees that support it, which
658+is format 2a and later.
659+
660+Content filtering is configured by rules that match file patterns.
661+
662+Filters
663+*******
664+
665+Filters come in pairs: a read filter (reading convenient->canonical) and
666+a write filter. There is no requirement that they be symmetric or that
667+they be deterministic from the input, though in general both these
668+properties will be true. Filters are allowed to change the size of the
669+content, and things like line-ending conversion commonly will.
670+
671+Filters are fed a sequence of byte chunks (so that they don't have to
672+hold the whole file in memory). There is no guarantee that the chunks
673+will be aligned with line endings. Write filters are passed a context
674+object through which they can obtain some information about eg which
675+file they're working on. (See ``bzrlib.filters`` docstring.)
676+
677+These are at the moment strictly *content* filters: they can't make
678+changes to the tree like changing the execute bit, file types, or
679+adding/removing entries.
680+
681+Conventions
682+***********
683+
684+bzrlib interfaces that aren't explicitly specified to deal with the
685+convenient form should return the canonical form. Whenever we have the
686+SHA1 hash of a file, it's the hash of the canonical form.
687+
688+
689+Dirstate interactions
690+*********************
691+
692+The dirstate file should store, in the column for the working copy, the cached
693+hash and size of the canonical form, and the packed stat fingerprint for
694+which that cache is valid. This implies that the stored size will
695+in general be different to the size in the packed stat. (However, it
696+may not always do this correctly - see
697+<https://bugs.edge.launchpad.net/bzr/+bug/418439>.)
698+
699+The dirstate is given a SHA1Provider instance by its tree. This class
700+can calculate the (canonical) hash and size given a filename. This
701+provides a hook by which the working tree can make sure that when the
702+dirstate needs to get the hash of the file, it takes the filters into
703+account.
704+
705+
706+User interface
707+**************
708+
709+Most commands that deal with the text of files present the
710+canonical form. Some have options to choose.
711+
712+
713+Performance considerations
714+**************************
715+
716+Content filters can have serious performance implications. For example,
717+getting the size of (the canonical form of) a file is easy and fast when
718+there are no content filters: we simply stat it. However, when there
719+are filters that might change the size of the file, determining the
720+length of the canonical form requires reading in and filtering the whole
721+file.
722+
723+Formats from 1.14 onwards support content filtering, so having fast
724+paths for the case where content filtering is not possible is not
725+generally worthwhile. In fact, they're probably harmful by causing
726+extra edges in test coverage and performance.
727+
728+We need to have things be fast even when filters are in use and then
729+possibly do a bit less work when there are no filters configured.
730+
731+
732+Future ideas and open issues
733+****************************
734+
735+* We might benefit from having filters declare some of their properties
736+ statically, for example that they're deterministic or can round-trip
737+ or won't change the length of the file. However, common cases like
738+ crlf conversion are not guaranteed to round-trip and may change the
739+ length, so perhaps adding separate cases will just complicate the code
740+ and tests. So overall this does not seem worthwhile.
741+
742+* In a future workingtree format, it might be better not to separately
743+ store the working-copy hash and size, but rather just a stat fingerprint
744+ at which point it was known to have the same canonical form as the
745+ basis tree.
746+
747+* It may be worthwhile to have a virtual Tree-like object that does
748+ filtering, so there's a clean separation of filtering from the on-disk
749+ state and the meaning of any object is clear. This would have some
750+ risk of bugs where either code holds the wrong object, or their state
751+ becomes inconsistent.
752+
753+ This would be useful in allowing you to get a filtered view of a
754+ historical tree, eg to export it or diff it. At the moment export
755+ needs to have its own code to do the filtering.
756+
757+ The convenient-form tree would talk to disk, and the convenient-form
758+ tree would sit on top of that and be used by most other bzr code.
759+
760+ If we do this, we'd need to handle the fact that the on-disk tree,
761+ which generally deals with all of the IO and generally works entirely
762+ in convenient form, would also need to be told the canonical hash to
763+ store in the dirstate. This can perhaps be handled by the
764+ SHA1Provider or a similar hook.
765+
766+* Content filtering at the moment is a bit specific to on-disk trees:
767+ for instance ``SHA1Provider`` goes directly to disk, but it seems like
768+ this is not necessary.
769+
770+
771+See also
772+********
773+
774+* http://bazaar-vcs.org/LineEndings
775+
776+* http://bazaar-vcs.org/LineEndings/Roadmap
777+
778+* `Developer Documentation <index.html>`_
779+
780+* ``bzrlib.filters``
781+
782+.. vim: ft=rst tw=72
783
784=== modified file 'doc/developers/index.txt'
785--- doc/developers/index.txt 2009-08-12 08:03:11 +0000
786+++ doc/developers/index.txt 2009-08-25 06:35:29 +0000
787@@ -126,6 +126,8 @@
788 * `Computing last_modified values <last-modified.html>`_ for inventory
789 entries
790
791+* `Content filtering <content-filtering.html>`_
792+
793 * `LCA Tree Merging <lca_tree_merging.html>`_ |--| Merging tree-shape when
794 there is not a single unique ancestor (criss-cross merge).
795