Storm

Merge lp://staging/~therve/storm/binary-and into lp://staging/storm

  1. binary-and
  2. Merge into trunk
Proposed by Thomas Herve
Status: Work in progress
Proposed branch: lp://staging/~therve/storm/binary-and
Merge into: lp://staging/storm
Diff against target: 305 lines (has conflicts) 
Text conflict in storm/databases/mysql.py
Text conflict in storm/databases/postgres.py
Text conflict in tests/databases/base.py
To merge this branch: bzr merge lp://staging/~therve/storm/binary-and
Reviewer Review Type Date Requested Status
James Henstridge Approve
Gustavo Niemeyer Approve
Review via email: mp+387@code.staging.launchpad.net
To post a comment you must log in.
lp://staging/~therve/storm/binary-and updated
222. By Thomas Herve

 * Fix precedence management. It should now do the right thing under Postgres
 * Rename the XOR operator to BitwiseXOr

223. By Thomas Herve

Make explicit compilation of XOR for postgres backend.

224. By Thomas Herve

Re-add precedence in compile_python.

Revision history for this message
Gustavo Niemeyer (niemeyer) wrote :

Looks great! +1!

review: Approve
Reply
Revision history for this message
James Henstridge (jamesh) wrote :

Looks good, but the compile_python precedences look wrong still. You have:

+compile_python.set_precedence(40, BitwiseNot, BitwiseOr, BitwiseXOr, BitwiseAnd)

While according to http://docs.python.org/ref/summary.html, the 4 operators have different precedences.

According to http://www.postgresql.org/docs/8.3/static/sql-syntax-lexical.html#SQL-PRECEDENCE it looks like the defaults you've got for compile are okay (it puts shifts at the same precendence, but that doesn't matter too much).

review: Approve (tweak)
Reply
Revision history for this message
James Henstridge (jamesh) wrote :

As an example of the python precedence problem, consider the following:

    >>> (1 | 6) & 4
    4

    >>> from storm.expr import compile_python, BitwiseAnd, BitwiseOr
    >>> expr = BitwiseAnd(BitwiseOr(1, 6), 4)
    >>> compile_python(expr)
    '1|6&3'
    >>> eval(compile_python(expr))
    5

More URLs for operator precedence:

    http://www.sqlite.org/lang_expr.html
    http://dev.mysql.com/doc/refman/5.0/en/operator-precedence.html

Reply

Unmerged revisions

224. By Thomas Herve

Re-add precedence in compile_python.

223. By Thomas Herve

Make explicit compilation of XOR for postgres backend.

222. By Thomas Herve

 * Fix precedence management. It should now do the right thing under Postgres
 * Rename the XOR operator to BitwiseXOr

221. By Thomas Herve <therve@twisted>

Add more tests for operator precedence.

220. By Thomas Herve <therve@twisted>

Merge forward

219. By Thomas Herve <therve@twisted>

Use clean way to emulate XOR on sqlite, and fix BitwiseNot precedence for
sqlite < 3.5.3

218. By Thomas Herve <therve@twisted>

Fix precedence of bitwise operators

217. By Thomas Herve <therve@twisted>

s/BitWise/Bitwise/g

216. By Thomas Herve <therve@twisted>

Add XOR support for postgres and sqlite.

215. By Thomas Herve <therve@twisted>

Rename to bitwise, add other operators for standard cases.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'storm/databases/mysql.py'
2--- storm/databases/mysql.py 2008-08-16 07:47:33 +0000
3+++ storm/databases/mysql.py 2009-08-13 20:21:01 +0000
4@@ -30,9 +30,15 @@
5 except ImportError:
6 MySQLdb = dummy
7
8+<<<<<<< TREE
9 from storm.expr import (
10 compile, Insert, Select, compile_select, Undef, And, Eq,
11 SQLRaw, SQLToken, is_safe_token)
12+=======
13+from storm.expr import (compile, Select, compile_select, Undef, And, Eq,
14+ SQLRaw, SQLToken, is_safe_token, BitwiseOr, BitwiseAnd,
15+ BitwiseXOr)
16+>>>>>>> MERGE-SOURCE
17 from storm.variables import Variable
18 from storm.database import Database, Connection, Result
19 from storm.exceptions import (
20@@ -58,6 +64,10 @@
21 return expr
22 return '`%s`' % expr.replace('`', '``')
23
24+compile.set_precedence(51, BitwiseOr)
25+compile.set_precedence(52, BitwiseXOr)
26+compile.set_precedence(53, BitwiseAnd)
27+
28
29 class MySQLResult(Result):
30
31
32=== modified file 'storm/databases/postgres.py'
33--- storm/databases/postgres.py 2009-07-07 02:06:59 +0000
34+++ storm/databases/postgres.py 2009-08-13 20:21:01 +0000
35@@ -42,8 +42,13 @@
36 Undef, Expr, SetExpr, Select, Insert, Alias, And, Eq, FuncExpr, SQLRaw,
37 Sequence, Like, SQLToken, COLUMN, COLUMN_NAME, COLUMN_PREFIX, TABLE,
38 compile, compile_select, compile_insert, compile_set_expr, compile_like,
39+<<<<<<< TREE
40 compile_sql_token)
41 from storm.variables import Variable, ListVariable
42+=======
43+ compile_sql_token, BitwiseXOr)
44+from storm.variables import Variable, ListVariable, RawStrVariable
45+>>>>>>> MERGE-SOURCE
46 from storm.database import Database, Connection, Result
47 from storm.exceptions import (
48 install_exceptions, DatabaseError, DatabaseModuleError, InterfaceError,
49@@ -202,6 +207,10 @@
50 for subexpr in expr.split("."))
51 return compile_sql_token(compile, expr, state)
52
53+@compile.when(BitwiseXOr)
54+def compile_bitwise_xor_postgres(compile, expr, state):
55+ return "%s%s%s" % (compile(expr.expr1, state), "#",
56+ compile(expr.expr2, state))
57
58 class PostgresResult(Result):
59
60
61=== modified file 'storm/databases/sqlite.py'
62--- storm/databases/sqlite.py 2008-06-27 07:43:45 +0000
63+++ storm/databases/sqlite.py 2009-08-13 20:21:01 +0000
64@@ -37,7 +37,8 @@
65 from storm.exceptions import install_exceptions, DatabaseModuleError
66 from storm.expr import (
67 Insert, Select, SELECT, Undef, SQLRaw, Union, Except, Intersect,
68- compile, compile_insert, compile_select)
69+ compile, compile_insert, compile_select, BitwiseXOr, BitwiseNot,
70+ BitwiseAnd, BitwiseOr, compile_binary_oper)
71
72
73 install_exceptions(sqlite)
74@@ -70,6 +71,19 @@
75 return compile_insert(compile, insert, state)
76
77
78+@compile.when(BitwiseXOr)
79+def compile_bitwise_xor_sqlite(compile, expr, state):
80+ # XOR doesn't exist in SQLite, so we have to simulate it
81+ return compile(BitwiseAnd(BitwiseNot(BitwiseAnd(expr.expr1, expr.expr2)),
82+ BitwiseOr(expr.expr1, expr.expr2)),
83+ state)
84+
85+if sqlite is not dummy and sqlite.sqlite_version_info < (3, 5, 3):
86+ # ~ precedence was fixed in 3.5.3, but before we have to force it to have
87+ # a lowest precedence than other bitwise operations
88+ compile.set_precedence(55, BitwiseNot)
89+
90+
91 class SQLiteResult(Result):
92
93 def get_insert_identity(self, primary_key, primary_variables):
94
95=== modified file 'storm/expr.py'
96--- storm/expr.py 2009-07-31 01:53:08 +0000
97+++ storm/expr.py 2009-08-13 20:21:01 +0000
98@@ -1012,6 +1012,14 @@
99 __slots__ = ()
100 oper = "<<"
101
102+class BitwiseAnd(NonAssocBinaryOper):
103+ oper = "&"
104+
105+class BitwiseOr(NonAssocBinaryOper):
106+ oper = "|"
107+
108+class BitwiseXOr(NonAssocBinaryOper):
109+ oper = "^"
110
111 class Like(BinaryOper):
112 __slots__ = ("escape", "case_sensitive")
113@@ -1300,6 +1308,10 @@
114 __slots__ = ()
115 suffix = "DESC"
116
117+class BitwiseNot(PrefixExpr):
118+ prefix = "~"
119+
120+
121
122 # --------------------------------------------------------------------
123 # Plain SQL expressions.
124@@ -1434,16 +1446,18 @@
125 compile.set_precedence(30, Or)
126 compile.set_precedence(40, And)
127 compile.set_precedence(50, Eq, Ne, Gt, Ge, Lt, Le, Like, In)
128-compile.set_precedence(60, LShift, RShift)
129-compile.set_precedence(70, Add, Sub)
130-compile.set_precedence(80, Mul, Div, Mod)
131+compile.set_precedence(60, BitwiseNot, BitwiseOr, BitwiseXOr, BitwiseAnd)
132+compile.set_precedence(70, LShift, RShift)
133+compile.set_precedence(80, Add, Sub)
134+compile.set_precedence(90, Mul, Div, Mod)
135
136 compile_python.set_precedence(10, Or)
137 compile_python.set_precedence(20, And)
138 compile_python.set_precedence(30, Eq, Ne, Gt, Ge, Lt, Le, Like, In)
139-compile_python.set_precedence(40, LShift, RShift)
140-compile_python.set_precedence(50, Add, Sub)
141-compile_python.set_precedence(60, Mul, Div, Mod)
142+compile_python.set_precedence(40, BitwiseNot, BitwiseOr, BitwiseXOr, BitwiseAnd)
143+compile_python.set_precedence(50, LShift, RShift)
144+compile_python.set_precedence(60, Add, Sub)
145+compile_python.set_precedence(70, Mul, Div, Mod)
146
147
148 # --------------------------------------------------------------------
149
150=== modified file 'tests/databases/base.py'
151--- tests/databases/base.py 2009-07-30 06:19:27 +0000
152+++ tests/databases/base.py 2009-08-13 20:21:01 +0000
153@@ -26,7 +26,12 @@
154 import os
155
156 from storm.uri import URI
157+<<<<<<< TREE
158 from storm.expr import Select, Column, SQLToken, SQLRaw, Count, Alias
159+=======
160+from storm.expr import (Select, Column, SQLToken, SQLRaw, Count, Alias,
161+ BitwiseXOr, BitwiseOr, BitwiseAnd, BitwiseNot, Add)
162+>>>>>>> MERGE-SOURCE
163 from storm.variables import (Variable, PickleVariable, RawStrVariable,
164 DecimalVariable, DateTimeVariable, DateVariable,
165 TimeVariable, TimeDeltaVariable)
166@@ -378,6 +383,7 @@
167 result.from_database = self.from_database
168 self.assertEquals(iter(result).next(), (2, 3))
169
170+<<<<<<< TREE
171 def test_rowcount_insert(self):
172 # All supported backends support rowcount, so far.
173 result = self.connection.execute(
174@@ -395,6 +401,55 @@
175 "UPDATE test SET title='whatever'")
176 self.assertEquals(result.rowcount, 2)
177
178+=======
179+ def test_bitwise_xor_support(self):
180+ expr = Select(BitwiseXOr(Variable(1), Variable(2)))
181+ result = self.connection.execute(expr)
182+ self.assertEquals(result.get_all(), [(3,)])
183+
184+ def test_bitwise_precedence(self):
185+ expr = Select(BitwiseOr(BitwiseAnd(Variable(1), Variable(3)),
186+ BitwiseAnd(Variable(1), Variable(2))))
187+ result = self.connection.execute(expr)
188+ self.assertEquals(result.get_all(), [(1,)])
189+
190+ expr = Select(BitwiseAnd(BitwiseOr(Variable(2), Variable(3)),
191+ BitwiseOr(Variable(1), Variable(3))))
192+ result = self.connection.execute(expr)
193+ self.assertEquals(result.get_all(), [(3,)])
194+
195+ expr = Select(BitwiseAnd(BitwiseOr(BitwiseNot(Variable(2)),
196+ Variable(3)),
197+ Variable(1)))
198+ result = self.connection.execute(expr)
199+ self.assertEquals(result.get_all(), [(1,)])
200+
201+ def test_bitwise_precedence_python(self):
202+ expr = Select(Add(BitwiseNot(2), 3))
203+ result = self.connection.execute(expr)
204+ self.assertEquals(result.get_all(), [(0,)])
205+
206+ expr = Select(BitwiseOr(BitwiseAnd(1, 3),
207+ BitwiseAnd(1, 2)))
208+ result = self.connection.execute(expr)
209+ self.assertEquals(result.get_all(), [(1,)])
210+
211+ expr = Select(BitwiseAnd(BitwiseOr(2, 3),
212+ BitwiseOr(1, 3)))
213+ result = self.connection.execute(expr)
214+ self.assertEquals(result.get_all(), [(3,)])
215+
216+ expr = Select(BitwiseAnd(BitwiseOr(BitwiseNot(2),
217+ 3),
218+ 1))
219+ result = self.connection.execute(expr)
220+ self.assertEquals(result.get_all(), [(1,)])
221+
222+ expr = Select(Add(BitwiseNot(2), 3))
223+ result = self.connection.execute(expr)
224+ self.assertEquals(result.get_all(), [(0,)])
225+
226+>>>>>>> MERGE-SOURCE
227
228 class UnsupportedDatabaseTest(object):
229
230
231=== modified file 'tests/databases/postgres.py'
232=== modified file 'tests/databases/sqlite.py'
233--- tests/databases/sqlite.py 2008-06-27 07:49:40 +0000
234+++ tests/databases/sqlite.py 2009-08-13 20:21:01 +0000
235@@ -18,9 +18,7 @@
236 # You should have received a copy of the GNU Lesser General Public License
237 # along with this program. If not, see <http://www.gnu.org/licenses/>.
238 #
239-from datetime import timedelta
240 import time
241-import os
242
243 from storm.exceptions import OperationalError
244 from storm.databases.sqlite import SQLite
245
246=== modified file 'tests/expr.py'
247--- tests/expr.py 2009-07-23 22:47:10 +0000
248+++ tests/expr.py 2009-08-13 20:21:01 +0000
249@@ -1052,6 +1052,34 @@
250 self.assertEquals(statement, "func1()>>?")
251 self.assertVariablesEqual(state.parameters, [Variable("value")])
252
253+ def test_bitwise_and(self):
254+ expr = BitwiseAnd(Func1(), Func2())
255+ state = State()
256+ statement = compile(expr, state)
257+ self.assertEquals(statement, "func1()&func2()")
258+ self.assertEquals(state.parameters, [])
259+
260+ def test_bitwise_or(self):
261+ expr = BitwiseOr(Func1(), Func2())
262+ state = State()
263+ statement = compile(expr, state)
264+ self.assertEquals(statement, "func1()|func2()")
265+ self.assertEquals(state.parameters, [])
266+
267+ def test_bitwise_xor(self):
268+ expr = BitwiseXOr(Func1(), Func2())
269+ state = State()
270+ statement = compile(expr, state)
271+ self.assertEquals(statement, "func1()^func2()")
272+ self.assertEquals(state.parameters, [])
273+
274+ def test_bitwise_not(self):
275+ expr = BitwiseNot(Func1())
276+ state = State()
277+ statement = compile(expr, state)
278+ self.assertEquals(statement, "~ func1()")
279+ self.assertEquals(state.parameters, [])
280+
281 def test_like(self):
282 expr = Like(Func1(), "value")
283 state = State()
284@@ -2112,6 +2140,21 @@
285 self.assertEquals(py_expr, "_0>>_1")
286 self.assertEquals(state.parameters, [1, 2])
287
288+ def test_bitwise_and(self):
289+ expr = BitwiseAnd(Variable(1), Variable(2))
290+ py_expr = compile_python(expr)
291+ self.assertEquals(py_expr, "1&2")
292+
293+ def test_bitwise_or(self):
294+ expr = BitwiseOr(Variable(1), Variable(2))
295+ py_expr = compile_python(expr)
296+ self.assertEquals(py_expr, "1|2")
297+
298+ def test_bitwise_xor(self):
299+ expr = BitwiseXOr(Variable(1), Variable(2))
300+ py_expr = compile_python(expr)
301+ self.assertEquals(py_expr, "1^2")
302+
303 def test_in(self):
304 expr = In(Variable(1), Variable(2))
305 state = State()

Subscribers

People subscribed via source and target branches

to status/vote changes: