Merge lp://staging/~akretion-team/openobject-server/openobject-server_5.0_patches into lp://staging/openobject-server/5.0

Backport from trunk to 5-0 lxml + etree standard Python XML libs instead of deprecated xpatch and dom which are not maintained anymore and hence cause a lots of headache to install OpenERP 5-0 branch on Ubuntu distros for instance. Now, OpenERP 5-0 wouldn't even run on new Ubuntu Karmic because no more port of the deprecated packages seems to exist.

In any case, the fix is clean and easy to test. I tested it extensively, I ask partners/community members to do the same. Then it's safe to merge it, it involve limited piece of code for which it's easy to find test cases (use demo data + manufacturing profile + translation for instance). I've been especially cautious with the merges/conflicts and tests. You can trust me at least more than the guys from Tiny who didn't even resolved conflicts properly when merging that old_trunk back into new trunk branch...
We did it carefully. This can be tested.

SO PLEASE, TEST AND TEST AGAIN, BUT PLEASE DO MERGE IT IN 5-0 TO SAVE USERS FROM INSTALLATION HEADHACHES, YOU WILL ALSO AVOID ALL UBUNTU USERS COMING TO THE FORUM AND ASKING WHY THE HELL IT WON'T INSTALL OR BEING DOOMED TO USE OLDER BUGGY VERSIONS OF OPENERP. YOU'LL ALSO AVOID DEBIAN/UBUNTU MAINTAINERS THAT ARE NOT OPENERP SPECIALISTS AT ALL TO MAINTAIN THEMSELVES UGLY PATCHES WHEN THEY HAVE TIME WHICH WILL HARDLY MATCH STABLE RELEASES OF OPENERP.

Thanks in advance

IMHO this is a move like the postgres compatibility fixes that has to be taken anyway in the lifetime of the 5.0 branch. It's better if it's done now than a few months later.

> IMHO this is a move like the postgres compatibility fixes that has to be taken
> anyway in the lifetime of the 5.0 branch. It's better if it's done now than a
> few months later.

+1.

this is also quite important for the debian maintenance of openerp: no (almost) bugfree packages working with lxml means no openerp in squeeze - no openerp in squeeze equals most likely to no openerp in ubuntus next release.

or in otherwords: for the sake of the debian and ubuntu openerp users, please consider applying.

+1

Hi

I desagree with this, The 5.0 stable must only received bugfix, and this issue is not a bug.
I consider the stable version of the server, must be run on the stable version of Debian (actual Lenny) and Ubuntu (Actuel Hardy (8.04 LTS)).

Otherwise if GTK client doesn't work with Karmic, then yes we must correct the issue.

5.2 must be freeze in December, and RC available (March/April) for the Ubuntu latest stable (10.04 LTS)

Regards,

That RC next March and April will hardly be usable in production for most of the people.

5.0 already had much more serious changes than this fix.

> 5.2 must be freeze in December, and RC available (March/April) for the Ubuntu
> latest stable (10.04 LTS)

That's a wrong reason because:
- the Debian maintainer says Debian is going to drop openerp altogether if it's not fixed
- debian freezes in december so the next stable will ship either with an early alpha, or with no openerp at all
- debian imports for ubuntu lucid LTS freeze in February so it will ship with a beta at best, or with no openerp at all

5.0 has a life cycle of 5 years. That means the users of 5.0 will have to stay with lenny and hardy during all five years.

Hello Christophe,

I agree that the change is not small. But don't you agree it's testable?
Frankly using it since yesterday no issue at all. I think some of us could
confirm it safe and after that only we merge it. What I think is that this
is worth the effort, just saying fuck once more to all Ubuntu users for the
next say 6 months will by no means help the product popularity...

I insist, this is not like the report rewrite situation. No logic is changed
here, it's just about swapping a lib for an other. All changes code lines
are easily tested, available here, why not make the effort to give it a test
and then merge it eventually?

My 0.02cts

On Thu, Oct 29, 2009 at 6:28 AM, Christophe Chauvet - http://www.syleam.fr/
<email address hidden> wrote:

> Review: Disapprove
> Hi
>
> I desagree with this, The 5.0 stable must only received bugfix, and this
> issue is not a bug.
> I consider the stable version of the server, must be run on the stable
> version of Debian (actual Lenny) and Ubuntu (Actuel Hardy (8.04 LTS)).
>
> Otherwise if GTK client doesn't work with Karmic, then yes we must correct
> the issue.
>
> 5.2 must be freeze in December, and RC available (March/April) for the
> Ubuntu latest stable (10.04 LTS)
>
> Regards,
>
>
> --
>
> https://code.launchpad.net/~akretion-team/openobject-server/openobject-server_5.0_patches/+merge/14112
> You proposed
> lp:~akretion-team/openobject-server/openobject-server_5.0_patches for
> merging.
>

> 5.0 has a life cycle of 5 years. That means the users of 5.0 will have to stay
> with lenny and hardy during all five years.

Do you know the word "Production" ?

In production we doesn't change all 5 minutes the Ubuntu server because there is a new version.
I have in production, a server that run with Debian Sarge, it work beautiful all the day.

Apply this fix may introduce a dangerous (and uncontrolled) regression, this is a risk that i doesn't want to run.

Regards,

Hi Raphael

> I agree that the change is not small. But don't you agree it's testable?

Yes we can test it, to find all regression, but don't apply in 5.0 (see my last answer)

> Frankly using it since yesterday no issue at all. I think some of us could
> confirm it safe and after that only we merge it. What I think is that this
> is worth the effort, just saying fuck once more to all Ubuntu users for the
> next say 6 months will by no means help the product popularity...

For me, Ubuntu 9.10 is ready for Desktop, but i didn't take the risk to install a server with this version, and prefer 8.04 LTS.

> I insist, this is not like the report rewrite situation. No logic is changed
> here, it's just about swapping a lib for an other. All changes code lines
> are easily tested, available here, why not make the effort to give it a test
> and then merge it eventually?

If i listen this in geek mode, then yes we must apply the merge, but if i listen this in integrator mode, i don't want to see this in stable,

Do you have to test this with Redhat 5.X version (Only python 2.4.3) ? i have test and i doesn't work directly, for customer, we must compile ne python version 2.5, easy_install all packages, but NO CONTRACT SUPPORT FROM REDHAT :( . My reflexion is "Why Debian/Ubuntu and not RedHat/Centos ?"

Regards,

And i forgot, thanks Raphael for this good and meticulous works,

Regards,

We are not sure about the fate of this change because of the lack of unittests.

A good test set (ie one that test every feature) would prove that switching from a library to another does not have an impact on the overall stability of openerp.

Guys,

We all want unit tests, sure. The current one at test.openerpobject.com are
not enough yet, that sure. But nvertheless WE CAN TEST IT OURSELVES, even if
it's more work, I think for exceptionnal things it's valid too. Again, by
knowing OpenERP quite a bite, I affirm that it's quite easy to ensure we
pass in every line changes I made during field_view_get calls (calling
views) and translation opérations. There is no need to tweak any very
special business workflow to do that, we just need to test the regular
things, and more importantly call all the views we can, those will alone
call all those lines, I think others can confirm.

Christophe, I perfectly agree that in production you can use some oldy
Debian or Ubuntu, even if not all customer will like it (remember, lot's of
customers are very small company that already have trouble using Linux,
asking them to use Debian or not the easiest Ubuntu they have on their
Desktop is not appealing to them).
Now, 50% of the reason I want this applied, is US DEVELOPERS/INTEGRATORS and
the all the new potential developpers OpenERP can attract to pound in behind
the tests and contributions. Lot's of us/them will want to use a reasonably
recent Ubuntu distro on their Desktop, and lot's of them will have only one
laptop. By sayingt fuck to them for Hardy, we are certainly loosing lot's of
them for Openbravo, Tryton and god knows what other package.

5.2 might happen in late December (BTW, I'm concerned because I find this
quite soon and the community effort has not been coordinated yet, lot's of
required refactoring might not be undertaken unfortunately), Just like me,
you know perfectly that the slow one year dev cylces Tiny choosed make that
5.2 will be usable in prod only around February at best, because it will
start with lot's of bugs as usual. The more integrators know OpenERP/Tiny,
the more cautious they are to the new version that are developped in the
Indian dark during one year, receiving almost no test at all. Last year,
Smile participated to the 5.0 dev effort, deploying it by December 2008
already, we had to tackle A LOT of bugs with little reward because stability
happened only around March 2009. Meanwhile, I remember the experienced
CampToCamp guys laughing at my face, saying that they will stat testing 5.0
only by February 2009, because they couldn't afford loosing that much time
with bugs/regressions. So my point is that yes, we should take care of 5.2
right now, but it won't be usable any time soon. And in those coming 6
months, I think OpenERP will just suffer a lot in term of image if they
don't have an easy way to install a dev env of OpenERP easily enough on the
soon most popular Linux distro.

Moreover, as Numérigraphe and others say,
doing nothing WILL NOT result in increased stability. It will result in
Debian/Ubuntu dropping recent OpenERP support and offer only very alpha
unstable packages of OpenERP (like 5.0.3) or port it using very dubious
debdiff files Debian/Ubuntu maintainer have no skill to check, meaning bugs
WILL occur on Ubuntu distro; and as I said, this won't solve the dev
community issue, but rather frighten those devs.

Finally, if you look carefully at that bra...

I don't have time to test the patch, I let the quality team do this work. But, following the discussion, I think we should merge the proposed patch. Keeping an old library may leads to more troubles.

The automated tests from test.openobject.com will test a lot of things (all fields_view_get methods are tested at each commit). I would suggest the quality team to launch the automated tests from the base_module_quality tests on all official module. If it passes manual check and automated tests, you should merge.

All right,

I'm happy with this position, I urge all serious community members to test
it on their end, to ensure, wa have no Regression. At Akretion, we are
already two testing it fulltime for our regular projects, if something is
wrong with the patch, we will find out in time.

On Thu, Oct 29, 2009 at 12:18 PM, Fabien (Open ERP) <email address hidden> wrote:

> Review: Approve
> I don't have time to test the patch, I let the quality team do this work.
> But, following the discussion, I think we should merge the proposed patch.
> Keeping an old library may leads to more troubles.
>
> The automated tests from test.openobject.com will test a lot of things
> (all fields_view_get methods are tested at each commit). I would suggest the
> quality team to launch the automated tests from the base_module_quality
> tests on all official module. If it passes manual check and automated tests,
> you should merge.
> --
>
> https://code.launchpad.net/~akretion-team/openobject-server/openobject-server_5.0_patches/+merge/14112
> You proposed
> lp:~akretion-team/openobject-server/openobject-server_5.0_patches for
> merging.
>

+1 for the merge.

But, for a project this size, the lack of unit testing is starting to hurt.

Why don't Tiny and the community organizes a "Test Sprint" where we would dedicate some effort writing unit tests for 5.0/5.2? Yes, they are boring but with that number of regressions, and not-so-frequent releases, I worry about next stable release quality.

If Tiny writes some nice testing guidelines, I'm confident that the community would jump in.

Hello,

I just tested it into a replication of a production instance and it seems that everything is ok with this one. As people said, the lack of unit test starts to hurts, but for now, we have to deal with that...

I suggest to merge that branch even if we should avoid those kind of merge into stable branch, but here, the need is to big to let it only for the next versions...

So, for me, it's ok to merge that into stable release.

My2Cents,

Joël

P.S. We are working on a test bot for functional part here at Camptocamp. This will not be unit test, but more process test. Like : create an invoice, validate it and see the result is ok with tax amount, or try to get all partner, etc... I'll hope to post something soon to have some feed-back on it.

2009/10/30 Joël Grand-Guillaume @ CampToCamp <
<email address hidden>>

>
>
> P.S. We are working on a test bot for functional part here at Camptocamp.
> This will not be unit test, but more process test. Like : create an invoice,
> validate it and see the result is ok with tax amount, or try to get all
> partner, etc... I'll hope to post something soon to have some feed-back on
> it.
>

Joel, sounds great and is required indeed. Don't know what solution you are
putting in place.
But you should know that the best of bread of Ruby functional testing,
"Cucumber" http://cukes.info/ is compatible with OpenERP on the top of my
OOOR http://ooor.googlecode.com/ connector which gives access to all
OpenERP features. I know this is a different language, but if you don't have
good tools enough on Python, might be worth considering it. Note that it's
current to test Java code with Jython or JRuby... And arguably, Cucumber is
more an easy test DSL rather than plain Ruby.
I was also thinking about an enhanced module_recorder version that would
record each XML/RPC request and deal with database dump/restore to allow a
"replay" like feature to easily allow to create test suites. The same could
be achieved using Selenium + the browser eventually, but might just be a
little to harder to maintain.

All right, please all keep testing that branch so in a few days it could be
merged safely.

We are working on function testing :
Our goal:
Each module should has unit test to test functional
like unit test of sale module
 -- Create Sale Order with demo data ( Data is provided in unit test)
 -- confirm This Sale order
 -- Print Sale Order
 -- Make Invoice

This Unit test will test by base_quality_module and also test.openobject.com.
You can see test.py file in trunk-addons : sale/unit_test/test.py

we are using python unittest module (http://docs.python.org/library/unittest.html) to make unit test.

You can see more unit test on this branch :
https://code.launchpad.net/~openerp-commiter/openobject-addons/UnitTestCases

After finsh and review this work, we will merge this branch in trunk-addons

There already seems to have lots of testing efforts. Shouldn'tc we try
to converge them?

CJ
On 30-10-2009 10:37, Harry (Open ERP) wrote:
> We are working on function testing :
> Our goal:
> Each module should has unit test to test functional
> like unit test of sale module
> -- Create Sale Order with demo data ( Data is provided in unit test)
> -- confirm This Sale order
> -- Print Sale Order
> -- Make Invoice
>
> This Unit test will test by base_quality_module and also test.openobject.com.
> You can see test.py file in trunk-addons : sale/unit_test/test.py
>
> we are using python unittest module (http://docs.python.org/library/unittest.html) to make unit test.
>
> You can see more unit test on this branch :
> https://code.launchpad.net/~openerp-commiter/openobject-addons/UnitTestCases
>
> After finsh and review this work, we will merge this branch in trunk-addons
>
>
>
>
>
>
>
>

Hello Cloves,

I suggest you take a serious look ate the smart framwork released by
CampToCamp.
I believe business tests can't be written in a cleaner way today.
I'll make my best to provide a more generic OpenERP proxy to them based on
OOOR as soon as I'll package it as a GEM and remove
the Rails dependency.

Full story here: http://www.openobject.com/forum/post46142.html

On Fri, Oct 30, 2009 at 2:15 PM, Cloves Almeida <email address hidden> wrote:

> There already seems to have lots of testing efforts. Shouldn'tc we try
> to converge them?
>
> CJ
> On 30-10-2009 10:37, Harry (Open ERP) wrote:
> > We are working on function testing :
> > Our goal:
> > Each module should has unit test to test functional
> > like unit test of sale module
> > -- Create Sale Order with demo data ( Data is provided in unit test)
> > -- confirm This Sale order
> > -- Print Sale Order
> > -- Make Invoice
> >
> > This Unit test will test by base_quality_module and also
> test.openobject.com.
> > You can see test.py file in trunk-addons : sale/unit_test/test.py
> >
> > we are using python unittest module (
> http://docs.python.org/library/unittest.html) to make unit test.
> >
> > You can see more unit test on this branch :
> >
> https://code.launchpad.net/~openerp-commiter/openobject-addons/UnitTestCases
> >
> > After finsh and review this work, we will merge this branch in
> trunk-addons
> >
> >
> >
> >
> >
> >
> >
> >
>
> --
>
> https://code.launchpad.net/~akretion-team/openobject-server/openobject-server_5.0_patches/+merge/14112
> You proposed
> lp:~akretion-team/openobject-server/openobject-server_5.0_patches for
> merging.
>

The patch seems working,
I have been using is since 3 days and didn't get any error yet.
But it is not completely remove xml.dom dependency.
It is still exist in
server/bin/pychart/svgcanvas.py
server/bin/report/pyPdf/xmp.py
server/bin/report/render/simple.py
server/bin/report/custom.py
server/bin/tools/translate.py

The reason behind this is
from xml.dom import getDOMImplementation

I think we don't have similar functionality in etree for getDOMImplementation.

Can anybody suggest some way to completely remove xml.dom dependency?
any pychart is third party tool so we have to wait for them to remove xml dependency.

If we extract pychart or pypdf, we can have some bugs in the server, with the packages from the distributions. But we can remove some dependencies to xml.dom. For the rest, we can provide a patch.

Hello, no error on my side too, using it since some 5 days now.
Still, dukai, reported an error with this view: http://pastie.org/681248
I'll test that in a few hours.

About xml.dom, no idea.
BUT, I can tell you that using last Ubuntu Karmic this is not an issue.
So I don't know if xml.dom is deprecated or not, but at least it causes no
trouble on recent Ubuntu's/other Linux distros.

Regards,

Raphaël

On Tue, Nov 3, 2009 at 11:31 AM, hda (OpenERP) <email address hidden> wrote:

> The patch seems working,
> I have been using is since 3 days and didn't get any error yet.
> But it is not completely remove xml.dom dependency.
> It is still exist in
> server/bin/pychart/svgcanvas.py
> server/bin/report/pyPdf/xmp.py
> server/bin/report/render/simple.py
> server/bin/report/custom.py
> server/bin/tools/translate.py
>
> The reason behind this is
> from xml.dom import getDOMImplementation
>
> I think we don't have similar functionality in etree for
> getDOMImplementation.
>
> Can anybody suggest some way to completely remove xml.dom dependency?
> any pychart is third party tool so we have to wait for them to remove xml
> dependency.
>
> --
>
> https://code.launchpad.net/~akretion-team/openobject-server/openobject-server_5.0_patches/+merge/14112
> You proposed
> lp:~akretion-team/openobject-server/openobject-server_5.0_patches for
> merging.
>

Again, let's just check the Dukai issue with view http://pastie.org/681248
before merging.
I have no time now to check. But if one of you can, dukai is hanging right
now on #openobject IRC...

On Tue, Nov 3, 2009 at 11:40 AM, Stephane (Open ERP) <email address hidden>wrote:

> Review: Approve
>
> --
>
> https://code.launchpad.net/~akretion-team/openobject-server/openobject-server_5.0_patches/+merge/14112
> You proposed
> lp:~akretion-team/openobject-server/openobject-server_5.0_patches for
> merging.
>

All right guys,

I spent a few hours digging into Dukai's error case with adding such a view:

<record id="product_product_tree_view_replace" model="ir.ui.view">
            <field name="name">product.product.tree.customization_replace</field>
            <field name="model">product.product</field>
            <field name="type">tree</field>
            <field name="inherit_id" ref="product.product_product_tree_view"/>
            <field eval="7" name="priority"/>
            <field name="arch" type="xml">
              <tree string="Products" position="replace">
                <tree colors="red:virtual_available&lt;0;blue:state in ['draft', 'end', 'obsolete'] or exportable == False" string="Products">
                    <field name="default_code"/>
                    <field name="name"/>
                    <field name="variants"/>
                    <field name="uom_id"/>
                    <field name="qty_available"/>
                    <field name="virtual_available"/>
                    <field invisible="'partner_id' not in context" name="price"/>
                    <field name="lst_price"/>
                <field name="shop_price"/>
                    <field groups="product.group_product_manager" name="standard_price"/>
                    <field groups="base.group_no_one" name="state"/>
                    <field groups="base.group_no_one" name="exportable"/>
                </tree>
                </tree>
            </field>
        </record>

Yes I saw the error and I was unable to find a clean fix.

Now, I tell you, Dukai this is a psycho use case (no offense ;-) )
Indeed, Why the hell would you like to inherit a view to actually replace entirely its fucking content?
IMHO this is plain wrong and never seen such a case.

What you should do instead is just REPLACE the original view by your new definition, that is, simply use the same record id and don't use inherit_id at all.

Of course, REPLACING an existing view is EVIL, just like not calling super when overriding a method. Now, I fully understand why you want to replace it, it's because you would like to cheat the tree attribute and the current OpenERP framework don't let you do that.

This is until why get my blueprint implemented and merged two (all right could be trunk this time if you are hypocritically scared): https://blueprints.launchpad.net/openobject-server/+spec/merge-attributes-view-extension-point

The good part of me spending those few hours now is that I now I know how I'll implement that blueprint in the close future.

NOW, LET'S ME SUM IT UP FOR THAT BRANCH:

Dukai found a psycho fail test case. BUT, he should never have done it this way and there is a better way to do it (use same record attribute no inherit_id) AND I NEVER SEEN SUCH A PSYCHO CASE IN THE ADDONS CODEBASE AT LEAST. Moreover, for the psycho's around, the fix will be dead simple.
So I vote for not blocking this merge under the pretext we found a psycho case breaking it.

I tested it for more than one week, several real customer profile/databases and no bug at all unless we try to blend the views in such irrational ways.

All right,

So I think this is enough testing, we had a fear with D...

Hello Raphaël Valyi,

Your patch has been merged on stable and dukai's problem has been fixed too.
replace whole tree,form.. tag will work now.
please test it once again and send feedback hare.

Thanks for contribution.
HDA.

Hello,

I'm sorry, but why the hell the commit is a normal pig commit and not a
regular clean MERGE?????
Is that the the way you are improving the codebase?
Is there any good reason why it's not a fucking merge?

Let me explain better, this is how it look right now:
suppose I have OpenERP running in production for critical applications,
so I might track the 5.0 branch whenever I want to update the code/fix the
bugs.

Currently, If I look at rev #1866.1.3, all I see is a huge patch that seems
risk with almost no explanation.
So I would think: hey Tiny is not serious, those guys are just committing
large experimental change on claimed stable releases.
Furthermore, with the commit message, you associate my name to such a mess.
I'm sorry, this is exaclty the same story as here
http://www.openobject.com/forum/viewtopic.php?p=46328
repeating over and over. Guys, I'm sorry about that, but for successful
OpenERP we all need more professionalism.

Instead, if you did a clean merge, the enlighten user tracking 5.0 branch
that would tomorrow be your community basis
could instead know the history of the commits, were does this comes from
(trunk and would have been better even on trunk to merge original branches
from Almacom or Activity Solutions (sorry can't remember who did it
initially) properly). The enlighten user could also look here, find out
that this was not yet a risky
large random codebase change because this time it has been done in a branch
and tested for at least a week by several people. Should he have a question
he would get in touch with those people.

So please, unless there is a really good reason, re-do the merge properly in
the history and try to improve in the future, this is more important as what
you might think. Who was wondering 3 years ago those mistakes
http://www.openobject.com/forum/viewtopic.php?p=46328 would lag the project
for so long?
Trying to raise funds? You know what, you'll have experts tracking your
fucking codebase and the more pig it looks, the cheaper you'll sell it. Not
a good strategy.

Sorry for the rant, but I spent time getting this right, so I expect you
behave more professionally to show you do like community contributions.

Regards,

Raphaël Valyi

On Wed, Nov 4, 2009 at 4:35 AM, hda (OpenERP) <email address hidden> wrote:

> Review: Approve
> Hello Raphaël Valyi,
>
> Your patch has been merged on stable and dukai's problem has been fixed
> too.
> replace whole tree,form.. tag will work now.
> please test it once again and send feedback hare.
>
> Thanks for contribution.
> HDA.
> --
>
> https://code.launchpad.net/~akretion-team/openobject-server/openobject-server_5.0_patches/+merge/14112
> You proposed
> lp:~akretion-team/openobject-server/openobject-server_5.0_patches for
> merging.
>

Hello Raphaël Valyi,

Merged your branch properly now.
sorry for inconvenience.

Regards,
HDA.