Related Pages: [ FEE Paris 2000 ] [ Enterprises 2002 ] [ Euro Compliance ] [ 7 steps to success ] [ Eurotown Project ] [ Lessons learnt ] [ UK Decimalisation ] [ Euro Implications ] [ Managing Risk ] [ SME Changeover ] [ Euro Myths/Facts ] [ Leclerc 1996 trial ]

Euro 'Compliance' Certification

Quality issues in test cases for euro currency conversion

As part of a 1999 survey of accounting packages on sale in Ireland, I created a test scenario representing how a company would account for trading over the transition period. and applied it to three representative packages (single, dual, and multi-currency).

I identified a failure in a commercial small & medium enterprise (SME) accounting package that had been certified to the level 1 (1998) BASDA (http://www.basda.org) standard. Specifically, it converted EUR 39.00 to IEP 30.72 rather than 30.71. This is not an isolated incident - it happens in 50 price points under 100 euro or about 0.5% of cases. The suppliers fixed this free as part of normal update support. Once I discovered the problem, it was trivial to construct other tests to show that the software would fail in other currencies and triangulation (IEP 2.41 <--> EUR 3.06 <--> DEM 5.98 but get 5.99). Therefore, the first supplied version of the package strictly speaking failed the requirements for conversion and rounding expressed in Council Regulation 1103/97.

The original test cases (V3.3 parts 3.1.2.1 and 3.1.2.2 ) used arbitrary large numbers like 123,456.75 which miss the parts that users will reach someday. I understand they added a case to check for this "double rounding" error. However, even those are better than a test case I saw in a German national standard which uses even smaller round numbers like DEM 1740 and EUR 2000. And  the French testing authorities will not let anyone see their test specification!

An industry precedent to this is the Pentium FDIV bug from late 1994. The testing guru Boris Beizer pointed out that although Intel had tested the chip with "millions" of tests, they missed the flaws that became evident when a user reported the problem. He devised a set of ten tests given the architecture of the chip, and it failed on the third test.

A software tester needs to be like a sculptor or diamond cutter looking for the flaw line where a small tap cracks the whole thing open. I created tests based on the mathematics of the conversion between the euro and the Irish Pound. I emphasise that this is not the expected discrepancy from the "price accordion" (where price points in euro get squeezed out when represented in IEP) but the fact that crucial rounding boundaries were not covered in the test cases.

I'm not naming the one certified package I checked as my focus is on the certification test pack itself. As that test did not disclose the failure I found, all 25 certified packages theoretically could have that problem and may need to be rechecked with that extra test. If you are concerned, do your own test on your software with those numbers and call the supplier for your free upgrade if you find it. I have also checked some Windows and website euro calculators and found similar errors there.

Let's be realistic about this, the problem must be of little real-world interest to the (typically SME) purchasers of the software package I tested as they had not themselves as users raised the issue. A cent rounding difference is evidently not so important to them or they had simply not used the package enough or verified the results independently. Having said that, I would rather have a standard that was exacting and described where packages submitted deviated, so the user can make the decision for themselves. And if software suppliers may want to use that story to say in effect "we can certify ourselves", as in the IBM France catalogue, I should tell the story of an uncertified package from a small supplier that had three serious faults as first supplied in 1999:

a) They converted an account the wrong way
b) The program crashed if a certain sequence of actions was taken
c) Entering a euro invoice in a pound base currency could put the trial balance out by a penny.

They fixed them as I reported them in the course of the survey, but it shows that independent testing has value as well as limitations. And after all, what have I missed that is still to emerge?

But in the world of software engineering, it's a point to make about testing methodology. At the Unicom Dev-Test conference in London on March 22 where I presented my findings, Paul Gerrard (of www.evolutif.co.uk who run testing courses) characterised good testers as "nit-picking and pedantic", who are sceptical and love to break things, find out why, and show how to fix them so they don't break so easily. I will be making the presentation notes available to anyone who owns a copy of my book via an online support link at http://www.sysmod.com/maneuris.htm

In a subsequent article, I would like to focus on certification of software packages for euro processing, and the accreditation of the testing body itself by national standards institutes.

If you would like to express an opinion on this, please use the feedback form. Questions I propose to address are:

1) From the user point of view who wants certification? What do you understand by "euro-compliance"? What tolerances do you apply in real life and how much are you concerned by rounding errors? How much are you prepared to pay for the "certified" sticker?

2) From the vendor POV: Why be certified? Which accreditation authority would you use and why? How much are you prepared to pay for what degree of testing and certification?

3) From the POV of testers and accreditation bodies: How far should one go in testing software for euro handling? Who defines the requirements?

4) From the POV of information providers (magazines, euro changeover boards, business information services...): how can all this be made comprehensible? What are sensible criteria? What use is it to readers to reduce complex tests to a single checklist item "euro-compliant Yes/No"?

Patrick O'Beirne.