… Big …

It doesn’t get much bigger than this: 30 years after first joining, China has become the sixth permanent member of ISO, joining the original 5: the United States, Germany, Britain, France and Japan. More here.

… and Small …

Zooming from the world’s most populous nation to the level of the individual, Patrick Durusau has written a new piece entitled “My Standards Education” which (characteristically) presents a positive message, encouraging people who care, to get involved in standards. Anyone who has the wherewithal and is not currently contributing could does a lot worse than take up one of his suggestions and join OASIS – perhaps to contribute to the vitally important work of the new ODF Interoperability and Conformance TC. More about that particular effort from Rob Weir here.

Come on people – as Patrick would happily observe, we won’t get better standards just by writing blogs …

[Update: Patrick has just issued another post, on one aspect of different approaches ODF and OOXML take to markup.]

Later this week I will be participating in one of Microsoft’s DII (Digital Interoperability Initiative) workshops in Redmond (full disclosure: my company will be reclaiming travel and accommodation expenses from Microsoft) and am looking forward to assessing some of the work in this area not as a standards person, but as a (less circumscribed) free agent.

Now, “interoperability” has become a slightly contentious word. Sun’s Simon Phipps, for example, while speaking at The 2nd International ODF User Workshop showed a slide in which he argued for “Substitutability not Interoperability”. I am not sure there is a meaningful distinction between the two terms but it is true “substitutability” does convey forcibly what (I take it) the user requirement is in this area: to be able to take advantage of open standard formats to process data with a choice of software, and to avoid application lock-in.

The word “interoperability” also no doubt has significance for Microsoft in that it is at the centre of the questions currently being asked by the European Commission, not least in the ongoing anti-trust investigations. Microsoft have responded by conspicuously committing to an interoperability programme and have made noises of the right kind in press releases, which have drawn a very guarded response from the EU:

The Commission would welcome any move towards genuine interoperability. Nonetheless, the Commission notes that today's announcement follows at least four similar statements by Microsoft in the past on the importance of interoperability.

- there is definitely a sense that Microsoft are drinking at the last-chance saloon here.

Meanwhile, on the other hand Microsoft’s competitors are keen for Microsoft to fail. The game being played has several outcomes in which the key measures are whether Microsoft achieves interoperability in practice, and whether it is perceived to do have done so.

A. Microsoft fails to achieve interoperability, and is correctly perceived to have failed.

B. Microsoft does achieve interoperability, but is incorrectly perceived to have failed.

C. Microsoft fails to achieve interoperability, but is incorrectly perceived to have succeeded.

D. Microsoft does achieve interoperability, and is correctly perceived to have done so.

The optimal outcome for Microsoft (I assume it is a sane corporate entity, and so tends to monopoly) is box C. They effectively get to maintain barriers to market entry but are judged to have played well: the EU threat goes away.

The optimal outcome for Microsoft’s competitors (I assume they are also sane) is box B. In this outome, the market becomes more open to them as a result of Microsoft’s initiatives, but Microsoft is unjustly hammered for their efforts and have to deal with the regulatory and PR fallout.

As usual we poor users (and we poor standards people) are stuck in between. The best outcome for us is either box A or box D (with a preference for box D). We end up with the freer market that interoperability promises either through Microsoft’s cooperation (D), or  - less efficiently - through punitive action taken by a regulatory system (A).

Astute readers will note that the optimal outcomes for the corporations (both Microsoft and its competitors) have a defining property: they both rely on a mismatch between actuality and perception - misinformation. It is for this reason, I believe, that there has been such an all-out assault on the blogosphere by some of Microsoft’s competitors (and, to a lesser extent, by Microsoft itself). Have no doubt about it, this is war - as IBM’s Rob Weir put it, "It’s called a 'standards war'. Look it up. Whining about it won't make it go away".

And so, as in any war, truth is at a premium. Fortunately, when it come to Office Document interoperability we do not need to rely on bitter blog exchanges, the warm words of press releases, or even on the success (or not) of workshops in Redmond. Questions of interoperability will be decided by the cold hard fact that certain bytes arranged in certain ways will betoken good behaviour; other arrangements will betoken bad behaviour. We, the users, can measure which is which, and we, the users, can improve the tests by making the standards that govern office document more thorough. If we deal honestly and standardise well, the optimal outcome for us is within reach.

Reviewing the pieces

My particular interest in interoperability is between Office document formats – word processing documents in particular. The international standardisation of ODF 1.0 (IEC/ISO 26300:2006) and OOXML (IEC/ISO 29500:2008) has sparked off plenty of commentary, and also plenty of opportunities for the spreading of misinformation. I think, in the interests of “honest dealing” it will be useful to have a review of the current state of play: exactly what specifications do we have on the table?

“ODF”

“ODF” is a term that is often used very casually. However, when examining interoperability, details and exactness matter so let us enumerate the 4 main variants (in 5 documents) that we have:

1. The OpenDocument v1.0 specification was approved as an OASIS Standard on 1 May 2005. It is not an ISO/IEC standard.
2. ISO/IEC 26300:2006 Open Document Format for Office Applications (OpenDocument) v1.0 is a version of the OASIS 1.0 standard with substantive revisions, as applied through the JTC 1 standardisation process. It is an ISO/IEC, but not an OASIS standard.
3. The OpenDocument v1.0 (Second Edition) specification was approved as an OASIS Committee Specification on 19 July 2006. It is practically identical to ISO/IEC 26300:2006. It is not an OASIS standard.

(Note: that the above three “1.0” variants are, practically, obsolete – as a co-chair of the OASIS ODF TC put it, “No one supports ODF 1.0 today”.)

4. ODF 1.1 was approved as OASIS Standard on 2 February 2007. It is a small but significant update of its predecessor version. It is an OASIS standard, but not an ISO/IEC standard.
5. “ODF 1.2” is being drafted. It is not an OASIS standard. It is not an ISO/IEC standard. Nobody knows for sure what it will end up containing. It does not formally exist.

Estimates vary about when 1.2 will be finished and standardised. A reasonable guesstimate would be that is will be published as an OASIS standard in Q1 2009, and – if it is submitted to, and passes, a JTC 1 process – become an International Standard in Q1 2010. But these are my guesstimates based on conversations with OASIS and ISO/IEC people.

Now practically speaking there are some key points to note about these versions:

• The variant most in use currently is (4) – “ODF 1.1”. This is the variant to which the output of most “ODF” applications most nearly approximates.
• The variant that many countries and big international organisations have signed up for is (2) – that is because it has the all-important ISO/IEC (not OASIS) imprimatur.
• The recently released OpenOffice.org 3.0 application promises support for what is called “features of the upcoming version 1.2 of the ISO standard OpenDocument Format (ODF)”. Not quite right – and the kind of casual wearing of the “ISO” (actually ISO/IEC) brand that is causing deep disquiet among its custodians. The format that OpenOffice.org 3.0 would be more properly labelled  “OpenOffice.org 3.0 format”, and described as a guess by that product’s developers at what ODF 1.2 might eventually end up being.
• I have seen mentions in one or two anti-MS places that Microsoft “should be adopting ODF 1.2” in their upcoming version of Office. Absolutely not! The last thing we should be doing is encouraging Microsoft to target a non-existent format, thereby allowing their developers to improvise (just as OpenOffice.org’s have). It’s not hard to see how that would almost certainly lead to an interoperability disaster.

“OOXML”

The situation for OOXML is simpler, in part because it is a newer standard.

But first, the name. “Office Open XML File Formats” is what it is called. At the SC 34 Jeju plenary it had reached the third time when somebody had mistakenly drafted projected text mentioning “Open Office” rather than “Office Open” when I turned in exasperation to the delegate next to me and whispered “whoever came up with that name deserves to be shot”. The reply? “Errr, it was me.” The responsible Microsoftie shall remain nameless.

Anyway, if there is one thing worse than calling it “Office Open XML” is it the abbreviated form “Open XML”. That poor word “open” has had such a battering – and this standard really does not need a nickname. No, in my book “OOXML” or (better) “twenty-nine-five-hundred” it has to be.

There are two main variants of this specification:

1. Ecma 376. This was the specification prepared by Ecma International as an input into the Fast Track JTC 1 process. It also represents the format currently used by Microsoft Office 2007. I have heard mutterings that Microsoft Office fails to consume and emit conformant Ecma 376, but no test I have carried out or heard of suggests that these mutterings are true.
2. ISO/IEC 29500:2008. This is the standardised, but yet-to-be-published version of OOXML that has been very substantially revised by the nations participating in the JTC 1 Fast Track standardisation process. The publication text has already been distributed to the standardisation nations for their information, and the standard will appear later (guesstimate: December 2008). So, strictly speaking this standard (like ODF 1.2) does not formally exist yet. Note that this specification has become a multi-part standard – and so is effectively now four standards:
   • Part 1: Fundamentals and Markup Language Reference. The core of the standard.
     
   • Part 2: Open Packaging Conventions. A small  specification describing a ZIP-based packaging mechanism. This is generally rather liked and there has been some talk of future versions of ODF borrowing it.
   • Part 3: Markup Compatibility and Extensibility. A small specification describing how the markup underlying OOXML may be extended.
   • Part 4: Transitional Migration Features. This is the “toxic dump” – all those nasty features like the infamous autoSpaceLikeWord95 are confined to this section, which may be ignored by implementations pursuing "strict" conformance.

There is currently no implementation of ISO/IEC 29500. The current version of Micosoft Office does not emit ISO/IEC 29500 compliant documents because of some minor lexical differences between Ecma 376 and the ISO/IEC evolution of the standard (using, of course, its Part 4 - Transitional Migration Features).

Looking forward

So we have reviewed the set of specifications we are concerned with when it comes to testing interoperability. In my next post I will describe a simple technical test (which I will be taking to Redmond) that we can use with some of the major implementations to get some evidence on what the state of “interoperability” really is at the moment.

As companion pieces to those on JTC 1 Reform, I thought it would be interesting to look at some recent, not so-recent, and proposed initiatives taking place in International Standardisation that put web technology at the centre of the standardisation process.

BSI’s DPC System

The first of these is BSI British Standards Draft Review system. This recently-launched system makes available a number of Drafts for Public Comment (DPCs), and allows the general public to record comments on them with an easy-to-use web interface that permits any clause to be commented-on. As BSI explains:

Standards documents are circulated for public comment in order to get comments from as wide an audience as possible. The DPC stage occurs during drafting in national, European and international arenas and is an important part of the standards development process.

Those of you who have been following along purely for their interest in OOXML will remember that there was a public comment stage for that specification too (in the summer of 2007), in which members of the public submitted comments (then typically by email) which were fed into the process. Granted that mostly meant many copies of the then web-available objections were submitted – but there were some nuggets of original criticism too.

Wider spread

Again, those of you have been purely interested in OOXML will note the wide range of types of standard considered here. International standardisation is about much more than ISO and IEC, and here you can find drafts of CEN and CENELEC (European) standards and well as good old British Standards too.

So far as I am aware, this BSI system is blazing a trail on the international standards scene: it would be good to see other NBs too adopting such mechanisms for public comment collection. Even better if they adopted the same web-based APIs!

So come on, (British) readers: if you have any burning thoughts you wish to contribute on “Thief resistant lock assembly - Key egress” or “Code of practice for information and communications technology continuity”, then please do so.

The ongoing controversies surrounding the standardization of ISO/IEC 26300:2006 (OpenDocument Format 1.0) and ISO/IEC 29500:2008 (Office Open XML) have served to highlight several weaknesses in the International Standardisation processes for ICT specifications handled by JTC 1, the “Joint Technical Committee” that combines aspects of ISO and IEC for that purpose.
 
However newsworthy these particular projects have been, the underlying problems in JTC 1 go deeper, and I believe is incumbent on all who care to avoid solutions which smack of “single issue politics” – hard cases, after all, make bad law.
 
This post is the first in a series which aims to set out proposals for broad reforms that could help ensure JTC 1’s future. These posts are offered “out of process” as an informal starting point. They, like everything on this blog, are personal views – but have been informed by discussions with many experienced standardisers from many nations over the course of the last two years. My intention is that after the posts have been completed they will be assembled into a short paper (taking comments into account) which, I hope, may influence the onward international debate.

10 Recommendations for Reform

I make 10 recommendations and shall explore each in 10 forthcoming posts. In summary, they are:

1. Assert the worth of International Standardization

There are many organisations producing standards, but International Standardisation is the preserve of the de-jure organisations alone; this difference benefits the peoples of the world and must be preserved. The encroachment of “standardisation by corporation” must be resisted. More...

2. Recognise the distinctive requirements of ICT Standardisation

ICT standards are different from standards for piping, wiring or management processes; ISO and IEC need to make JTC 1 the sole steward of this distinctive subject area.

3. Re-draft the JTC 1 Directives

The JTC 1 Directives are an embarrassment; the current piecemeal patching efforts have palpably failed and serve only to empower administrators over nations. The Directives need to be re-drafted from scratch by professionals.

4. Move away from paper-based publication models

The current business model and many procedures within JTC 1 are predicated on producing and selling paper publications; this unnecessarily impedes the ICT standardisation process.

5. Widen International participation

In practice JTC 1 is currently dominated by a cosy club of rich, experienced nations; JTC 1 needs pursue a programme for fostering a much greater (i.e. genuinely international) reach.

6. Find a way for vendor-led standards to mesh with JTC 1 processes without compromising International control

Worthwhile standards will originate from outside JTC 1; a way must be found to make them International Standards avoiding the manifest flaws of the current accelerated adoption mechanisms.

7. Periodically change the nation having the Secretariat and Chair appointments

It is absurd that a purportedly International organisation has its effective HQ lodged for perpetuity in the USA.

8. Balance transparency and confidentiality

Openness and transparency can lead to better standardisation, but are by no means panaceas.

9. Clarify intellectual property policies

International Standards must have clearly stated IP policies, and avoid unacceptable patent encumbrances. More...

10. Encourage best practices at National level

International Standards rely on the efforts of the sovereign Nations that participate; JTC 1 should encourage these Nations to raise their games.

This blog theme is a bit ... austere.

Here's  a pici to cheer things up before I get the place nicely redecorated.

From jeju 2008