Mastodon
Where is there an end of it? | The Maintenance of ODF – an Aide-mémoire

Where is there an end of it?

Alex Brown's weblog

The Maintenance of ODF – an Aide-mémoire

06 November 2008 Alex (10)

There is some inaccurate information swirling around on the web about the maintenance of ISO/IEC 26300:2006 – Open Document Format for Office Applications (OpenDocument) v1.0.

For those following the story of document format standardisation, this blog entry sets out the current situation ahead of the upcoming JTC 1 plenary in Nara, Japan, where this very topic is likely to be discussed and, one hopes, get debugged.

Background

The diagram above illustrates the current and planned major variants of the ODF standard.

The topmost is the OASIS standard 1.0, published by OASIS afters its approval in May 2005.

This OASIS standard was submitted by OASIS to JTC 1 for PAS transposition in October 2005. It passed its ballot with no dissent in May 2006, although a number of countries requested substantive fixes and improvements.

Because there had been no negative votes (only approves and abstention) in the ballot, the ballot resolution meeting (BRM) for the new standard was cancelled. (The UK objected to this decision at the May 2006 SC 34 plenary meeting in Seoul.)

Based on the comments from NBs, some substantive fixes and improvements were duly made to ODF, and ITTF incorporated these into the text of ISO/IEC 26300:2006, published in November 2006.

An equivalent text, an OASIS Committee Specification (not a standard, N.B.) called “OpenDocument v1.0 (Second Edition)”, had been published by OASIS in July 2006.

OASIS subsequently authored and published a new OASIS standard, ODF 1.1. This was published three months after ISO 26300:2006, i.e. in February 2007. OASIS did not seek cooperation in this from any part of ISO/IEC, nor did them submit the revised specification to JTC 1.

OASIS then began work on ODF 1.2, again without any ISO/IEC involvement.

In July 2008 the co-chair of the OASIS ODF TC announced in a blog entry: “[n]o one supports ODF 1.0 today. All of the major vendors have moved on to ODF 1.1, and will be moving on to ODF 1.2 soon.”

Throughout 2007 Japan, who were translating ISO/IEC 26300 into Japanese, fed reports of defects to OASIS via an OASIS mailing list. A formal set of Defect Reports was submitted by the Japanese National Body in December 2007 and circulated to SC 34 members and liaisons (including OASIS). The JTC 1 Directives state that the Project Editor must respond to a Defect Report for a JTC 1 standard within two months. SC 34 received no response until August 2008, when it was informed by the OASIS ODF TC that a register of errata in the OASIS standard had been published.

OASIS have produced errata document which apply corrections for some of the defects that have been reported. Note however that OASIS cannot amend the text which is the basis of ISO/IEC 26300, as this text has only the status of “Committee Specification” within OASIS. Hence they propose amending the defective OASIS 1.0 (“1st Ed”) Standard, creating a new fork of the ODF specification. SC 34 are expected to cross-apply these fixes to their corresponding locations within ISO/IEC 26300.

It is unclear whether the reported defects which also apply to ODF 1.1 are to be applied in any way.

Communications from OASIS make it clear that OASIS believes it has entered into an agreement with JTC 1 which allows it to maintain ISO/IEC 26300 in a way which exempts it from the maintenance provisions of the JTC 1 Directives.

Problems

OASIS’s continually restated stated intention in its communications with JTC 1 is to prevent divergence of ODF versions. This goal has clearly not been realised, with a proliferation of versions of ODF inside OASIS and pronounced marketplace confusion.

For example, it should be of concern to JTC 1 members that the OpenOffice.org product is promoted as supporting “features of the upcoming version 1.2 of the ISO standard OpenDocument Format (ODF)”.

OASIS’s continually restated intention in its communications with JTC 1 is to maintain a collaborative relationship. However there has not always been evidence of collaboration. Input from the ISO/IEC members has not been sought. Where input has been provided, it has sometimes been met with delay and dismissiveness.

The agreement that JTC 1 has reached with OASIS appears to be being operated in a way which breaches the JTC 1 Directives. The relevant portions of the Directives are given below (all emphasis mine):

Maintenance for a transposed PAS is also negotiated in the Explanatory Report. JTC 1's intention for maintenance is to avoid any divergence between the current JTC 1 revision of a transposed PAS and the current revision of the original specification published by the PAS submitter. Therefore, the Explanatory Report should contain a description of how the submitting organisation will work cooperatively with JTC 1 on maintenance of the standard. While JTC 1 is responsible for maintenance of the standard, this does not mean that JTC 1 itself must perform the maintenance function. JTC 1 may negotiate with the submitter the option of maintenance handled by the submitter as long as there is provision for participation of JTC 1 experts, i.e. the submitter's group responsible for maintenance is designated as the JTC 1 maintenance group. (Directives, 14.4.2)
For the maintenance of an International Standard of whatever origin normal JTC 1 rules apply. Such rules distinguish between correction of defects and revisions of or amendments to existing Standards. Note: The JTC 1 rules for maintenance are found in clause 15 of the JTC 1 Directives. For the correction of defects, JTC 1 provides for the installation of an editing group. Active participation of the submitter in such an editing group is expected and strongly encouraged. Depending on the degree of openness of the PAS submitter, JTC 1 will determine its specific approach. (Directives, M6.1.5)

Therefore it is clear that while maintenance may (in the lax wording of the Directives) be “handled” by the submitter, it is not possible for the submitter to exempt themselves from normal JTC 1 rules, as “for the maintenance of an International Standard of whatever origin normal JTC 1 rules apply”. From this it follows that a submitter’s “handling” of maintenance is limited, and that the decision-making procedures and time periods specified by the JTC 1 Directives must apply.

Remedies?

Obviously this is all an enormous mess and while it is tempting to blame lawyerly over-cleverness on the part of OASIS, or insufficient alertness on the part of JTC 1, in negotiating their so-called maintenance agreement, the true culprit is, in my view, the JTC 1 Directives – such an impenetrable document has, evidently, led to a completely different understanding of the situation from the several parties involved. This procedural mishap is, I argue, further evidence of the need to scrap and re-write the JTC 1 Directives as a short, clear and professionally drafted document. Already this year we have seen that when tough questions get asked, the Directives are not fit for purpose; we are seeing the same thing again now.

The immediate problem faced is, however, the future of ODF in JTC 1. This is not a matter for SC 34, or for the ODF TC (both of which groups are full of excellent  technical experts wanting nothing more that to produce good standards) – this is something that must be resolved at a higher level between JTC 1 and OASIS. In the usual way of things, the developers are being hampered by the management.

The essence of the problem is that a central principle is being missed now: that only a standard that has a truly international dimension to its control should benefit from the ISO, IEC or JTC 1 “brand”. Some immediate remedies might include some mix of the following:

  • Since there seems to be general agreement that ISO/IEC 26300 is an obsolete version of ODF, perhaps it should be withdrawn as an IS – maybe in parallel with a PAS submission of ODF 1.1. That would at least give the world an IS that was widely used and a veil could be drawn over the 1.0 standardisation mess.
  • SC 34 has already stated it is open to suggestions how future maintenance should be arranged in a genuinely collaborative manner. Patrick Durusau (the ODF editor) has drafted a proposed agreement in that spirit. Also, OASIS might well have a thing or two to learn by looking at how Ecma has managed to enter into a collaborative arrangement for the maintenance of ISO/IEC 29500 within JTC 1.
  • The immediate defects in ISO/IEC 26300:2006 could be resolved by the formation of an editing group in SC 34. Indeed OASIS itself seemed to expect this in the explanatory report which accompanied their initial PAS submission which stated: “OASIS requests that any corrections of defects or errata from the JTC1 process be re-presented to the OASIS Technical Committee.” Per the Directives, OASIS TC members should be encouraged to participate in any such group.

Ultimately, it is for the nations participating in JTC 1 to decide how this matter can be resolved. The current situation sells-out the nations by allowing their brand (“international”) to be perpetuated in a process from which they are effectively excluded. This is “standardisation by corporation” through the back door. Whatever is decided, this must not go on.

Tags : , , ,

Comments (10) -

  • AlexH

    11/6/2008 10:33:22 PM |

    I'd noted this and commented to the developers previously; if you save an ODF file in OpenOffice.org 3.0 it puts office:version="1.2" attributes in the file.

    Clearly, with the RDF stuff, it's never going to validate as an earlier version, and older software doesn't seem to recognise the newer version so you lose data silently(!!).

    For my money, Sun are making the exact same mistake here that Microsoft made with Office 2007 (which doesn't write the ISO OOXML either), except that - I guess - Microsoft at least attempted to implement the ECMA spec. ODF 1.2 isn't even a committee spec. yet.

  • Alex

    11/6/2008 10:44:51 PM |

    @AlexH

    I completely agree - it's like OO.o is taking a leaf out of (wthat we think of as) Microsoft's play book here.

    However, this is a very dangerous and short-term game. Users (and this includes big players like governments and interntational organisations) could accumulate what they are being told is "ODF 1.2" only to find the spec gets revised in an OASIS or JTC 1 committee and they end up with proprietary non-standard OO.o content.

    The commercial wheels are turning here and I think it's pretty clear that MS's competitors see ODF 1.2 as a weapon to hit MS with, even if it means that weapon might get broken in the process.

    - Alex.

  • AlexH

    11/6/2008 10:56:49 PM |

    Right. The real danger here is that pre-emptive marketing of "ODF 1.2 applications" actually makes ODF 1.2 itself, when it's finished, both a. out-of-date and b. incompatible with the apps that tout it.

    I'm not totally sure why this is being conflated with the issue of "control". There problem here is one of maintenance, not development.

  • Alan Bell

    11/7/2008 10:23:54 PM |

    So what comes first the software or the standard? The standard would not have been written if the software to implement it, I know there are no reference implementations allowed, but it seems that if software is to implement a standard then the standard should lead. Maintenance of Internet standards generally follows the software. Feature innovation is done in an implementation and at some point if the feature is considered worthy it gets standardised and later all implementations add that feature. Think Framesets in HTML. In the real world I guess the standards are supposed to lead and the implementations follow. I am not quite sure that the people who want to make better standards are going to be able to lead the people who want to make better software.
    If withdrawing an IS is a remedy for a standard that has no compliant implementation then I would like to see that rule applied even handedly across all standards.

  • Alex

    11/7/2008 10:56:36 PM |

    @Alan hi

    I suppose in reality when developing a standard in tandem with an implementation, they tend to alternate for primacy as they move towards convergence in a final state. So when developing a DSDL implementation for a draft standard, things will often come to light in the implementation experience which feeds back into the standards text.

    Ultimately though, the standard text should be the final authority for a "standard".

    There is, though, no shame in claiming conformance to a "draft" standard (I glance at my wireless router). The problem with ODF 1.2 marketing is that the word "draft" is not prominent: look at the save options dialog in OO.o v3 -- it allows you to specify saving in "ODF 1.2" ("recommended"), for example.

    It doesn't matter for letters to the bank manager or shopping lists, but for serious deployments this sort of thing is really rather important.

    As for withdrawal, I don't think it's a question of there not being compliant implementations -- some very important (what I call) "aspirational standards" have not (yet) had full implementations but have nevertheless enjoyed influence and usefulness in partial implementations: SGML and HyTime spring to mind. But if everyone agrees that a standard is (a) not used and (b) causing confusion, then that is a classic candidate for withdrawal.

    - Alex.

  • orcmid

    11/8/2008 2:21:27 AM |

    Here's my personal perspective as a newcomer to the ODF TC bringing a strong concern for maintenance and up-down-level interoperability.  I'm referring to the OASIS errata document (OpenDocument-v1.0-errata-cd03 of 20 October 2008) that is now undergoing a public review that will end November 13.  See
    lists.oasis-open.org/.../msg00155.html for details.

    Procedurally the ODF TC cannot issue errata for OpenDocument-v1.0ed2-cd02 because it is not an OASIS Standard.  I sense no inclination to change that, partly because ODF 1.1 exists and mainly because focus is on conclusion of the ODF 1.2 work.  

    Nevertheless, the errata document now under review does identify exactly where corresponding changes can be applied to edition 2 and, because of textual identity, to IS 26300:2006 as well, were they adopted at JTC1 by some means.

    In reviewing and preparing the current Errata document, we were very careful to ensure that the items applied accurately to both documents even though we have no arrangement for approving the errata for anything but the OpenDocument-v1.0-os Oasis Standard specification.  

    If we've failed to line up, it is a mistake.  I know of one case where we could not navigate a reported defect from SC34 because the two specifications differed.  In that case, we deferred that defect item to future resolution.

    I know this is procedurally very awkward and we are all learning from this.  James Bryce Clark, OASIS Director of Standards Development, has notified SC34 that the errata document is now under review and it may be useful to SC34 even though it is officially an action solely concerning the original OASIS ODF 1.0 Specification and not IS 26300:2006.  See
    lists.oasis-open.org/.../msg00181.html

    With regard to your diagram, I would be more inclined to put a bubble for "+ 1.0 errata" document (though still a fork) below the original ODF 1.0 specification to which it applies, but vertically above the ODF 1.0 ed.2 document, the parallel IS 26300:2006 document, and the ODF 1.1 and 1.2 documents further down.  If I was being really fancy, I would direct the line out of "+ 1.0 errata" back toward the ODF 1.0 (1st ed) bubble and put dotted lines (maybe with "?") toward both the 2nd edition and IS 26300 bubbles.

    Looking ahead, we know there is another JTC1/SC34 defect report on IS 26300:2006 already.  The treatment of errata, if any, for ODF 1.1 also needs to be resolved.  (I have noticed that some of the errata items no longer apply in 1.1.)  We also need to ensure that the errata are reflected directly in ODF 1.2 wherever they remain applicable in our in-progress drafts.  Finally, it seems understood that there needs to be an account of any substantive changes to ODF 1.2 that impact existing features of earlier versions of ODF.

    I have no insight into how we close the gap with ISO/IEC JTC1 SC34 and IS 26300.  I suspect that the ODF TC hopes we can accomplish that with ODF 1.2 and remain synchronized from there on.  That's the obvious opportunity.  

    I'll not speculate whether synchronization at earlier OASIS versions is of material importance and how that might occur if found necessary by those with greater perspective than I possess.  

    My personal inclination is to automatically see prior-version maintenance as the responsible and transparent thing to do, but we're talking about people having to take on serious work that is a deflection from the drive toward ODF 1.2.  We also might need to be carefully selective about what issues merit errata and what ones are best left to resolution in later versions, not unlike the triage of bug reports for software.  It will take considerable discussion to work this out.

    One consolation/opportunity.  We are backed up on defect items on ODF 1.0.  OASIS procedures restrict us from issuing another errata for that OASIS Standard in less than another six months.  Meanwhile, as we develop dispositions, including identification of further errata items, we can probably do more toward providing a more-transparent process and visible demonstration of progress.  

    That interval is an useful period for also working out a systematic coordination mechanism with SC34, maybe even a roadmap for synchronization of OASIS and ISO/IEC editions of ODF specifications.  Such alignment will definitely happen above my "pay grade" as a volunteer on the TC.  

  • Paul E. Merrell, J.D. ("Marbux")

    11/8/2008 9:20:24 AM |

    Alan, a lack of implementations is certainly grounds for withdrawing an IS. JTC 1 Directives are an implementation of the governing Agreement on Technical Barriers to Trade. That treaty provides, in Article 2:

    2.3      Technical regulations shall not be maintained if the circumstances or objectives giving rise to their adoption no longer exist or if the changed circumstances or objectives can be addressed in a less trade-restrictive manner.

    www.wto.org/.../tbt_01_e.htm A lack of implementations would seem to be a qualifying changed circumstance.

    While that provision governs "technical regulations," a term of art under the treaty, the treaty also requires under section 2.4 that member nations use "international standards" or "relevant parts of them" as their technical regulations. It therefore seems that withdrawal of a technical regulation would require that the international standard be withdrawn before or simultaneously.

  • Rick Jelliffe

    11/8/2008 5:05:16 PM |

    There is also in important aspect here that is more general than ODF. It is the issue of whether "coarse sieve" technologies should be standards.  

    By a coarse sieve technology, I mean one in which conformance does not imply interoperability. RS-232 and the SCSI interface were also notorious examples of this. They defined parts of the picture, but not enough that you could guarantee that joining a plug and socket would allow connectivity even at the physical levels, due to different implementation of signals.  

    XML may indeed be another example of this, in that one XML system may not use another XML system's documents. However, this is not due to incompleness in the parsing rules. So it is just thin layering not "coarse sieve".  

    ODF 1.0 is clearly a coarse sieve. For example the formulas. But each version is making the sieve finer. And where there is graceful degradation and low user expectations, that is not so bad.  In the case of SCSI and RS-232C they got very bad names (among engineers, though user's expectations never had been raised so they tended to blame brands or the general idea of interoperability rather than the standards).

    One way out of this is to refuse to make coarse-sieve technologies standards: make them technical specs until they stabilize. The  horse has bolted though. OASIS management board needs to tell the ODF TC to fulfill their responsibilities: it must be embarrassing for them, after seeing how much opposition there was to ECMA "rubberstamping", to find themselves in fact far guiltier.

    I personally think that coarse sieve technologies are perfectly appropriate for subordinate technologies in a standard (such as SVRL is to Schematron) or where there is graceful degradatation (such as formatting hints.) Otherwise, there needs to be clear warnings in standards that conformance does not guarantee interoperability or substitutability.

  • Alex

    11/11/2008 3:23:24 AM |

    @orcmid

    Thank you for that fantastically detailed comment!

    Wrt your suggestions for the diagram, if I start moving the boxes around too much then it loses the sense of time (N.B. the grey arrow on the left). Oh well, maybe a different design would be optimal ...

    Overall, I agree the current situation is awkward. It is to be hoped with all the evidence of good will on both sides - at least among the technologists, who aren't grinding commercial axes or engaging in turf warfare - that something positive can be achieved. I think it is going to be very difficult to undo the mess we have accumulated to date, but it should be possible to adopt some more sensible procedures going forward. The noises I hear coming out of the JTC 1 Plenary in Nara give me hope that there will be consensus on a fettled maintenance regime that makes more sense.

    I very much agree there is a further opportunity for synchronisation with ODF 1.2. When (if) this is submitted to JTC 1 though OASIS will find a very different recipient to the one it found in 2005. Nations' attitudes have become hardened by recent events, and the coming into force of Directive M6.1.5 makes it clear beyond doubt that future PAS submissions need to be maintained under JTC 1 rules. Indeed there is a large body of thought that these large standards like ODF shouldn't really go through PAS anyway, but would be better (and maybe more quickly) processed through a JTC 1 subcommittee (SC).

Comments are closed