HTML5 and W3C Focus (10/2008)

HTML5,HTML,W3C,web standards

I sent the email below to the W3C members-only mailing list w3c-ac-forum@w3.org on 14 October 2008, as part of a discussion about HTML management. It might be a little dated, but maybe it will explain a bit where I'm coming from.

Note added 2/28/2010:

By “the expansion of W3C activities outside the core of the web” I did not mean style sheets, video, 2D or 3D graphics, HTTP, URLs and all the various URL schemes, web security, which are all at the core of the web. Web applications too! But of the W3C list of activities and publications, some are pretty far away from the "core of the web" as most know "the web", and even if relevant and visionary, but they're not on the trajectory of "the web" as it is being deployed.

"Leadership is the skill of getting others to follow": get out in front, but not so far in front they can't see you, and certainly not chasing after them with waggling fingers.

---

From: Larry Masinter
To: w3c-ac-forum@w3.org
Subject: HTML5 and W3C Focus
Date: Tue, 14 Oct 2008 23:47:15 -0700

To make some progress on the HTML5 situation, I think it's important to look at the general principles of standards-making, and not get too focused on the specifics of the individuals and their personalities. Certainly, just changing the players without addressing the process issues isn't constructive.

Good standards are hard and take time. Many people are frustrated with the standards process and would like to short-circuit the procedures, especially when the official process yields a specification which doesn't address their needs (as happened with XHTML). Many standards efforts do not succeed because the process of developing the specification ignored the needs of critical constituents in the effort to get "something" soon.

The web has many constituents, not just browser implementors. These include end users who depend on the web functioning smoothly and interoperably, but also authors, makers of authoring tools, dynamic web content generation tools, search engines, machine translation services, content management systems, advertisers, IT managers concerned with web security, publishers, government agencies wanting to publish information on the web that is universally accessible, publishers supporting multi-lingual information, gateways and translators to and from web content to print, etc., etc.

The design of HTML matters to each of those constituents, and changes that favor one category may adversely impact another; for example, the more expressive something is, the harder it is to analyze or translate.

Any single individual (editor or chair) may be committed to giving everyone a fair hearing, but it is impossible for any one person to comprehend all of the requirements and the impact that any change or addition might have to the web on all of those constituents. It is inevitable that they internalize the requirements of those they talk to the most, depending on their organization and place in it. With a complex, highly structured, and interdependent technology, it is unreasonable to expect that a small set of individuals who mainly represent a subset of one constituent ("browser implementors primarily concerned with new mobile platforms") to fully carry out the analysis and design work without substantial additional review.

Making changes or additions to HTML, and encouraging content creators to use additional features where universal acceptance is not likely is irresponsible, because it fragments the web, and encourages creation of content which only works for some people and not others. The chaos of the browser wars between Microsoft and Netscape was very damaging, and recovery is not yet complete; we should work to avoid a repeat.

Allowing a small specialized design committee to proceed without substantial oversight loses sight of all of the rest for whom the W3C is supposed to serve as guardian of the Web. Additionally, others who are not of the inner circle and who come to the process are at a disadvantage, because it's often difficult to articulate their requirements in a context that is understood by the design committee. Often proposed solutions don't entirely fit and are easy to ignore because the motivation is poorly understood.

Standards processes and standards organizations need staff and mechanisms to ensure that other voices from other constituents are heard clearly, that requirements are exposed and considered, that designs are not just reviewed after the fact, but that review comments are given sufficient discussion to expose the underlying requirements and to understand more clearly whether there is some way of meeting them, even if the offered solution isn't accepted.

In my experience, one of the most challenging tasks in standards-making was to push "proposed solutions" back to "requirements". Very often, someone would make a suggestion for some fix or patch or change they wanted to the specification. They were often unable (or, in some cases, for proprietary reasons, unwilling) to articulate the requirements or needs they were trying to satisfy by the proposed feature. Much of the difficult work was to distill these requirements and prioritize them, make them explicit, and explore alternative solutions.

Transparency in the standards process (e.g., a chart of suggestions ignored by the HTML5 editor) might help a little, but what's needed is a more thorough analysis of the requirements and evaluation of the specification against requirements in a neutral way. Doing so is a distinguishing characteristic which would differentiate an open and consensus driven standards organization from a consortia or a company producing proprietary solutions.

As W3C examines its budget priorities, I believe that the only proper response to both the W3C financial crisis (as well as the dissatisfaction with prominent ongoing activities such as HTML5) is to look harder at the organization's priorities. An effective W3C is one in which W3C can provide technical management. Technical management of a standard involves more than just making sure the meetings are held on time and that there are minutes. The W3C organization needs to provide leadership that reflects its mission statement: Leading the Web to Its Full Potential. The implicit assumption is that it is the full potential for all, not a small subset. The management team, staff, TAG and other important controls of the W3C process should be sufficiently focused not only on the technical aspects of the specifications being developed, but also on the impact the specification has on the broader range of constituencies, even when those constituencies are not directly engaged in the process. The structure of W3C process and technical oversight - director, TAG, staff - have intrinsic limits as to the scope of activities that can be effectively managed.

Many of the other core specifications which underlie the web (beyond HTML) are also in need of maintenance. In the IETF, the HTTP specification is being updated to correct errors and clarify ambiguities, yet this effort has received scant W3C attention. The URI specifications have several areas which need clarification and further standardization work (the "file:" URI comes to mind), yet there is no motivation or effort to fix the potholes - everyone is focused on building bridges, with the hope that some day they might lead somewhere.

I'm calling for more focus of W3C staff on the core specifications which are of critical interest to the broadest constituencies. To do this and stay within budget implies a drastic retrench of the current W3C activities, even at the expense of driving a number of activities outside of the W3C.

Over-specification is anti-competitive

HTML5

If there are 300 implementations of a specification, all different, but you take the 4 "important implementations" and write a specification that is precise enough to cover what those 4 "important implementations" do, exactly, precisely, and normatively require
("MUST") that behavior, then you inevitably wind up of making many of the remaining 296 implementations non-conforming, because the MUST requirements are too stringent.
The process then favors the 4 "important implementations" over the 296 other ones, and makes it harder for any of them to be offered as compliant implementations.
This is an example of "structural bias", as I wrote about earlier.

This problem is widespread in the HTML specification, and unfortunately really difficult to eliminate.
The example where I explored this in depth was in the calculation of "image.width" and "image.height", where a precise algorithm that required a state transition from:
[image not available, not loaded] to [image available, being loaded]
and then to EITHER [ image available, completely loaded] OR [ image not available, load failed].
HTML5 requires if the image is "available" (whether "being loaded"
or "completely loaded") that both image.width and image.height were both non-zero.
This behavior, I was assured, was necessary because there was some (how much? how often? still deployed?) javascript code that relied on exactly this state transition behavior.
Another implementation, which did not cache image width and height, or which let the cached image.width and height expire, and thus would allow [image available, being loaded] to transition to [image not available, not loaded], but that would be non-compliant with the HTML spec.
This non-compliance is not justified by significant interoperability considerations. It's hard to imagine any reasonable programmer making any such assumptions, and much more likely that the requirement is imaginary. By putting "compatibility" with a few, rare occurrences of badly written software which only works with a few browsers as the primary objective of HTML5, the result is an impenetrable mess.
The same can be said for most of the current HTML spec. It is overly precise, in a way that is anti-competitive, due to the process by which it was written; however, it is not in the business interests of the sponsors of the self-selected "WHATWG" steering committee to change the priorities.
Much was written about the cost of reverse engineering and how somehow this precise definition increased competition by giving other implementors precise guidelines for what to implement, but those arguments don't hold water. The cause of "reverse engineering" is and always has been the willingness of implementors to ignore specifications and add new, proprietary and novel extensions, or to modify behavior in a way that is "embrace and pollute" rather than "embrace and extend". This was the behavior during Browser Wars 1.0 and the behavior continues today.
None of the current implementations of HTML technology were written by first consulting the current specification (because the spec was written following the implementations rather than vice versa) so we have no assurance whatsoever that the current specification is useful for any implementation purpose other than proving that a competitive technology is "non-compliant."

Trip to Shikoku after IETF in Hiroshima

In front of hotel in Matsuyama.

After IETF meeting in Hiroshima, I took the weekend to do a little sightseeing. I stayed at a hotel with traditional Japanese rooms, in a town with an famous old hot springs, the Dogo Onsen.

Platform architecture: orthogonality of interfaces

Over at the TAG (Web archtecture)  blug I posted an essay that wasn't even discussed at the TAG meeting I'm at. Bad form? Presumption? Undemocratic? I suppose. Take a look.

Democracy, the oracle, and the W3C TAG

Athenian treasury at Delphi
Originally uploaded by Larry

Recently on a travel/study cruise focused on ancient Greece and the classics, we toured the Temple of Apollo at Delphi, from which the Delphic oracle spoke.

When does someone consult the oracle? I carry a pocket oracle: an iPhone "Magic 8-Ball" application which I consult at times, usually when I can't make up my mind, but don't want to waste any more time thinking about the decision.

Did the ancients also consulted the oracle when the course of action wasn't clear? Having weighed the pros and cons, the choices seem balanced. But often making some decision is imperative.

In a tyranny, the tyrant can decide. But if there is a group decision process and a need for agreement, then sometimes some external force is still needed to help the group come to a timely conclusion. Even if the ancients couldn't agree on a decision, they could agree to consult the oracle, and the oracle's pronouncements could then be used as justification for the ultimate decision.

The treasuries at the Delphi site impressed me; they were large constructs to hold treasures given as gifts to the gods who provided help in deciding important issues. The treasuries at the Delphi site clearly represent a community contribution, a kind of endorsement, sign, of the oracle as an authority.

The W3C TAG is an Architecture Group to which I was elected. The W3C TAG produces "findings" -- readings of general principles for the World Wide Web. Our pronouncements might be used as justification one way or another to encourage consensus, in those cases where otherwise reasonable participants cannot come to a conclusion on a course of action, but where a coming to a decision is important. Standards are often about making difficult choices. Let Sir Pythia speak, and we Delphic priests will interpret (alas, not in elegant hexameter).

Why saying 'HTML5 is not a standard' matters

HTML,W3C,standards

(Repost of comment on W3C blog)

Let's focus on the goal: insuring that everyone "on the web" can reliably communicate. HTML was intended to be the "common language" that every device, browser, computer could read and interpret, even though there are other languages and systems and formats and features. Increasing the capabilities of the common language to include "web applications" is an important subsidiary goal, as long as the original purpose of HTML isn't lost.

It is important to avoid the tragedy of the commons, where individuals and organizations acting independently in their own self-interest can ultimately destroy a shared limited resource. The "commons" is "global interoperability".

W3C provides a forum where organizations that might otherwise be in competition can get together and agree on a common technical goals -- one that they all agree to implement -- and to document that agreement in a way that anyone who wishes to can participate in interoperability. I don't think the quibbles about "Standard" vs "Recommendation" matter. It's critical, though, that whatever is documented and published (and promoted as a "standard") actually represents agreement.

I don't see how it is helpful if the status of the agreement of the participants is obscured. Calling something a "standard" (even if qualified as "under development") before there is agreement doesn't distinguish between "proposed but not agreed", "agreed but incomplete", or even "agreed, but document not finished". How does this help reach agreement?

Is shipping an implementation of a proposal before there is agreement helpful? Perhaps as a way of resolving disagreement by "fait accompli", but if different vendors ship different, incompatible versions of their own interpretation of the "draft standard", that would be counter-productive. If vendors ship implementations and call them "standard", but in the end the feature doesn't ever become standard -- doesn't this encourage users to create content that only works in some browsers and not others? It becomes another instance of "best viewed by".

I can't see how continuing independent tracks of web development (one for browsers and another for XHTML/XML/SVG-based workflows) can evolve into a single web useful for all.

Whether something is called a "Standard" or not doesn't matter as much as whether doing so helps or hinders where we need to go. Let's focus on that.

Structural Bias -- Standards and elsewhere

How Power Corrupts: Indirect Force

Consider the following:  if you take white noise -- sound at all frequencies -- and filter it, you get bias: if you filter out all the low frequencies, you'll get noise that sounds high-pitched. If you filter all the high frequencies, you get a low-pitched sound. If you filter everything below 339 and above 441, you will hear a 440 A tone, even though you started with white noise.

Something similar happens with software. There was a time when Microsoft was accused of introducing OS bugs that impacted other people's applications more than theirs! Of course, they hotly denied this. They hate bugs! They spend an enormous amount of energy on reducing bugs.

However, bugs in its software are "noise": fallout from adding features, working on tight schedules without much time to think.  Software vendors rely on testing. Testing is a kind of filtering for bugs.  And of course, filtering is, unfortunately, selective. It is somehow more important to eliminating bugs that affect you or your friends more than it is those that affect your competitors. The result is bias with no paper-trail: you did nothing wrong, every act is above-board, open, transparent, improving software, etc.

The same thing can happen in standards! People take positions, have opinions! There are lots of ideas generated, some good, some bad, suggestions, positions, what have you. Many contributors are independents, students, and many -- most really -- are sincere employees and software developers.

However: not everyone gets funded. Not everyone has a PR person assigned to report "this week". Not everyone is friended or employed or encouraged, or have the luxury of spending half- or full-time on their life's work.

Those with an agenda, implicit or not, apply filters: fund those whose work supports their agenda, not fund those whose work is counter. Individuals who are promoting a position which is in favor will be amplified, invited to meetings, recommended, and those whose position is not in favor will not. It's natural.  Bias is endemic. No individual needs to work according to an agenda to be a player.

Of course, I'm part of this game. I work for Adobe. I don't think I shill for Adobe, and most of my view of "the Web" was formed when I worked for Xerox, or later AT&T. But I can see Adobe's point of view, the sincerity of the people who work on its software, the lunacy of some of the silly anti-Flash attitudes not based on anything other than polemics that evaporate on examination.

Does this make me biased? Not any more than anyone else in the process.  If there's bias, you've already shown it by choosing to read this.