Masinter and Web Standards, Oh My

w3c,html,html5,masinter,adobe,hooey,conspiracy,standards,google,apple

There's been so much hooey flung around that it's hard to know where to start. This is a little haphazard, so I might reorganize some of the topics later.

Not me

Just in case you weren't watching, there was an enormous flap, started by a completely astounding blog post that somehow "Adobe is blocking HTML5!" and reference to some mystery, when nothing had ever been blocked. If you want to read about this and the nonsense around it, it's easy to find. Not sure how much of the muck-racking I need to repeat here. If you don't know what I'm talking about, let me know and I can point you to it.

Secret mailing lists

In the great tradition of conspiracy theories, it was claimed that there was a secret mailing list on which objections were being discussed? What secret mailing list was everyone talking about? Who subscribes to it? What is its charter? What topics are discussed on it? Who is the chair? How do decisions get made in it? (Post answers in blog comment! T-shirt to first to send correct answers.)

Transparency in Standards

Is it OK with everyone that the working group charter says one thing and the truth is something else? What I was asking for was transparency. I said they were not in scope for the current charter. And now we have an updated charter, at least. The W3C HTML WG charter still needs "annotation" to make it truthful clearer about communication (how many mailing lists and bugzilla databases do you need to watch to know what's going on?), participation (how many invited experts or members from a single company are there, in "good standing", even though they never actually participate?), and decision policy (there's a separate HTML WG only Decision Policy, with its own bugzilla database, in a Nomic twist.)

Updating the charter to say these things were in scope was always a way of bringing them into scope. That W3C wanted to save face and call it an "annotation" and skip the normal W3C rechartering process -- well, so many exceptions were made for HTML in the first place, that's fine.

What makes a "standard" process "open" isn't just "everyone can read the mailing list". Trying to follow the HTML standard requires withstanding a "denial of service" attack; thousands of emails, messages, posts, edits to track every single month. No one person can really follow what's going on..

Organizational scope and modularity

Well, this is a longer topic, and there are a lot of others who are blogging or discussing this, and I've blogged about it too. Short answer is: big committee, big spec, big mess. Splitting things up to minimize interactions between pieces is what distinguishes "architecture" from "hack". Anyone with serious experience building large systems should know this.

Masinter as Adobe's representative

I do work for Adobe. I get paid by them. And I do work on Standards now as my job. So you might think that, well, if I say something, Adobe is saying it?  I suppose I need to start putting more disclaimers on my email.

My work at Adobe has been mainly in the Advanced Technology Group working on a pretty wide variety of topics (content management, archiving, layout) and then in the Creative Solutions Business Unit (focus on workflow and metadata for video and timed media.) I kept up my standards affiliations because I thought it was important and I had some history, but really only started doing standards full-time in late 2008.

Other people and their affiliations

The idea that I am Adobe's Larry Masinter, but Hixie is Hixie and diveintomark (or just 'Mark' on some blogs) aren't somehow representing Google... or that Maciej isn't representing Apple? I guess I'm just jealous. Why do they get a free pass? I'm going back to using LMM@acm.org as my posting address, not to hide my affiliation but to make it clearer when I'm speaking for myself. Which leads me to try to explain why you should believe that.

My history in standards

I've worked on web standards but also the processes around developing them for long before I joined Adobe; I chaired the URI working group, and the HTTP working group for many years, and also participated significantly in the original development of HTML. Back in '93, I convinced everyone to use MIME in HTTP to allow extensibility (HTML wasn't the only document format then, and isn't now) and interoperability between the web and other Internet applications. I spent many years trying to bring stability, extensibility, architecture, internationalization, and security into the web platform. I not only worked on the web technically, but also worked on the standards processes. Chairing the URI working group and the HTTP working group while the browser vendors were trying to kill each other in the marketplace was a great career success for me: to (usually) get people to leave their partisan bickering outside the process. I was on the W3C Advisory Board and quite active in developing the W3C process document (including the little-known provision that the W3C Director does *not* have final authority; almost all Director decisions can be appealed, at least if people know about them.)

I had brought to IETF and W3C previous experience in standards work in ANSI X3J13. I had spent a lot of time in that standards effort developing the essentials of an "issue" as a resolution process (documented in a 1988 paper), and drove HTTP using it; as a process, it spread through much of IETF and W3C. Then, as now, the separation of understanding of the 'problem' from the proposed 'solution' is crucial.

I've worked on scores (more than 20 and probably more than 40) different working group charters. I think I really understand why working groups have charters, how the words in a charter are chosen carefully, and why it's important to keep things in scope.  Is that being nit-picky? Yes, but that's what I do. And, I claim, it's what makes good standards: follow process, pay attention to details.

My perspective is that what makes a good standard is very different than what makes a good implementation guide. Many otherwise good software engineers (even those with a great deal of experience) really don't see it, since their experience and intuitions have served them well in building complex software systems, or leading open source projects.

In the late 90s, I was on the tutorial circuit focusing on Web Standards, and put together several tutorials about the web, the web standards process, how to get involved, and what was and wasn't important about standards. These are linked from my home page, but I'll drag some of them out and blog about what I think is still true and what's changed in the intervening decade.

Tweets and Blogs Oh My

OK, so, there's still a lot of trash talk around the Internet. One poster said:

"Adobe's Larry Masinter should get out of the game. He did some great things in his career, but now he is just a well-paid Stanford-PhD-bearing obstructionist. Please retire and let the world move on. HTML5 is an amazing step forward as an open platform for the web, barring M$ and Adobe FUD-slinging."

Thanks for the "great things"! I try to be modest about my accomplishments, really! And I don't think credentials are worth much, although I do value knowledge and understanding, and I think there really is a "science" to "computer science" and not just a bunch of hacks. If I retired, though, I'd miss all this fun! Frankly, though, I don't think I was the one slinging any FUD at all, you know? Look at it.

More?

This is pretty haphazard, but it will do for now. I'll expand on things if you ask, honest. Comment here or by email; LMM@acm.org.

Users and Standards

In a comment an an earlier blog post, I was asked about users and the effect of different kinds of standards on users -- I had talked about the effect of standards on implementors but not about end users. The commenter made some claims about what end users wanted, with reference (presumably) to users of browsers.

This is hard, so I might take a couple of whacks at it, but here's one pass....

What Users Want


My main point is that what users want varies a lot, and that making a standard that was good for "users" involves making tradeoffs.

• Even a single user wants different and somewhat contradictory things. A single user wants both innovation and stability -- for new features, but also for old features to continue to work somehow. Just optimizing that alone involves a lot of tradeoffs.

• Second, there are different categories or roles of users. I'm a writer-user publishing this blog, and you (dear reader) are a reader-user. Although both roles may desire communication, sometimes the desires of the writer and the reader conflict. For example, writers often want readers to also see ads, while readers would just as soon skip the ads. To benefit both the needs and wants of both roles users requires some trade-offs.

• Third, even within a single role, there are variations. For example, among writer-users, some care about ads, some care about exact presentation, some care only about readers who have particular technology, or are running the latest software. Some care about users with accessibility needs and, unfortunately, others don't.

• Fourth, there are other applications, software, and work-flows those users might engage in (for example, copying web pages into email, reposting them in news feeds) and the user needs and wants may vary based on interoperability needs outside of the context of the user role in this individual action.

• Fifth, there are vendors who have different subsets of users who are important to them, based on their business model, and what other activities those vendors also support. As the business model varies (e.g., between selling advertising linked with uniquely provided services, closed hardware platforms combined with content monetization, or tooling and infrastructure), this influences which users, workflows, roles, and associated compatibility issues are important to the vendors who are implementing the standards. And vendors don't want impossible implementation goals in order to meet their user's needs. While you might not care about the desires of the vendors, motivating them is what causes standards to be implemented.
• Finally, it's important to think not only about needs now but also in the future. Do the standards allow sufficient evolution that new reader technology will work well with old content, while old readers continue to work with new content? Has the standard provided for a clean path for future extensibility?

What makes good standards for those users?

The standards, the specifications, and the discussions around them require a very difficult balance between all of these ultimately user-driven factors.

Given all of this, a standard for the web should try to meet all these needs, not just some. While some end users indeed want "a browser that work on any web site they pick", that would also mean supporting ActiveX, Java, QuickTime, and the five-letter F word.

Those who are designing and coding web sites have complex needs (some motivated by "money"), butthe link to technical requirements is not very simple.

A publisher can't depend on anything being broadly implemented just because some spec says a browser MUST do something. A MUST in a specification isn't a law; it provides no push. The only role a MUST in standard actually has is to provide a check-box for implementations; if the vendor of the implementation says "I implement standard X", they mean, among other things, "I follow every MUST in the spec, and I also follow every SHOULD except when I have a good reason not to, which I can explain". That's it. That's all the standard really does, is give you something to measure against.


A good standard is one where MUST and SHOULD are used as sparingly as possible; only those things that are really necessary to accomplish interoperability in the simple sense that the roles (readers and writers, intermediaries, editors, transformers) can function with expected results. Not necessarily the same results, but within the range of expectation.

HTML5 "experiment" and W3C Priorities

HTML5 W3C

in yet another astounding revelation, here's another post, again to the W3C secret Member-Only w3c-ac-forum cabal's list. Since this is my blog, I can repost all this historical stuff, right? Of course, this was a while ago, in the context of W3C budget discussions, and before there was even the (buggy!) HTML working group "Decision Policy".

---

From: Larry Masinter <masinter@adobe.com>
Date: Thu, 25 Sep 2008 15:18:29 -0700
To: "w3c-ac-forum@w3.org"

My personal observation is that the current HTML5 process combines the worst elements of the IETF process and the W3C process. From the IETF, there is the chaos of an open mailing list, wide ranging comments and free participation, but without the "adult supervision" that the IETF supplies in the form of the Internet Engineering Steering Group (IESG) and the area directors. From the W3C, there is the overhead and cost of W3C activity management and staff, and a process that assumes that voting members actually have a say in the final specification, but without any actual responsibility or response to normal W3C safeguards and cross-checks.

Recall the original history of HTML standardization: it started in the IETF, but succumbed there to an irresolvable debate between "inline font tags" vs "style sheets". The move to W3C was to provide a more structured environment, and strong enough management to resolve the ongoing positioning by Netscape and Microsoft.

At this point, the players have changed and there are more of them, but the jockeying for position is ongoing, but W3C seems to have abdicated any responsibility for ensuring that comments get answered, that questions are addressed on their technical merits, that reasonable voices are considered and points of view addressed, even when they differ from the point of view of the current editor of the document.

The W3C process document was built to safeguard the process. If there is an 'experiment' and the experiment is going awry, then the experiment shouldn't continue in its current form.

My basic premise for W3C prioritization is that the expansion of W3C activities outside the core of the web, in an attempt to gather more members, has basically been counter-productive, because activities have a lifetime beyond the interest of the few incremental members attracted. The basic direction I would suggest is to trim staff and expenses, and refocus the remaining staff on the core fundamentals of the web. Yes, some special interests may leave if their projects are no longer being developed by W3C, but the W3C process has a heavy overhead which was designed for a much narrower scope. Even if the peripheral activities seem like they're self-funding, every new topic stretches the ability of staff, TAG, Advisory Board, and ultimately the Director to understand and manage the process.

HTML5 and W3C Focus (10/2008)

HTML5,HTML,W3C,web standards

I sent the email below to the W3C members-only mailing list w3c-ac-forum@w3.org on 14 October 2008, as part of a discussion about HTML management. It might be a little dated, but maybe it will explain a bit where I'm coming from.

Note added 2/28/2010:

By “the expansion of W3C activities outside the core of the web” I did not mean style sheets, video, 2D or 3D graphics, HTTP, URLs and all the various URL schemes, web security, which are all at the core of the web. Web applications too! But of the W3C list of activities and publications, some are pretty far away from the "core of the web" as most know "the web", and even if relevant and visionary, but they're not on the trajectory of "the web" as it is being deployed.

"Leadership is the skill of getting others to follow": get out in front, but not so far in front they can't see you, and certainly not chasing after them with waggling fingers.

---

From: Larry Masinter
To: w3c-ac-forum@w3.org
Subject: HTML5 and W3C Focus
Date: Tue, 14 Oct 2008 23:47:15 -0700

To make some progress on the HTML5 situation, I think it's important to look at the general principles of standards-making, and not get too focused on the specifics of the individuals and their personalities. Certainly, just changing the players without addressing the process issues isn't constructive.

Good standards are hard and take time. Many people are frustrated with the standards process and would like to short-circuit the procedures, especially when the official process yields a specification which doesn't address their needs (as happened with XHTML). Many standards efforts do not succeed because the process of developing the specification ignored the needs of critical constituents in the effort to get "something" soon.

The web has many constituents, not just browser implementors. These include end users who depend on the web functioning smoothly and interoperably, but also authors, makers of authoring tools, dynamic web content generation tools, search engines, machine translation services, content management systems, advertisers, IT managers concerned with web security, publishers, government agencies wanting to publish information on the web that is universally accessible, publishers supporting multi-lingual information, gateways and translators to and from web content to print, etc., etc.

The design of HTML matters to each of those constituents, and changes that favor one category may adversely impact another; for example, the more expressive something is, the harder it is to analyze or translate.

Any single individual (editor or chair) may be committed to giving everyone a fair hearing, but it is impossible for any one person to comprehend all of the requirements and the impact that any change or addition might have to the web on all of those constituents. It is inevitable that they internalize the requirements of those they talk to the most, depending on their organization and place in it. With a complex, highly structured, and interdependent technology, it is unreasonable to expect that a small set of individuals who mainly represent a subset of one constituent ("browser implementors primarily concerned with new mobile platforms") to fully carry out the analysis and design work without substantial additional review.

Making changes or additions to HTML, and encouraging content creators to use additional features where universal acceptance is not likely is irresponsible, because it fragments the web, and encourages creation of content which only works for some people and not others. The chaos of the browser wars between Microsoft and Netscape was very damaging, and recovery is not yet complete; we should work to avoid a repeat.

Allowing a small specialized design committee to proceed without substantial oversight loses sight of all of the rest for whom the W3C is supposed to serve as guardian of the Web. Additionally, others who are not of the inner circle and who come to the process are at a disadvantage, because it's often difficult to articulate their requirements in a context that is understood by the design committee. Often proposed solutions don't entirely fit and are easy to ignore because the motivation is poorly understood.

Standards processes and standards organizations need staff and mechanisms to ensure that other voices from other constituents are heard clearly, that requirements are exposed and considered, that designs are not just reviewed after the fact, but that review comments are given sufficient discussion to expose the underlying requirements and to understand more clearly whether there is some way of meeting them, even if the offered solution isn't accepted.

In my experience, one of the most challenging tasks in standards-making was to push "proposed solutions" back to "requirements". Very often, someone would make a suggestion for some fix or patch or change they wanted to the specification. They were often unable (or, in some cases, for proprietary reasons, unwilling) to articulate the requirements or needs they were trying to satisfy by the proposed feature. Much of the difficult work was to distill these requirements and prioritize them, make them explicit, and explore alternative solutions.

Transparency in the standards process (e.g., a chart of suggestions ignored by the HTML5 editor) might help a little, but what's needed is a more thorough analysis of the requirements and evaluation of the specification against requirements in a neutral way. Doing so is a distinguishing characteristic which would differentiate an open and consensus driven standards organization from a consortia or a company producing proprietary solutions.

As W3C examines its budget priorities, I believe that the only proper response to both the W3C financial crisis (as well as the dissatisfaction with prominent ongoing activities such as HTML5) is to look harder at the organization's priorities. An effective W3C is one in which W3C can provide technical management. Technical management of a standard involves more than just making sure the meetings are held on time and that there are minutes. The W3C organization needs to provide leadership that reflects its mission statement: Leading the Web to Its Full Potential. The implicit assumption is that it is the full potential for all, not a small subset. The management team, staff, TAG and other important controls of the W3C process should be sufficiently focused not only on the technical aspects of the specifications being developed, but also on the impact the specification has on the broader range of constituencies, even when those constituencies are not directly engaged in the process. The structure of W3C process and technical oversight - director, TAG, staff - have intrinsic limits as to the scope of activities that can be effectively managed.

Many of the other core specifications which underlie the web (beyond HTML) are also in need of maintenance. In the IETF, the HTTP specification is being updated to correct errors and clarify ambiguities, yet this effort has received scant W3C attention. The URI specifications have several areas which need clarification and further standardization work (the "file:" URI comes to mind), yet there is no motivation or effort to fix the potholes - everyone is focused on building bridges, with the hope that some day they might lead somewhere.

I'm calling for more focus of W3C staff on the core specifications which are of critical interest to the broadest constituencies. To do this and stay within budget implies a drastic retrench of the current W3C activities, even at the expense of driving a number of activities outside of the W3C.