Why is the Documentation so Bad?

Cynical comments about technical documentation are so common that an entire genre of jokes has arisen along these lines.  There's one about a guy in a helicopter in the fog over Puget Sound.  He flies around lost for a while.  Eventually he hovers near an office building where the workers begin to peer out the windows at him.  He holds up a sign: Where am I?  The occupants of the building respond by displaying another sign: You are in a helicopter.  The pilot's irritation abates with the recognition that he must be in Redmond over the Microsoft campus.  He sets his instruments and flies to a safe landing place.  When asked how he was able to navigate, he explains: I knew I must be over Redmond; where else than at Microsoft do you get information that is technically accurate but completely useless?

Back to the question we started with: why is the documentation so bad?  Microsoft bosses have been hiring the smartest people they can find anywhere in the world for twenty five years.  It doesn't seem likely that this is a simple case of Putt's Law and the competence inversion.  There are hundreds or thousands of managers at Microsoft of superior intellect.  You can tell by the way they drive--in the same mind-set as Bill Gate's famous gripe: "I'm surrounded by idiots."  Bill is a smart guy and a manager.  But that documentation is often hilariously obtuse.  There are detailed instructions about each menu item and how to open dialog boxes in which settings must be set and information entered, but no description of the options or their meaning.  Documentation for programmers is, too often, written by people who seem to have only the slightest familiarity with the technology they document.  I have it on good authority that trying to gain operational knowledge of programming can be hazardous to a writer's career. Why?

One explanation is that Microsoft doesn't want people to know how things work.  This theory had more credibility when the early documentation for Windows programmers was being written.  The cynical perspective then was: Windows isn't finished until Lotus doesn't run.  One way of eliminating competition for Microsoft Office was to make programming for the Windows operating system difficult for all competitors.  Smart managers hired people with English degrees to write documentation of COM interfaces and binary protocols.  The majority of the writers then were women who wouldn't punch software developers in the mouth, because trying to get information to do their jobs, writers are often exemplary in the frustration-aggression hypothesis of Konrad Lorenz.

Despite early success in making the game easier to win by elimination of other players, Microsoft does now have competition.  It seems that management would reward documentation personnel who produce lucid explanations of technologies invented, or stolen, by Microsoft.  This is to underestimate the subtle elixir of innovation.  Innovation in technology is often so intricately complex, so visionary in scope, that those whose reviews depend upon the timely delivery of innovations often don't know how their cutting edge features will perform in every permutation, or if they perform even in any basic scenario for which they are designed. Innovation might be a solution to an unknown problem. In any of these situations, smart managers and developers closely guard information, not only from competitors, but, even more importantly, from documentation personnel.  Writers can sometimes find out enough to make features comprehensible to customers who will then attempt to use the innovations, possibly before the product team can get past their performance reviews and move to other projects.

Finding an explanation for bad documentation starts to seem even more feasible when it dawns that management hires writers primarily on the conventional wisdom that a manager should never put anything in writing.  It follows from this that managers, and developers, should not review documentation, obviously because reviewing anything in print makes one accountable for the accuracy of information that could be in the documentation despite attempts to conceal everything of substance from writers.  This would also eliminate the recourse of blaming the documentation when customers can't make anything work.

For more fun, see The Devil is in the Documentation.