Chapter 4: Form and Function of
Document Processors
"Form ever follows function." -Louis Henry Sullivan
Document processors take a number of forms and perform a variety of functions.
What does the term document processing mean? After all, a document is something to be written and then read. Talking about "processing" a document can seem out of place.
When we use a computer to write or edit, we use a publishing tool such as a word processor or a page layout system to place our thoughts onto paper or display. The concept of processing is central to this task.
As we think about document processing we can view a document in two ways. First, a document functions as data, which are entered by the author and processed by the document processor. Second, the document functions as a piece of software, which directs the function of the document processor. Each document processing system tends to lean toward one approach or the other.
We now ask, when is a document data, and when is it software?
Documents function like software when they direct a document processor to perform according to procedures that are embedded in the document. Such documents are usually created with a simple text editor with a variety of special commands embedded in the text.
In contrast, documents that function more like data are created using a word processor or a WYSIWYG (What You See Is What You Get) editor. The user is not given the opportunity to enter data that will corrupt the document or stop the formatting abruptly. Thus, the user interface prevents most basic errors and bad data. Of course, this is an oversimplification of real life. Users can and will do all sorts of nasty things that can break documents; however, they must try hard.
Let's examine a few ways in which writing a document with a document processor is like writing software. The content (source code) of a document must be put into a form that can be processed (compiled). The actual printing (execution) or display of the document is the final step. The document processing program interprets the content and produces a printable form. All the various codes in a documentsometimes hidden, sometime visiblemust be syntactically correct, or the document will not process correctly. Rigorous syntactic checking is possible when documents are created using certain international standards such as SGML or HTML (see Chapter 5 Document Standards).
Another important similarity between document processing and writing software is in the area of debugging. Sometimes, when a document is printed, it looks crazy. It is not unusual to produce a document with very large errors, such as margins off by a couple of inches, illustrations that don't appear, and fonts of the wrong size. Often this is due to a "bug" in the document. For example, a table of contents might be generated by looking for all paragraphs tagged with the name SUBSECTION. If a particular subsection was not tagged correctly (i.e., with the tag SUBSECTION), then it would not appear in the table of contents. These types of problems can become insidious. Usually, they are not easy to spot and can be very difficult to track down. Wouldn't it be nice if document processing systems included debugging tools as part of the system?
The two major differences between document processing systems are how the user enters the information and how that information is interpreted. Therefore, writing with one system is different from writing with another system. The internal capabilities of the system will affect writing, design, and production. For example, many technicallyoriented publishing systems do not support a feature such as the automatic flowing of text around a graphic. (See Section 3 . 1 . 4 Page in Chapter 3 Points of View for an illustration of flowing text.) It is unwise to design a layout that needs this feature if the publishing system doesn't permit that type of text flow.
Let's turn now to a discussion of the various types of document processing systems.
4 . 1 Types of Document Processors
Build a system that even a fool can use, and only a fool will want to use it. George Bernard Shaw
There are many different types of document processors. A useful way to analyze them is to put them along a line that goes from simple text editing to WYSIWYG.
The path from text editors to WYSIWYG systems is not a path representing systems of increasing functionality. WYSIWYG systems do not have all the functionality of languageoriented systems; similarly, languageoriented systems do not have the functionality of structure editors. Specific products such as WordPerfect and MS Word blur the line between traditional word processors and WYSIWYG systems with new versions running under a Graphical User Interface (GUI), such as Windows 3.1 or the Macintosh.
4 . 1 . 1 WYSIWYG Features
As the name implies, the main characteristic of a WYSIWYG (What You See Is What You Get) system is not only that you can see the document before you print it, but also that you can edit it while you are looking at it. The display happens in real time, without any significant delay.
Most WYSIWYG systems use one of two approaches to editing specialized document types, such as tables, equations, and graphics. Some systems try to provide everything by having graphics and equation editors always available. Others automatically popup specialized editors when the document type is selected. From a user's perspective, either approach can be implemented smoothly. A smooth, consistent user interface is particularly important in a WYSIWYG system.
Another way to handle a variety of document types is to actually launch other applications that are external to the publishing system. For example, if a spreadsheet included in a document is selected for editing, the spreadsheet program might be started when the figure is selected. (See Section 3 . 4 The Engineered View in Chapter 3 Points of View for a discussion of these issues.) Web browsers use the concept of "helper" applications to accomplish the viewing of data types not explicitly understood by the Web browser.
Most WYSIWYG systems provide tools to associate visual styles to document elements. For example, a paragraph tag called CODE may be used to visually set off the computer source code, in a set of software documentation, with a particular font.
One of the more significant challenges for WYSIWYG systems is to create visual systems for global processes. The management of large numbers of files is not something you really want to do in a WYSIWYG form. When handling large volumes of documents, you certainly don't want to be forced into more input interaction than is absolutely necessary. Changing the layout of several thousand documents needs to be an automatic process.
WYSIWYG systems tend to be more closed than their languageoriented cousins. To have realtime WYSIWYG editing, publishing systems use their own proprietary formats for the sake of efficiency.
However, WYSIWYG systems are not necessarily closed. Some systems provide interfaces to programmatically get at the internal representations. More commonly, systems sometimes define and use published interchange formats (see Section 5 . 1 . 3 Lots `O Formats in Chapter 5 Document Standards) , which can be used as the basis of translators.
Now that we've seen WYSIWYG, let's turn our attention to the more complex languageoriented publishing systems.
4 . 1 . 2 Language Characteristics
Languageoriented document processing systems are much better than their WYSIWYG relatives at performing global or bulk actions. Manuals with thousands of pages of reference material do not necessarily have to be seen to be formatted. In fact, "What WYSIWYG advocates forget is: sometimes you don't want to see it at all."(1)
Some documents span many thousands of pages. Typical among these are technical documents such as maintenance manuals and software documentation. Two major classes of tools can handle these documents. One is the older (some would say more mature) markup, commanddriven, languageoriented document processors typified by programs such as troff and TeX. The other class are WYSIWYG publishing packages, typically running on workstations, exemplified by Arbortext's The Publisher, FrameMaker, and Interleaf.
Perhaps the best argument in favor of languageoriented document processing systems, according to Brian Kernighan, is that, "Once a task is well understood, it should be relegated to batch processing." (2) Nothing is quite so frustrating as being forced into repeated cumbersome interactions with a system to accomplish routine tasks. The ability to automate your routine tasks, which may, in fact, not be routine to anyone else, is critical. The specification of these tasks may take some time and be difficult. However, once they are specified, they are easily used again and again.
The features provided by a publishing system vary according to the scale of documents it was designed to handle. A system intended for simple reports will not be able to manage multiauthor documents with thousands of pages. Largescale systems must support more rigorous forms of change control and the ability to make global changes across entire sets of documents.
4 . 1 . 3 Specialized Languages
Various document types such as tables, equations, and graphs have their own special properties. Several specialized languages describe and create these document elements. In the Web world, a good example is the extensive set of markup tags to define tables that have already been developed and implemented.
Specialized document processing systems are a microcosm of the general case of document processing. For example, there are WYSIWYG flow charting systems and languageoriented flow charting systems. The same is true for equation processors and tables. (See Section 3 . 6 Specialized Views in Chapter 3 Points of View for a discussion and illustrations of these specialized processors.) These specialized languages, sometimes called "little languages,"(3) are used to perform very specific functions.(4)
Graphs, an important part of business and scientific publishing, deserve their own document processing language. The little language, grap, serves this need. Grap is a troff preprocessor language for the specification of graphs.
"graping around" - A sample grap specification and the resulting graph.
For the sake of completing the major UNIX troff preprocessor languages, eqn (to specify equations) and tbl (to specify tables) must be mentioned. Eqn and tbl are the "elder statesmen" of troff preprocessors and have been used for almost 20 years. They are robust, industrialstrength languages and are part of the standard UNIX document processing tools.
Bibliographies represent another document type for which specialized languages ease the processing. The management of bibliographies is an area where languageoriented systems are far superior to the WYSIWYG folks. TeX users have a set of macros called BibTeX, and troff users can use refer to manage references and bibliographies. The appearance of a reference in the text can be modified by the author. These packages can also work with relational databases or other filing mechanisms to store bibliographies used by an entire organization. WYSIWYG publishing systems are still playing catchup in this domain.
Many languageoriented document processing systems can be used with previewing software. This software lets you look at the document on the screen before printing, saving some trees in the process. Previewers represent a midway point on the line between languageoriented and WYSIWYG document processing systems, discussed earlier in this section. Previewing systems allow you to look at the printed form of the document on the screen, just as you do in a WYSIWYG system; however, you cannot modify the image, and the display may take a while. Ghostscript, the GNU project's free version of PostScript, will let you preview PostScript files and is available on many computer platforms including PCs.
Typically, you use a previewing system by observing approximate views of the page, making some edits, observing again, making more edits, observing the page, and so on, finally printing the page. Switching back and forth between the previewing mode and the editing mode can be timeconsuming. However, it is faster and more flexible than printing each page. TeX and troff previewers are widely available. Previewers are great aids, especially if the speed and availability of printers in an organization are poor. More importantly, previewers help to balance the often rigid complexities of languageoriented document processing systems with the adhoc nature of WYSIWYG systems. A relatively new system, Adobe's Acrobat (see Section 7 . 4 . 2 Applying Standards) can also be used this way.
4 . 1 . 4 WYSIWYG versus Languages
Each class of programs has its advantages and disadvantages. The features you need depend on what you are trying to do.
The WYSIWYG class of document processors is more appropriate for seat-of-the-pants design and for rapid changes with less rigidity. However, the languageoriented document processors have a significant advantage when it comes to global changes. This is a desirable characteristic if you are concerned with uniformity over many thousands of pages.
Languageoriented document processors are awkward to use and are anything but intuitive. The publishing department's staff will need a significant amount of time and experience to become familiar with the idiosyncrasies of the software. Writing a document with such a package is remarkably similar to programming and requires as much attention to detail. However, the payoff on the investment in time and training is significant. In particular, if your organization must repeatedly accomplish the same complicated tasks, you may be able to automate much of the process using languageoriented document processors. You can create style sheets with specialized commands for your own needs. These commands will not only be relatively simple to use but will also ensure conformance to your organizations requirements. The open nature of the languageoriented document processors is also extremely important for those exceptions to the rulewhich seem to happen every other day.
Numerous WYSIWYG software packages address the needs of the technical documentation ("techdoc") market.(5) Interleaf, FrameMaker, and Arbortext's The Publisher are three of the more significant ones, each with interesting characteristics.
Interleaf is oriented toward the dedicated publishing department. It has robust facilities for sharing documents and handling publication series with large collections of documents. If you use Interleaf on any number of computing platforms, the user interface will be virtually the same and familiar to the user.(6)
FrameMaker takes the good-neighbor approach to user interfacing. It is integrated with the normal working environment and window system's look-and-feel for any particular host machine. For example, if you use FrameMaker on a DECstation under a Motif look-and-feel window system, then FrameMaker behaves as a normal Motif application. On the Macintosh, FrameMaker looks and feels like a normal Macintosh application.
Arbortext's ADEPT*Publisher is a good example of a system that balances the closed nature of turnkey systems with the complexity of open systems. It provides a number of specialized WYSIWYG editors such as table and equations editors. Using the SGML version of the product, the entire document can be output as a validated SGML document. Validation is done in an interactive manner, not as an afterthefact process. In addition, ArborText offers a feature called the ADEPT Command Language (ACL) for languageoriented capabilities, such as the automated editing of large documents.
All these systems walk the tightrope between WYSIWYG and languages. Too much emphasis on the language side, and the user interface suffers; too little, and global automation is difficult.
Let's turn our attention now to a comparison of the functionality that WYSIWYG and languageoriented systems offer.
4 . 1 . 5 Comparative Functionality
How do document processing systems compare with respect to functionality? In particular, how does the WYSIWYG versus language orientation affect the functionality? As in Chapter 3: Points of View, it is useful to examine functionality in terms of the various points of view, such as design, communications, engineered, and specialized.
Most often, the primary function of any document processor is to format a document. The inherently visual nature of a WYSIWYG system allows for a more interactive, adhoc design of a document. More often than not, it also gets in the way of routine repetitive tasks. Languageoriented systems provide more opportunities for automation.
DESIGN
The fonts used by a system can profoundly affect the look and portability of the document. WYSIWYG systems have an inherently more difficult time dealing with fonts because they must have versions that can be displayed on the screen as well as printed. Adobe Type Manager solves this problem for the Adobe PostScript fonts, and TrueType from Microsoft and Apple uses the same font description both for printing and display.
The selection and adjustments of fonts and the overall typography of a document are easier with WYSIWYG systems than with languageoriented systems. The tedious trial and error of document editing, printing, and document editing, again and again and are very time consuming.
Many implementations of both major types of document processing systemslanguageoriented and WYSIWYGallow the creation and use of document templates. The terminology for templates varies. Sometimes they are called styles; other times, macros. However, they usually have the same function. A template is a particular document format that can be used over and over again. Some systems refine the concept of templates, categorizing them by types. Fonts, paragraphs, and page layouts can be categorized with individually named styles for particular use. These named styles can be used for many documents.
COMMUNICATIONS
When considering the communications point of view, WYSIWYG systems provide most of the tools. Spell checkers and a thesaurus usually require interaction. Languageoriented grammar checkers can also provide useful reports, which you can use later to help edit the document.
The ability to search the text and replace it with another piece of text is one of the more basic of all functions that an editing system should provide. However, search and replace functionality can become much more sophisticated once we remove the limitation of textonly search and replace. Some systems allow the searching of tags and styles. The searching itself can use flexible "wild cards" or regular expressions to match patterns of text. (See Section 3 . 1 . 7 Enterprise in Chapter 3 Points of View for an explanation of regular expressions.) The ability to search for an item, such as a cross reference, is also useful.
ENGINEERED
From an engineering functionality point of view, WYSIWYG and languageoriented systems provide similar capabilities. We are all familiar with the ability of even simple document processing systems to generate page numbers. Tools for large documents are often able to generate much more, including tables of contents, lists of figures, and lists of tables. The careful, consistent use of tags or styles enables these systems to determine the textual items to include in the various lists.
The ability to create running headers and footers, such as those used in this book, is a valuable feature for larger documents. They give the reader an instant context for the page being read, as well as an overall professional appearance. Document processing systems can automate this entire task by using particular tags or styles to identify the header and footer text. Languageoriented systems are much easier to automate.
A flexible autonumbering mechanism is a must for technical documentation. (How else could we reliably refer to Section 1.3A.4.2.9a?) A range of choices for numbering is important. For example, you may want some sections to be numbered in roman numerals (I,II,XI,VI,L), other with letters (a, b, c, aa, ab, ac), and still other with digits (1, 2, 2.1, 2.2, 3).
Another extremely important feature is index generation. The system should support several different indexing schemes. Critical among these is the ability to sort the index on a variety of criteria. In more extensive indexes, the ability to highlight the primary entry makes the index even more useful.
For example, the following index entries show how terms may appear under several headings.
In addition, one particular entry may be the major one. If so, it is often highlighted.
The ability to put cross references in the index is also useful. A good index is vital to any large reference document. The more flexible the document processing system is, the better.
From a functional point of view, index generation is virtually identical for WYSIWYG and languageoriented systems; you must always mark the item to appear in the index by hand. If, however, you do want to develop a semiautomated scheme, it would be easier with languageoriented systems because of their inherently more open nature.
Cross referencing is another area where larger document processing systems excel. Traditionally, cross references have been a prime source of errors. This problem seems quite natural; the section you referred to in one part of the document moves or is even eliminated over the course of editing. To ensure accurate cross references, document processing systems are almost essential. (Of course, nothing can replace a good copy editor.) WYSIWYG and the languageoriented systems are virtually the same here, although selecting a target reference is somewhat easier with the WYSIWYG system.
One final aspect of the engineering point of view is structural validation. Structure editors, which use standardsbased markup, can let you know if the document is structurally valid. They can tell you if a document has all the right pieces and if they are in the right order. In the case of Web documents that use HTML, there are HTML validation programs (and services) that can tell you if the structure of your document is correct.(7)
SPECIALIZED
Documents are rarely composed of text only. A good document processing system must be able to handle images, graphics, tables, and equations. The degree to which a system allows manipulation of these specialized items may prove significant for your particular application.
Most systems allow the exact position of the graphic or table to float. This means that the system will move the position of the itemsometimes to the next pageto avoid large areas of white space.
Some WYSIWYG publishing systems provide the usual sort of drawing manipulation tools. These include cutting and pasting, along with rotating, stretching, and scaling. The system may also support positioning functions such as alignment and distribution of many items. You should also look for the capability to manipulate images with respect to brightness, contrast, and other factors. These capabilities, however, are not easily translated to the languageoriented systems, so a strict comparison is not appropriate. If you are using a languageoriented system and need these functions, the best solution is to process the items in an auxiliary system and import them into the publishing system.
