Book-Reader-Assist/library/The Psychology of Computer Programming/Chapter-13-Other-Programming-Tools/Chapter-13-source-text.md

The Psychology of Computer Programming — Chapter-13: Other Programming Tools

• Source: /library/The Psychology of Computer Programming/source-file.pdf
• PDF pages: 308–339
• Pages without text: none
• Note: Includes the Silver Anniversary comments that immediately follow the original chapter. Text comes from the scan's OCR layer and may contain recognition errors.

---

OTHER PROGRAMMING TOOLS 13 Ithough programming languages get the lion’s share of the at- tention in print, the working programmer has other tools of the trade. He spends much of his time reading and writing docu- ments, wrestling with the operating system, and wringing the bugs out of his programs. In these tasks, too, he could use whatever crumbs the social sciences might throw him. Yet these tasks are even more neglected from the psychological point of view than are program- ming languages. In the following sections we shall try to suggest how cer- tain insights could be used to better the design of tools for debugging, for management of operations above the program level, and for documen- tation. There are very few programming sources to call upon, but perhaps this effort will open up the subject to an era of new design efforts. We certainly could use them! 246

Other Programming Tools 247 PROGRAM TESTING TOOLS In September of 1962, a news item was released stating that an $18 million rocket had been destroyed in early flight because ‘‘a single hyphen was left out of an instruction tape.” The article did not say how large the “instruction tape’’ was, but we can imagine that it might have contained 100,000 or so instructions. In any other business, one error in 100,000 would be considered a phenomenal reliability record, but in programming, it cost $18 million. The cost may be exceptional, but the story is repeated daily. Indeed, the job of programming can be fruitfully looked at from the point of view of testing alone—considering that the only real problem in programming is getting the program to work correctly and proving it. It has been said that ‘‘the expert is a person who avoids the small errors as he sweeps on to the grand fallacy.” In programming, however, avoiding even the small errors is no mean task. In reality, there are no “small” errors, since even a “single hyphen” can result in disaster. The nature of programming being what it is, there is no relationship between the “size” of the error and the problems it causes. Thus, it is difficult to formulate any objective for program testing, short of ‘‘the elimination of all errors’’—an impossible job. Obviously, we need all the help we can get in testing programs, but the problem of program testing has received less than its share of atten- tion from designers of programming tools. Moreover, when testing tools are designed, the question of the psychology of testing is rarely evident. Yet testing is first and foremost a psychological problem. Consider, for instance, the question of confidence. The ideal testing tool should give us confidence in our program exactly proportional to the confidence it deserves, so that we neither pass on a program containing errors nor continue probing a program which is error-free. What program testing system today gives us even the minimum information in this regard? For example, to have confidence in the testing of a program, we should want to know to what extent our tests actually covered the coding. By this we mean, as a Start, that the testing system should keep and present to us a record of those areas of coding that have and have not been executed in the course of running tests. It is in the nature of people in our society to believe that things are as they want them to be, so a pro- gram testing tool must struggle to show us things as they are. Certainly, we could expect that the testing tools we use would continue to pester us until all pieces of code had been executed at least once. Since people tend to be optimistic about the state of their code, a good testing tool might be designed on the principle of destroying confidence— a confidence that can be restored when necessary. In the case of one

248 Programming Tools program—which calculated the prices of more than 30,000 manufactured objects—the customer for the program accidentally came across four cases in which the prices were wrong. His confidence in the program was destroyed, and even when the programmer was able to show him that the error occurred only when a certain dimension exceeded a certain size, he remained skeptical. As it happened, there was only one additional case in which this error existed, but having found four by himself rather quickly, the customer was not about to believe the programmer. Overconfidence by the programmer could be attacked by a system. that introduced random errors into the program under test. The location and nature of these errors would be recorded inside the system but concealed from the programmer. The rate at which he found and re- moved these known errors could be used to estimate—in some sense— the rate at which he is getting out unknown errors. A similar technique is used routinely by surveillance systems in which an operator is ex- pected to spend eight hours at a stretch looking at a radar screen for very rare events—such as the passing of an unidentified aircraft. Tests of performance showed that it was necessary to introduce some nonzero rate of occurrence of artificial events in order to keep the operator in a Satisfactory state of arousal. Moreover, since these events were under control of the system, it was able to estimate the current and overall performance of each operator. Although we cannot introduce program bugs which simulate real bugs as well as we can simulate real aircraft on a radar screen, such a technique could certainly be employed both to train and evaluate program- mers in program testing. Even if the errors had to be introduced manually by someone else in the project, it would seem worthwhile trying out such a ‘‘bebugging’”’ system. It would give the programmer greatly increased motivation, because he now would know that there were errors in his program, that he did not put them there, and that if he didn’t find them, other people would know about it. We have only to contrast this state of affairs with the currently prevalent one: he is not really convinced that there are errors there; he knows that if there are, they are ‘his’; and, if he doesn’t find them, he is pretty sure that nobody will know about it. The amount of confidence we ultimately have in the testing of a particu- lar program, however, does not depend on the testing procedure alone. The a priori probability of a program being correct has a great influence on the amount of testing that has to be done to reach a given level of confidence. To take the simplest possible case, consider the two code fragments: IF N = 1 THEN A= A + 1.5;

Other Programming Tools 249 +s a |

2 THENA=A

• 2.9; IF N= 3 THEN A A + 0.1; ’ IF N 20 THEN A = A — 3.2; and A= A+ B(N); Clearly, the second case is less subject to errors although we certainly cannot rule out errors in constructing the array, B. For example, suppose one card in the first case were mispunched, so that in the middle of the sequence somewhere we found IFNGSO1t) THEN “ASAE cai; IF N= 11 THENA=A
• 2.6; The cases for N = 11 and N = 12 would both be completely wrong, but these cases might not arise in the testing. In the second method, how- ever, each case is independent, that is, if we get that element of B correct, nothing else in B can influence it. Moreover, the second program handles the case of N fess than zero or greater than 20 by raising the SUBSCRIPTRANGE condition, whereas the first program simply ignores those cases, which could be a disaster. The psychological point of these examples is that we will have far more confidence in the second program than in the first, after an equal amount of testing. Why? Because the second program displays a uni- formity that the first does not. A single test on this program does the work of many tests on the other. Whether we get this uniformity from our programming language or from our particular conception of the pro- gram structure, the effect is the same: For a given amount of testing, we receive a greater amount of confidence in the program. Thus, the best possible testing tool must be based both on a language and a program structure with a high degree of uniformity. In a similar way, we can argue that lack of locality and lack of com- pactness make a program more difficult to test. But is there some way a testing system could measure the uniformity, locality, and compactness of a program? If there were, a scraggly, scattered, prodigal program could be sent back to the programmer or his supervisor for inspection after compilation and before testing is allowed to begin. We have no experimental evidence that such a testing aid could be built, but we can make several suggestions about directions for research on the problem. For one thing, locality could be measured by counting the number of labels, GO TO statements, and references to variables declared on a different page. More sophisticated measures of locality could be obtained by calculating some sort of ‘‘mean distance of refer- ence,” either for the whole program or for sections of it. Potential

250 Programming Tools trouble spots, such as partially open subroutines or ON-units which change variables, could be counted and warned about, in addition to the usual compiler warnings we get today. Uniformity and compactness might be a little more difficult to measure, but an aid in judging compactness would be some idea before we started coding how long a program would be reasonable for a particular job. One measure of uniformity and compactness could be the number of references per variable. A small number of references to a variable could indicate that it has been introduced unnecessarily and could be eliminated; and a large number of such references could mean that scattered operations could be consolidated through a different program structure. We could supply a tally of the number of times each array was referenced without using array expressions as a passable indicator of lack of uniformity, for an array is, in a sense, a collection of homo- geneous data items, and therefore a candidate for homogeneous treat- ment. Another approach to detecting lack of uniformity and compactness would be for the compiler to detect certain patterns in the code. As a simple example, consider the use of similar names such as N1, N2, N3, N4, and N5 in the same program. In the first place, as we know, such a collection of names is an invitation to error, for one name may be easily written or read for the other. But, if the use of such names really represents some uniformity of meaning, perhaps the variables they represent could be collected into an array and treated in a more uniform manner. A more subtle example of pattern detection would be the re- peated use of similar code sequences. Such repetition might indicate that more compression could be achieved by making a loop, a subroutine, Or a macro definition, or it might indicate nothing at all. In any case, the field is wide open for the investigator with a few ideas and a knowledge of programming psychology. Another well-known psychological bias in observation is the overde- pendence on “early’’ data returns. In program testing, the programmer who gets early ‘‘success’”’ with his program is likely to stop testing too soon. One way to guard against this mistake is to prepare the tests in advance of testing and, if possible in advance of coding. We refer here to tests concerned with detecting the presence of errors—not to all tests. Obviously, we cannot construct tests in advance for locating the source of an error; nor can we construct the procedures for correcting the error once it has been found. But all testing begins with detection, so advance work on test cases is never wasted—unless we yield to the temptation to bypass the remaining tests in view of the ‘excellent’ results we have So far. The debugging system could help us to resist this temptation by

Other Programming Tools 251 forcing us to specify in advance the amount of testing we plan to do. Failure to complete this number of tests could result in a management report, or some other form of prodding to the programmer. In general, of course, anything in the testing system that simplifies the preparation and execution of test cases will help the programmer to overcome the temptation to quit too soon. One such system in use today is particularly designed to counteract the temptation to skip retesting when a “small” change has been made to the program. If the test cases are stored in the system and can be rerun automatically on demand, the programmer is less likely to skip the retest. The typical system of this sort, however, _ produces vast amounts of output. It is hardly useful to rerun test cases if nobody looks at the results of the rerun. A great improvement could be wrought in these systems by providing automatic comparison between old results and new ones, thus calling the programmer’s attention only to those cases that differ from one run to the next. One final observation must be made about the possibilities for im- provement of the psychological behavior involved in program testing. Typically, as we discussed under the concept of ‘locality,’ when a programmer has a difficult time finding a bug, it is because he is looking in the wrong place. As simple as this sounds, that is how difficult it is to get the programmer to look in a different place, once he has “locked in’ on the wrong one. In general, the best advice for human debugging consultants is to make the client look somewhere else, or at least do something different with the same place. Possibly an automated debugging system could force the programmer’s attention to new areas of coding after he had been looking at the same area for a certain length of time. All sorts of algorithms could be devised for choosing the place to show him next, but the most important thing is just to get him unstuck. In fact, a good system is simply to turn off his terminal for a while, giving him a chance to look at some trees or grass or mini-skirts— anything but that wrong section of code. OPERATING SYSTEMS One of the persistent controversies in programming language design is this: What is the proper place for debugging tools—in the language or in the operating system? By raising the question in this section, we may seem to be taking sides in the argument, but no such conclusion should be drawn. Both sides have much going for them. On the one hand, the programmer should not be required to learn an entirely new language just to debug his programs—the horror of dumps makes that point clear. On the other hand, the language level conceals from the

252 Programming Tools programmer many of the practical realities of an actual operating en- vironment, and it is just these realities which may lead to the greatest problems in program testing. The proper compromise—at least on psychological grounds—seems to be a certain amount of extension to the basic programming language which faces up to the realities of the operating environment. This ex- tension, however, should not be added willy-nilly, as most operating system features seem to be, but should have a high degree of uniformity with the rest of the language. The requirement for uniformity, however, places a great burden on the operating system implementation, because the typical operating system supports not just one, but several, program- ming languages. Thus, it seems easier for the operating system people to supply a single, separate language of their own, a language which programmers in each of the supported languages must then master. In the case of OS/360, for example, this language was based on the assembly language for System/360—with some consequences we have already noted. To the operating system designers and implementers, this choice must have seemed entirely natural, for didn’t everyone know as- sembly language? But even for those people who did know assembly language, its format was probably not the most suitable for a job control language—at least as chosen by the designers. For example, the OS/360 job control language relies heavily on the use of so-called “positional parameters’? and much less heavily on the “keyword para- meters.” But positional parameters would seem to be most helpful when the user was very familiar with coding in the language and wanted the greatest possible compression. In the typical use of job controij, however, the user never gains much experience with the different options and is more likely to create his job control with the open manual at his side. In that case, the safe and sure identification of keyword parameters—no order to remember, no commas to indicate omitted items, mnemonic names with each keyword—would seem to have favored their use. Two of our students tested these intuitive feelings about the superiority of keyword parameters for coding in relatively unfamiliar situations. They used groups of experienced and inexperienced programmers, coding some nonprogramming material first in one form and then in the other. They found that positional coding was somewhat faster, even for new material and inexperienced people. But the rate of making errors was from two to four times as high. Considering the situation in job control —where little time is spent in the actual coding of the parameters and little assistance is given to help find bugs in them—three times as many errors seems too high a price to pay for a 30 percent increase in coding speed. Such matters were probably never considered in the choice of a

Other Programming Tools 253 job control format, for that seems such a small part of the problem of implementing an operating system. Another problem, which probably seems quite small to operating system designers, is the matter of ac- counting, or, in a generalized sense, performance evaluation. For many programmers, debugging is not the end of the line. The program may run, but it may cost too much to run. Operating system designers seem to be reluctant to ask for information on how long a program will run, how much core it occupied, how many records were processed on which files, how much CPU time was used, and so forth. Some cynics have even suggested that manufacturers do not want their customers to have such information, lest they see how little actual production is being done ~ by the “giant brain.” Regardless of how the manufacturers feel, performance evaluation in- formation is an essential part of the operating system’s task. For one thing, performance figures often give independent clues to program bugs. A program that runs too fast may be bypassing certain segments of code; and one that runs too slow may be looping in a place where no looping was expected. But more than this side benefit for debugging, there is the direct need of such performance information if programmers are ever going to learn to turn out programs that perform well. Without the simple and automatic feedback of such information, those pro- grammers who need it most will never see it. Moreover, good accounting information, sorted by programmer, aids the installation management in determining which programmers are in need of help and which pro- grammers are prepared to give it. It seems astonishing that we can have progressed as far as we have in computing without every com- puter user demanding that the manufacturer furnish him with some sort of minimal accounting scheme. Computer manufacturers sometimes argue that the customers have unique needs and would be offended if the manufacturer tried to tell them the kind of accounting information required. A more blatant rationaliza- tion would be hard to imagine, and it makes one think that the cynics might be right, after all. We could certainly identify a common set of requirements to serve as a Starter for most installations and provide them in a standard accounting routine for the operating system. Any customer who was dissatisfied with this routine could replace it with his own— certainly no more confusing than what he has now. In fact, with one manufacturer at least, since there was never any standard accounting routine supplied with the system, the interface for the “‘user-supplied”’ accounting routine had a way of changing with each new release of the system. After five or six releases, most customers had learned their lesson and had stopped trying to keep up with the changes. Of all the people in the business, the manufacturers seem to have the most prag-

254 Programming Tools matic grasp of psychological principles—at least when the principles can be used to avoid doing something they don’t wish to do. If we were to make a list of the information desired from an accounting routine, a measure of typical turnaround time in batch systems would be near the top. Of course, the operating system is not in a position to record the complete turnaround unless there is remote entry and exit of information under its supervision, but lacking the complete picture, the time between machine entry and exit would answer many questions. Yet, even if we had a complete grasp of the factors affecting turnaround, we would still have to face the problem—largely psychological—of what constitutes ideal turnaround. One of our students undertook to investigate the question of ideal turnaround. He polled both programmers and their managers, using a technique in which individual identities were concealed but group membership could be known. His first important finding was that different groups tended to have different perceived needs for turnaround, but that within a group the opinion was relatively stable. Presumably, different kinds of work require different kinds of service. A second finding was that in several groups, the manager and the group members had dia- metrically opposed ideas about the group’s needs. In one group a manager was pushing for remote access equipment ‘‘to improve turn- around” when the group members felt that what they needed—because they were debugging the input-output section of an operating system— was more opportunity to be in the machine room when their runs were made. Sometimes, managers are inclined to think that the programmers want “zero turnaround’—instant acceptance and return of their jobs. This survey and other observations give evidence to contradict this im- pression, for the programmers’ demands seem quite modest. In another study—one designed to compare the efficacy of two different program- ming languages—several of the participants complained that the turn- around was too good! They had been given special priority in order to facilitate the study, with the result that the measured average turnaround was 31 minutes from the time the job was passed through the window until the time the output was placed on the output table. The complaint was that if turnaround had been a little poorer, they would have spent a bit more time in making corrections and in perusing their output for additional errors. In a way, then, if the programmer has invested very little waiting time in a run, he may tend to value that run less—an observation which seems to hold for certain terminal systems as well. At the other extreme, very poor turnaround puts the programmer in an almost neurotic state when he submits a run. The cost of making the smallest error is so great—

Other Programming Tools 255 he might lose a week while waiting—that he may find himself making errors out of sheer nervousness. Moreover, he will try to put everything he can think of into the run, thus increasing his chance of making an error and also increasing the run time. It is common for a shop to drift into a poor turnaround situation with everyone making extra-long runs out of self-defense, and with these very extra-long runs causing the turn- around situation to deteriorate. Because the type of work differs from run to run, it is really impossible to state a single optimal figure for turnaround time. Usually, more than four, or possibly five, runs per day of the same job cannot be utilized fully, and fewer than two begins to induce anxiety and unsound practices. For initial work, however, when the programmer is just getting syntax and keying errors out of his code, terminal access would seem to be desirable. However, people in shops where four runs per day are assured are apparently not as troubled about these “‘trivial’’ errors as are programmers in less well-run shops. As we have noted before in other contexts, the mean time is not the only important psychological factor. If the turnaround is very uncertain —six runs one day and none the next—the programmer is not able to plan his work effectively. In a short time, programmers will react to uncertainty by setting up their test runs to defend themselves against the worst situation, with resultant waste and bad feeling. Another source of bad feeling with respect to turnaround is the doling out of priority. Programmers in one of our studies complained that certain (other) people seemingly got favored machine access for no apparent reason. No particular rancor was evident when priority was given out in a uniform way with respect to some particular job char- acteristic, such as expected running time; but when no reason was given, or when the reason was inconsistent, enmity seemed to be di- rected against both the grantor (the manager) and the grantee (the pro- grammer). Inasmuch as neither form of enmity would appear to be terribly healthy for a programming group, it might be best to dispense with such special favors altogether. When a priority system is established to favor certain types of jobs over others, there may be some initial resistance, or at least anxiety, but this usually diminishes as the effects of the new system become evident. The existence and publication of good operating statistics ap- parently help in gaining acceptance, for then each programmer can judge for himself the consequences of certain actions and adjust his runs accordingly. A typical situation is one in which lower priority is given to longer jobs, jobs that take more than a certain amount of core storage, or jobs that require special setups. At first, everyone seems to feel that his jobs should be made an exception to these rules; but if the

256 Programming Tools effect is actually better turnaround for those who fall within the limits, the nature of the jobs being submitted soon begins to change. Estimates of running time and core storage become better, and special setups that were once fervently defended as ‘essential’ now seem to disappear. Such a situation is just one more example of the “law of effect” as applied to the operating environment. Whatever the operating environ- ment—be it good or bad—the programmers will eventually adapt to it in ways to do the best they can for themselves. Thus, if a programmer is rewarded for behavior that is detrimental to the operation as a whole— or even if he is simply not penalized for it—all programmers will eventu- ally be adopting that behavior if it suits them. For example, some in- Stallations require that the programmers make time estimates of their jobs so that the operators can schedule the work better. If, to take one actual case, all jobs of fifteen minutes or less estimated time are treated in the same way, all programmers will eventually discover that it is best —for them—to put down an estimate of fifteen minutes even if they think they will need only one minute. In fact, there is no sense bothering to ask the programmer to make finer judgments than the operating system is prepared to act upon. The installation might simply ask the programmer to classify his job as “under two minutes,” “two to fifteen minutes,” or “over fifteen minutes,” possibly indicating at the same time the priority consequences of each choice. Similarly, if the system is multiprogramming and there are only two sizes of job partitions, the programmer should be asked to select one or the other—not give a space estimate down to the nearest byte. Setting rules that are not enforced, or setting limits that are not used, can only result in contempt for the system by those who must work under those rules and limits. The action of the law of effect can be blatantly obvious or surpassingly subtle. Moreover, it is not just the usual “system” that is involved. Programmers adapt to any aspect of the environment that affects the performance of their jobs, regardless of whether or not the system manager considers it part of their system. Sackman, for instance, ob- served in one of his studies that waiting time for jobs or terminals dropped substantially for subjects with more than four problems, which he inter- prets in the following way: “It seems as if the cadets became more efficient in their timing tactics for requesting computer service as they gained more experience.” Although Sackman calls this result ‘‘unexpected,’’ his surprise can probably be attributed to his being a psychologist and not a programmer. Any professional programmer has observed himself and others making such adaptations to the peculiarities of the system whose operation de- termines in large measure his success or failure. G. H. Stange, in a

Other Programming Tools 257 study for our psychology of programming class, studied the introduction of two remote entry systems which were intended to replace a dis- patcher system. He observed many adaptations to the dispatcher’s schedule and peculiarities, but the most unusual of these was the ob- servation that the turnaround time of a job depended on the way the cards were packaged! A deck wrapped with a rubberband was the best (50 percent of the jobs were back in less than three hours), a cardboard card box was next (32 percent were back), and a metal card box was worst (only 12 percent back in the same interval). Stange checked on a possible correlation between container and time or core requirements and found none. Evidentally, the smaller /ooking jobs got preferential treatment by the operators—and some of the programmers had begun to be aware of the difference. To compensate, they made every effort to make their decks look as small as possible; placing a small deck in a metal box would be a disaster. When a remote job entry system was introduced into this shop, such preferential treatment of decks was theoretically removed, but Stange goes on to show how other factors led to differential treatment to which the programmers could adapt. The first factor was knowledge of the new system, which led to a certain incentive to learn how to use it and to keep others from knowing. But then it began to be clear that untrained people using the system led to card jams and other malfunctions that only interfered with other people’s work. At this point the more knowl- edgeable people began to hang around and act as operators, so as to be Sure that their own jobs would not be delayed. Because of this pro- tective waiting, much of the apparent savings of the system was lost, because the programmers did nothing while waiting except make sure that nobody messed up the system. Ultimately the congregation of programmers around the terminal led to other effects—in particular, to exposure of programs to other people. The printer was extremely slow, so the assembled throng could watch programs coming off and make comments. Also, people conversed about the RJE system itself and about other aspects of programming. What this increased social contact would have done eventually is not known, because it was cut short by the emergence of one individual as un- Official ‘‘operator.”” He took the job to protect his own interests when the system would break down often—he reentered all jobs, but always man- aged to get his in first. This improved service for his jobs was apparently his only reward for operating the system, but others accepted this price in return for not having to wait in the entry room. And so, after a short perturbation, the system settled back to a pattern that looked very much like the old expediter pattern, with another human being interposed be- tween the programmers and the ‘‘system.” The job of getting one’s work

258 Programming Tools done became once again the job of manipulating a human being rather than the operating system itself. Moreover, the old expediter system was not removed, since the expediter served other programmers in the area as well. His service was used as a backup when the RJE system was behaving badly, as a way of running jobs with many cards or large printout without tying up RJE, and as a way of getting overnight service on jobs without the attention needed for RJE. The second RJE system was used by some programmers. It had a CRT device, and was found to be useful in initial preparation of decks, but particularly for monitoring the progress of jobs submitted by the other two methods. Overall, the installation of these two systems eventually had the de- sired effect—an increase in the number of jobs per programmer per day and a reduction in turnaround time. This improvement was not achieved simply by overpowering the problem with equipment, but by providing an enriched environment—an increased set of choices to which the programmers could adapt their strategy of submitting jobs. We do not know, of course, whether the increased flexibility available to this group was at the expense of some other users. Eventually, some rules and limits have to be established to maintain a balance among the various competing users. If sensible rules and limits are to be established, however, the operat- ing system must have sufficient flexibility to accept them and act upon them. Flexibility in the operating system is also desirable when setting up the procedures by which it is to be used. The best way to ensure that the system is used efficiently and effectively is to make the efficient and effective ways the easiest to use. In a well-designed operating sys- tem, the entire face of the system can be changed by making ‘‘cata- logued procedures” of job control language that is, or should be, frequently used. In this way, the programmer can be guided into better ways of using the machine by the reward of making the job easier. In view of the arguments about the specialized needs for accounting routines in each installation, it is interesting to observe how certain manufacturers tend—by deed if not always by word—to encourage customers to use their “standard” job control procedures, rather than custom building their own. The standard procedures, of course, must be sufficiently general to fit any configuration and any mix of words, so they are not usually optimal for any particular work or installation. Perhaps what we have here is an example of the law of effect operating on a grander scale. After all, computer manufacturers get paid more if more machine time is used. It is only natural, therefore that they should dis- courage the use of things (such as accounting routines), which might lead to greater efficiency, and encourage the use of things (overgeneral- ized job control procedures), which might lead to lessened efficiency. Caveat emptor!

Other Programming Tools 259 TIME SHARING VERSUS BATCH Suspicion of computer manufacturers is nowhere greater than in the reaction to the introduction of time-sharing systems. Old-timers are often heard muttering that time sharing is merely another scheme to introduce even more inefficiency into computing, so as to further line the pockets of the capitalists. Certainly time sharing, like other computer innovations, was undertaken on a large scale with no psychological investigation whatsoever. People thought it would work, or wanted to think it would work, so it was pushed onto the market and the battle began. By and large, the fight between the time sharers and the batchards has been conducted on the level of name-calling. Then, suddenly, we had a breath of experimental fresh air rushing through the argument, as Sackman published his book, Man-Computer Problem Solving: Ex- perimental Evaluation of Time-Sharing and Batch Processing. Those readers who still doubt the ability of psychological methods to clear up computing problems should read Sackman’s book from cover to cover, as should all advocates of either side in the time sharing versus batch controversy. We_ cannot hope here to survey the problem in the depth that Sack- man has obtained, but we cannot leave without some comments on the kinds of lessons which the time sharing versus batch controversy has given us, since this is the one area in which we have the most empirical information. One good starting place is the types of assumptions which creep into a field and may be taken for granted by even the most thorough investigator. In Sackman’s case, the idea of the individual as the proper unit of study is never questioned at all, which is hardly unexpected since that assump- tion is built into the very hardware and software of all time-sharing systems! We do not, for example, have terminals that are suitable for two or more people working together at them and, lacking the terminals, we completely lack the software or even software ideas to support them. Moreover, Sackman’s assumption is reinforced by the school environment in which most of his tests were conducted. Certainly at the Air Force Academy any cooperative work on a program would be regarded as cheating—heaven forbid! Another lesson from Sackman’s work is that even the most thorough investigator can occasionally make a wrong interpretation of his results. In one of his studies, Sackman plots mean number of syntax errors versus session number in the batch mode and finds “the classic error- extinction curve.” Although it is true that errors are “extinguished,” this is not at all the extinction which psychologists observe, for instance as a subject reduces the number of errors made on successive attempts to

260 Programming Tools memorize a list. If our studies have any relevance to Sackman’s group, the typical programmer working in batch goes through two phases. First he works toward his first error-free compilation, which might take 0-5 runs. Once he has that, he proceeds to change a single statement at a time, in the usual case, as he converges toward a semantically correct program. He may make an error in this single statement—especially if he is the ‘assertive’ type we postulated earlier—but the bulk of his statements are never touched and thus remain error-free through no effort of the programmer himself. It is the punched card which “ex- tinguishes” errors, not the programmer’s memory. Well, so what does it matter who does the extinguishing, as long as the errors do get extinguished? To the systems designer, it should matter a great deal. Let me give an example. A few years ago, | walked into a programming shop at the time they were firming up specifications for a syntax checker for a remote entry environment. The checker for PL/I! interested me, and | took a look at the specifications. Whoever had written them had asked that the checker be able to detect “any error that the batch compiler could detect” within the context of a single statement. The image here was of a programmer attempting to get a perfect compilation on his first try, which would be unlikely anyway because of global errors not detectable by a line-at-a-time syntax checker. To determine how valuable such a checker could be, one has to have an accurate model of the error patterns to be fed on it. Our model gives the following picture for PL/! programmers. Before the first successful compilation, the major cause of multiple compilations to eradicate syntax errors is the failure to close comments or strings, which causes large segments of code to escape syntax checking at all. But these are precisely the two errors not detectable by the batch com- piler, and they were going to be left out of the syntax checker! By simply checking those two things and nothing else, the average number of compilations to reach an error-free state could probably be reduced by one, and the variance reduced even more. Once the first successful compilation has been made, syntax errors are largely the most crude errors, such as those are caused by hasty punching. These errors are often global, in the sense that a name is mispunched, and are otherwise rather trivial, such as unmatched paren- theses and misspelled keywords. The most useful checker here would be one that finds crude errors and then makes a check for compatibility against the symbols saved from a previous compilation. If, of course, each new session results in major changes in the program, saving the symbol table will not be worthwhile, and only an explicit model of programmer behavior tells us how to design our system. If we believe, with Sackman, that each new batch run is essentially a fresh start in

Other Programming Tools 261 which syntax errors have been ‘‘extinguished” by learning by the program- mer, we will choose to make a syntax check that makes no reference to the previous compilation. But, if we believe that the programmer’s knowledge of syntax remains largely unchanged (although obviously not completely so) over the course of one problem, and that he “extinguishes’’ errors simply by eliminating them one statement at a time, then we will choose a rather different system design—one that will give greater pro- grammer satisfaction and even more efficient system performance. The question of system performance pervades Sackman’s work, and although he tries, as every good scientist should, to isolate the com- ponent that can account for a behavior, he consistently runs up against interaction effects, which make such separation impossible. For example, “language” is confused with “system” by the programmers, and dif- ferences in performance might be attributable to the use of ALGOL in the batch system and BASIC on-line. But even if the “same’’ language is used in both systems, they are not the same, since each system has its special control statements, which must differ from batch to on-line, as well as certain features that are essential in one environment and useless in the other. Another problem in making batch and on-line systems “comparable” is the question of terminal availability versus turnaround time. Is a six-hour turnaround with a variance of plus or minus three hours com- parable to an on-line system in which terminal users are limited to fifteen-minute shots on terminals that are available 90 percent of the time? Sackman is constantly running up against unsolvable questions such as this, and eventually he concludes that a bad terminal system is worse than a good batch system, whereas a good terminal system is better than a bad batch. Some may think this to be a trivial conclusion, but it represents a major step in the direction away from such vacuous statements as ‘on-line is better than batch,” or “batch is better than on-line.” Now we must continue Sackman’s work to identify those factors that make a batch system good, and those that make a terminal system good. Unfortunately, we probably won’t be able to separate “factors” as neatly as we would like, since our systems really are systems. Finally, it may not be worth the effort anyway, in view of the in- dividual variations found at every turn. Perhaps we should let Sackman summarize the entire case in his own words, and show us where the batch-versus-on-line controversy should now turn: Perhaps the central result from these studies is that no really substantive case can be made for the overwhelming general-purpose advantage of one mode over the other. As in all studies, without exception, individual differences in

262 Programming Tools performance overshadow computer system differences. The key to major break- through in improved man-computer effectiveness seems to lie in the nature and structure of individual differences and in human problem-solving styles. Whether the object of an institution is to teach computer science effectively or to pro- duce cost-effective computer programs, the path to effective achievement of system and institutional goals is fundamental knowledge of human problem- solving in man-computer communication. (p. 223) DOCUMENTATION Documentation is the castor oil of programming—managers think it is good for programmers, and programmers hate it! In fact, the managers know it must be good because programmers hate it so much, just the way we used to know that iodine was good for that abrasion on our knee because we could feel the sting. Well, folk medicine dies hard, in programming as at home, so it is going to be difficult to sell the idea that documentation, as such, has no intrinsic value. The value of docu- mentation is only to be realized if the documentation is well done. If it is poorly done, it will be worse than no documentation at all. As far as can be determined, there is no way to force programmers to produce good documentation. Some programmers can be forced to produce documentation, just as some children can be forced to take castor oil; but there are so many ways to sabotage the usefulness of documentation that forcing only results in documents that are not worth the paper used to print them on. The only hope, then, for producing good documentation is to convince the programmer that it will benefit him to do so. If the act of producing the document has no value to the programmer, the job will always be done in the minimum possible way; but if producing the document has demonstrable good effects, there will be no way of preventing reasonably good documentation. What do we mean by reasonably good documentation? Our emphasis is really on the word ‘‘reasonably,” for we usually err in expecting just too much from documentation. In the first place, different people have different needs when they look at the documentation for a program, so that no system of documentation is going to satisfy them all equally well. We should set a goal that is reasonable and try to achieve good documentation which meets that goal. If it happens to meet some other goal reasonably well, that is a bonus; but if not, then we shall have to produce some different documents for the different goal, or do without. Secondly, there are certain things that no documentation will ever do. Primary among these things is making it possible for an untrained person to understand all there is to know about a program in twenty-five words or less.

Other Programming Tools 263 Using the documentation for a program will always require some work and some minimum prerequisite knowledge. If the reader does not understand English, he is not going to understand a program document written in English, and nobody would blame the documenter for that. But it is equally unreasonable to expect that someone who does not understand FORTRAN can understand fully a program written in FOR- TRAN. He may be expected to understand how to use a program, if it is designed to be used by people, rather than to fit as one small cog in a larger system—if he understands the use for which the program was intended. We must not, however, expect that the use of a program for solving differential equations can be made understandable to a person who has not passed high-school algebra. What we can expect, as a minimum, is that if the reader has a high school education, he should be able to see from the first page of the documentation that he will not be able to understand it if he should get in any deeper. “Depth” is one of the most important documentation concepts, es- pecially for large systems. If a system is of any size at all, different users of the documentation will need different levels of detail in the information they extract. The highest level should be just sufficiently detailed to tell the user whether or not he will be able to read the documents. At each level—because we can never be sure which document will fall into the user’s hands—the first and most conspicuous thing in the document should be a reference to the documents above and below it in depth, along with an indication of the depth to which this document goes and the knowledge the reader must have to use it. Without this minimum of self-describing structure, no system of docu- mentation can be expected to succeed, but the user must also take the time to understand this structural information and to use it. We cannot expect that any random document we pick up will be the proper entry point to the path of understanding a system, any more than we can expect to understand the role of Aloysha if we start reading The Brothers Karamazov on page 353. In novels, of course, there is a standard struc- ture which the reader is assumed to know, and it simplifies the author's task no end to know that most serious readers are going to start at the beginning and read toward the end. In documenting a large system, however, the writers—in addition to suffering the deficiency of not being a Dostoevski or a James—have to contend with readers who don’t know where to begin. Thus, each physical document must contain some guiding information. Of course, it is a bit shortsighted of us to confine our thinking to phy- sical documentation such as books. Most likely, the problems of pro- gramming documentation will not really be solved until we start using our data-processing machines to document their own program in a creative

264 Programming Tools way. To be sure, we have started in this direction now—as we usually start toward automation—by mechanizing the same functions we used to do before mechanization. Thus, typesetting and composition are done with “administrative terminal systems,” with all the benefits of automatic assistance of updating—that oh-so-frequent problem in programming. But, we are still by and large limited in what we can communicate to the user of the document because we are still working through the medium of books. Now, it would not be smart for an author to decry too violently the con- tinued use of books, but there are limits to the proper functioning of this medium. In the first place, books are not the most flexible medium for accepting and disseminating updated material. From the time these words are impressed on the typing paper until they are seen in print by the first reader’s eyes, sufficient time will have elapsed in which to make a baby, starting with the basic ingredients. In some cases, at somewhat increased cost in money or accuracy or both, the time span can be reduced a bit, but the medium is not inherently fast enough for material which changes as fast as program documentation for a large system. In the second place, no matter how nice it is to cuddle up before the fire with a cozy book in your hand, few people use programming docu- mentation in that environment. For a general overview document—for teaching and learning general concepts of a system—a book might be the Suitable medium. But for answers to specific technical questions, some- thing else is needed. We cannot trust the user to find his way about in the maze of documentation which must of necessity exist for a complex system, so we must provide a system that guides him in an active manner. What could be better at this kind of active guidance than a computer, viewed through the window of a suitable terminal? Well, of course, there is the cost of it all. Who shall pay? Psychologi- cally, the question of cost of documentation always strikes a hard blow, for we are always unprepared. Why? Because we have not accustomed ourselves to thinking of the cost of documentation as a part of the pro- gramming cost. At bottom, then, when you talk in terms of what managers talk in terms of—money—you find that they, too, think of documentation as a separate, or separable, part of programming. It’s a wonderful thing to have good documentation, but not if it has to be paid for. From time to time, schemes are brought forth for some form of ‘‘auto- matic documentation’’ of programs—schemes which are supposed to be painless or costless or both. One example of this type is the many flow Charting programs which exist. If you feed one of these programs your program input, it spews out a flow diagram as output. Certainly the proc- ess is—or can be—painless, and, in fact, turns out not to be very expen-

Other Programming Tools 265 sive. But is it documentation? Probably not. What it may be—and this is nothing to sneeze at—is one more weapon for the debugging arsenal. The programmer, if he has been working without flow diagrams of his own, may find it difficult to follow just what his program is doing, par- ticularly if its locality is not very good. A flow diagram, properly con- structed, can help to gather many nonlocal points of flow into a two- dimensional form so that they may be seen on one page. This new view of the program may be extremely helpful to the programmer, not only revealing such solid errors as endless loops or unconnected branches, but showing whether or not his code has realized his intentions. If the programmer has been working with a flow diagram all along, the machine-drawn chart can be used for comparison, which will certainly reveal any errors in transcription from the diagram to the code. In more linear languages, such as PL/I, the two-dimensional advantages of the flow diagram lose their strength. When a PL/I program without any labels was put into one of these flow charting programs, what came out was the Original program, in the original order, with nice little boxes drawn around different pieces of code. What the flow diagrammers’cannot do is add any- thing new inside the boxes, although seeing the code with boxes around it may help the programmer see certain things he had failed to notice. However, if the contents of the boxes are obscure to all but the original programmer, the flow diagram drawn automatically from them—in any language—is not going to be much help to the person who needs docu- mentation. If the comments were misleading, they will still be misleading —unless they mislead about the destination of a branch. If symbols were poorly chosen, they will remain the same symbols on the flow diagram. We have made a few pilot studies to test the documentation value of such automatically drawn flow diagrams, and we find no evidence that the original coding plus flow diagram is any easier to understand than the Original coding itself—except by the original coder. Perhaps some of the companies that sell such flow charting programs would sponsor more carefully controlled studies into this problem. A flow diagram, of course, is but one alternative way of representing the information that is in the actual code. Even if its total value turns out to be quite small, there are several reasons not to give up on it as a docu- mentation technique. First of all, since it is an automatic procedure carried out by computer, we can expect that the cost of doing it will decrease, rather than increase, as computers get more powerful. Sec- ondly, it has the absolutely outstanding advantage of being drawn directly from the code itself, so that maintaining the documentation is precisely the same act as maintaining the code. Therefore, there is no possibility of slippage—the death rattle of all documentations systems.

266 Programming Tools One extreme example of the view that the documentation should spring from the code itself is expressed by Klerer and May about their system. Not only can the written documentation for their system be placed on two sides of a sheet of paper, but they expressly state that they do not expect the programmer to learn how to use their system from the documenta- tion but, rather, from experiments with the system itself. Instead of load- ing up the documentation with thousands of words to explain an obscure point that nine-tenths of the programmers will never use, Klerer and May leave the work of clearing up obscurities to the machine itself. This is, after all, only a return to the normal practice which we have been using for years. The machine is, and always will be, the court of last resort on documentation disputes. In the past, however, the machine has been a rather uncommunicative documenter. In order to rely more on the machine itself, we must use more imagination in getting the machine to tell its story in as many alternative ways as possible. Each particular way, such as flow diagram- ming, may give but a small addition to the clarity of a program, but many small changes in viewpoint may add up to a sum much greater than its parts. If, as we believe, the main impediment to program debugging and program understanding is psychological set, any technique for getting the mind unstuck should be explored as a possible tool of debugging and documentation. We have been conducting a series of studies on the usefulness of various alternative ways of looking at a program. We have tried to draw on any source of inspiration in coming up with alternative views. For instance, think of the painter, who finds that familiarity with objects may cause him to distort his perception of their actual color. To overcome his set, he bends over and views the scene upside down through his legs. In a similar manner, we are experimenting with printing programs back- wards—either by character or by symbol—just to see what will happen. Some of the other methods we are trying are more familiar, but a list of them may help inspire the reader to think of others:

1. Set off keywords in boldface or underline.
2. Set off nonkeywords in boldface or underline.
3. Set off keywords or nonkeywords in lower case.
4. Change multi-use symbols, such as the ambiguous equals sign, and parentheses at different levels of nesting.
5. Mark potential trouble spots—such as dummies created where refer- ences might be desired or references used where dummies might be wanted.
6. Strip comments from a listing and present the ‘‘naked” listing on one page and the comments on another.
7. Strip literals in a similar way.

Other Programming Tools 267 8. Rename all variables. 9. Expand all abbreviations, or abbreviate all expansions. 10. Permute options to a standard order, or simply to a different order than the programmer used. 11. List symbols in alphabetical order, together with references to where used, set, or passed as references. 12. List symbols in groups according to specific attributes, such as all EXTERNAL variables or all FLOAT variables together, with multiple listing according to multiple attributes. 13. List the scope of symbols. 14. List potential ambiguities. 15. Extract and print separately groups at successive levels of nesting of DOs, BEGINs, and PROCEDUREs. 16. Decompose conditional statements into some canonical form, per- haps including decision tables. 17. Strip out expressions and list them with their values calculated under several sets of values of their contained variables. 18. Execute sample flow paths. None of these viewpoints are intended to change the physical meaning of a program, but all are intended to change its psychological meaning. What we are hoping to do is present the programmer’s eyes (and perhaps other sense organs) and mind with a richness of views on which he can apply his native talents as an adaptive system. In other words, we are not trying to make the machine solve the debugging problem and the documentation problem; we are trying to make the machine help people take advantage of the immense psychological resources they have in overcoming their immense psychological shortcomings. It would be nice, of course, if these programs did solve, or partially solve, the documentation problem. Another worthy effort in this direction involves a bit more work by the programmer, but also promises to be useful. In this system, when the programmer has finished the program, he sits down at a terminal and is interrogated by the computer about the function of each of the various program and data elements in the coding. The replies he makes are stored away and presumably shipped to any- body who requests a copy of the program. The users, in turn, can go to a terminal and ask questions about the program and data elements that interest them—the computer keeping from them the job of sorting through the mass of documentation material. There is no chance of their going to the wrong place for the information, for they always go to the same place—the terminal. This type of system really seems to get at the problem of the user who does not understand the structure of the documentation, but it still leaves the problem of entering the documentation data originally. The inter-

268 Programming Tools course with the computer will certainly be a motivating factor for some programmers; but even the most motivated programmer, if he cannot write an English sentence, will not be transformed into an able com- municator by this system. One possible modification would be to use the computer merely as a referee in a dialogue between the programmer and a number of ‘‘typical’’ users. The users would ask questions through their terminals, and the programmer would attempt to answer them through his. The computer might even help the users by prompting them to ask certain questions, but would not record the programmer's answers until they had been accepted by the users as giving them the information they desire. The entire set of answers could then be classified and shipped with the program for other users to interrogate. Even better, of course, though conceptually further in the future, would be a single central pool of answers to such questions. Any user, or potential user, of the system could interrogate the central pool. If he obtained the information he needed under the computer’s guidance, that would be that—although statistics could be kept on frequent requests so that they might be answered more directly in the future. When, however, the user could not satisfy his information needs from the answers already in the pool, the computer would issue a request to someone at the programming facility to get on the line with the user. In the ensuing dialogue, the question would eventually be answered, but it might take a few days in the worst cases. Once the answer had been obtained, however, it would be added to the records and thus made available for future users. What such a system does—with suitable provision for updating as changes are introduced—is to take us back closer to the time when the documentation was the programmer himself. In a small shop, on a small program, little attention was paid to explicit documentation, for it was always easier to go to the originator to get the answer. After all, you probably had to ask him in order to find the documentation. In fact, in such a situation today, this is probably the best method. Any programmer knows a lot more about his program than he will ever be able to put in his documentation, or than anyone would be able to find as needed even if it had been put in. The major problem with this method is when a programmer leaves the shop, but if egoless programming is practiced— and if the management is not so bad that all the programmers leave at once—this problem is minimized. There are, of course, limits to this personalized method. One par- ticularly strange situation occurred in 1961 concerning a network analysis program | had written in 1956 while working in San Francisco. The pro- gram had been distributed informally and used widely during those years, but | had in the meantime wandered from place to place, finally resting

Other Programming Tools 269 for the moment in New York. Then, one day, | got a long-distance call from Stockholm. At the other end of the line was a desperate programmer who had been tracking me down for three days. He explained that he was trying to modify the program to handle a certain case which was not in the original conception of the problem, and then asked, “‘If | change the sign of location 1003 to minus, will that do the job?” Over five years, and across the continent, | had hardly retained the memory of having written the program, let alone a memory of what was in location 1003. Quickly calculating the cost of a transtlantic telephone call and comparing it with the cost of computer time, | told the caller, “Yes, that should do it. Why don’t you put it on the machine and try it out. If it doesn’t work, call me back.’’ Fortunately, | never heard from Sweden again. Even when programmers remain fixed, their memories fade. Very few programmers can recall what was in any location—symbolic or absolute —of any program they wrote five years ago and have not looked at since. But then, very few programs of one-man type can be expected to remain in use over such a long time span. It reaches a point, even with the best documentation, that it pays to throw a program away and start over. What we must consider, then, when deciding how to document a pro- gram, is who will use it, how it will be used, where the users will be, and for how long it will stay in use. If it will only be used locally, for the type of work it was designed to handle, by people who know the programmers, and for a year or two or three, then perhaps the money that might have been spent on elaborate documentation could be put to better use on some other project. If many people in remote locations are going to use it, the cost of elaborate documentation must be reckoned with from the first, and the programmer will have to be involved from the beginning in making documentation a possibility. Notice that we did not say ‘in making documentation” but, rather, ‘‘in making documentation a possibility.” When a program is going to be used as a marketable commodity—whether or not it is actually sold— the users have a right to expect that the documentation be done as pro- fessionally as the programming. And, considering the matter coolly, there is no reason to believe that a professional programmer will be qualified as a professional documenter. Thus, if professional documentation is to be done, it should be done with the assistance of the programmer, but not by the programmer. If documentation is elevated to a professional status all its own, so that the documenter can work side by side with the pro- grammer without being made to feel inferior, we have the right to hope that documentation will improve. We have come through a strange cycle in programming, starting with the creation of programming itself as a human activity. Executives with

270 Programming Tools the tiniest smattering of knowledge assume that anyone can write a pro- gram, and only now are programmers beginning to win their battle for recognition as true professionals. Not just anyone, with any background, or any training, can do a fine job of programming. Programmers know this, but then why is it that they think that anyone picked off the street can do documentation? One has only to spend an hour looking at papers written by graduate students to realize the extent to which the ability to communicate is not universally held. And so, when we speak about com- puter program documentation, we are not speaking about the psychology of computer programming at all—except insofar as programmers have the illusion that anyone can do a good job of documentation, provided he is not smart enough to be a programmer. SUMMARY The main lesson of this chapter is simple: Systems are complex. A computer system is not just hardware, not just software, not even just people plus hardware plus software. The procedures, formal and in- formal, that have evolved with the system are part of the system; so is the current load on various components, and so is the attitude and ex- perience of the users. Even among the commonly accepted “parts” of a system, clear lines of separation do not exist. Hardware merges with operating system, operating system merges with programming language, programming language merges with debugging tools, debugging tools merge with documentation, and documentation merges with training, and all of them mingle with the social climate in which the system is used. This complexity means, on the one hand, that experimental evidence is doubly essential, but, on the other hand, that it is doubly difficult to obtain and interpret. The time has come to sweep the myths out of our closet and replace them, if possible, with hard experimental evidence. Not that any of these problems will ever be “solved”—the complexity is too great for that. But what worthwhile problems are ever solved? It’s not so much the solutions we need, anyway; but the experience in try- ing to get them. QUESTIONS For Managers

1. Have you ever been offered a ‘‘debugging compiler’ or other debugg- ing aid? How did you decide whether or not to purchase it?
2. Do you ask your programmers to estimate their level of confidence in

Other Programming Tools 271 their “debugged” programs? Do you try to make your own estimate of confidence? What factors do you use in making this judgment? . Describe the systems of performance evaluation in use in your shop. How do you use them in being a manager? What additional informa- tion would you like to have them supply? What information do they now supply that is useless to you? . What do you consider to be ideal turnaround in your batch system? What do you consider to be ideal access to terminals in your on-line system? What do your programmers think about these issues? . Do you give certain programmers priority in using the batch system? The on-line system? Have you ever observed adverse effects of your favoritism? What do you think you could do in the future to give priorities without creating bad feeling? It is possible that the bad feelings created do more harm than the priorities are supposed to do good? | 6. Which is better for programming, on-line or batch? . How are the documenters chosen in your shop? Are your documenta- tion rules sufficiently flexible to permit different kinds of documenta- tion for different types of programs? Is there resentment against docu- mentation work? If so, what do you think is the source, and what can be done about it? For Programmers

1. Recall from your experience a “tiny” error that had a big ultimate cost. What debugging tools or techniques could have prevented that error? Would their cost have been greater than the cost of that one error? . Give examples of when you have been biased in your program testing by “early returns,’’ both good and bad. Give examples of other factors that have biased your debugging strategy, such as desire to piease your manager, knowledge of program structure, or interaction with other programs or programmers. . What do you consider to be ideal turnaround in your batch system? What do you consider to be ideal access to terminals in your on-line system? What does your manager think about these issues? . Give an example of behavioral adaptation to system features, using yourself as the subject. . Which is better for programming, on-line or batch? . What other views of a program would you like to have supplied by

• your operating system and compiler? Explain how these views could help you, giving specific examples for each. . What do you think of documentation work? What do you think of

272 Programming Tools people who do a good job of documentation? What would have to be changed to make you do a good job of documentation? BIBLIOGRAPHY Rosenthal, Robert, On the Social Psychology of the Psychological Experiment, American Scientist, June 1963. Rosenthal has pioneered in studying the observational biases that creep into ex- perimentation, especially because of expectations on the part of the experimenter. The parallels with debugging are obvious, so that programmers should be able to translate the results in this article directly into their own work practices. Parzen, Emanuel, Modern Probability Theory and Its Applications, New York, Wiley, 1960. On the matter of the influence of a priori probability on the confidence in the results of a particular test, see, for example, pp. 119-120. In general, programmers would benefit from thinking of debugging as a probabilistic process, and Parzen’s book would be a good starting place. Weinberg, G. M., PL/I Programming—A Manual of Style, New York, McGraw-Hill, 1970. For a discussion of the advantages of proper language and proper program struc- ture in program testing, see Chapter 4. Berkeley, Edmund C., The Personality of the Interactive Programmed Computer, Computers and Automation, Dec. 1965, pp. 42-46. This article is a bit superficial, but raises the interesting question of the ‘‘person- ality” which a system presents to the user. As Sackman has shown, the attitude a user has toward a system influences his performance in using that system, and also that the user’s respect for the system grows with increased familiarity. No doubt the personality of the system plays a part in this acceptance, and users could be tested for their perception of the personality of different systems, in order to try to determine which personalities programmers like to work with. Sackman, Harold, Man-Computer Problem Solving: Experimental Evaluation of Time-Sharing and Batch Processing, Princeton, N.J., Auerbach Publishers, 1970. As the subtitle suggests, and as we have previously indicated, this book sum- marizes the work to date in this field, and is essential prerequisite reading for anyone about to compare operating systems. Stange, G. H., The Initial Effects of the Installation of Two Remote Computer Access Systems. Unpublished class report. Salsbury, R. G., Ideal Turnaround—An Opinion Survey. Unpublished class report. Heiss, William J., and Frederick Schwartz, Positional versus Keywood Coding. These unpublished class reports provided learning experiences for all involved, and demonstrate that useful work in this field can be done without going off the deep end. Rosin, Robert F., Supervisory and Monitor Systems, Computing Surveys, 1, No. 1 (March 1969). In little more than a dozen pages, Rosin manages to make clear the general out-

Other Programming Tools 273 lines and important issues in operating systems. By taking an historical view, he is able to show how each successive generation of operating system design was more or less intended to patch up the problems left over from the previous genera- tion. One is led to speculate on the way operating systems of the future will look as they begin to patch up the psychological holes left by present systems. London, Ralph L., Bibliography on Proving the Correctness of Computer Programs, Machine Intelligence, 5, New York, American Elsevier, 1970. The dream of all debuggers is to someday be able to have the machine prove whether or not a program is correct. A great deal of work is being done on this problem, although it is certainly unsolvable in general. None of the work done to date has any real practical application to debugging, but reading some of these papers will definitely have an effect on the debugger. No definitive summary of work has been written, or can yet be written, so this bibliography will have to suffice. Weinberg, G. M., and G. L. Gresset, An Experiment in Automatic Verification of Programs, Communications of the ACM, 6, 10 (October 1963), pp. 610-613. In this early study, the authors tried to show how error detection properties of a compiler depend upon the error environment in which they are used, and they made a number of suggestions for improvement of error-detection performance in an actual environment. Few of these suggestions have been carried out in any major compiler. Irons, E. T., An Error Correcting Parse Algorithm, Communications of the ACM, 6, 11 (November 1963), pp. 669-673. LaFrance, Jacques, Optimization of Error Recovery in Syntax-Directed Parsing Algorithms, ACM SIGPLAN Notices, 5, No. 12 (December 1970), pp. 2-17. lrons’ paper represents the earliest, and LaFrance’s one of the most recent, on the subject of automatic correction of syntax errors by compilers. Underlying any “correction’’ scheme must be a concept of “closeness” which dictates that con- struction is to be tried if an error is detected. Workers in this field would like to have a concept of ‘‘closeness” which is independent of the psychology of the user, but this seems to be a vain hope. At the very beginning, at least, the major progress will be made by classifying the most frequently made errors taken from actual studies of working programmers using that language and system and then providing special routines to unscramble these errors. For instance, in the PL/C compiler developed at Cornell University, a major step in correcting and success- fully compiling PL/I programs is made simply by limiting the user to comments and strings that do not run from one card to the next. This limits the scope of propagation of an error and greatly increases the chances of a proper correction or at least a more meaningful compilation so that everything can be corrected in one more run. Only after the major psychological causes of error have been eliminated by these ad hoc methods should we be forced to resort to more elegant methods as suggested in these papers. Teitelman, W., PILOT: A Step Toward Man-Computer Symbiosis (Doctoral Disserta- tion, M.I.T.), Cambridge, Mass., June 1966. The PILOT system is a tool written in LISP for assisting LISP programmers with the job of modifying programs. It contains many interesting features, but it is not clear just how helpful this system is to programmers who habitually use it—or how useful it is compared with simple error detection. Studies along this line would be useful.

274 Programming Tools Stockham, Thomas G., Jr., Some Methods of Graphical Debugging, Proceedings of the IBM Scientific Computing Symposium on Man-Machine Communication, 1966. Stockham reviews the use of CRT devices for dynamic debugging, tracing its his- tory back to 1958, at least. He bemoans the fact that these early experiments did not evolve into common practice, a failure which he attributes to a “variety of technical and economic reasons.” But it could be that lacking any real behavioral data on the value of these systems, there was no motivation to overcome the technical and economic reasons. Certainly there is sufficient economic potential in the cost of poor debugging to justify a big technical push, if we can but demonstrate the rewards. Mills, Harlan D., Syntax-Directed Documentation for PL360, Communications of the ACM, 13, 4 (April 1970). Mills presents a rather ambitious scheme for extracting the material for docu- mentation as a part of the interaction process with the computer when the pro- gram is being written, then making the documentation available through computer processing at a later time. This approach seems to get to the heart of the matter in two ways—it does not seem to require highly motivated people to produce the documentation, nor does it seem to require masses of printed documentation through which one must wade unassisted. In one sense, though, it is the typical computer user’s approach to the problem—‘“‘let’s see if we can make the machine do it.” This is an idea that deserves testing, first to see if it really produces docu- mentation that can be used, and then to see if that is what we really want. Klerer, M., and J. May, Reference Manual, Revised ed., Dobbs Ferry, N.Y., Colum- bia University Hudson Labs., July 1965. This is the two-page document with which the user starts his exploration of the Klerer-May system. The entire document together with the one-page appendix, is reprinted in Sammet’s “Programming Languages” (See Bibliography at the end of the previous chapter). Leeds, H. D., and G. M. Weinberg, Computer Programming Fundamentals/ Based on the IBM System/360, New York, McGraw-Hill, 1970. In this book, you will find a more extensive discussion of documentation of pro- grams than is habitual in programming texts. One chapter deals with flow dia- gramming, another with writeups, and other aspects of documentation are dis- cussed throughout in connection with related topics. Still, it is far from the com- plete or last word on the subject.

Silver Anniversary Notes 13.i Comments on Chapter 13: Other Programming Tools Were | writing this book today, | would reduce the two chapters on program- ming languages to two paragraphs, and | would expand this tools chapter to five or even ten volumes. When | wrote at the opening of the Part 4 Comments that | drooled at the tools of today (drool-tools?), | was not referring to languages, but to the many tools we subconsciously considered as secondary a quarter-century ago. Much of the long-standing prejudice favoring languages over other tools remains today, as we can see most easily when we consider the subject of test- ing and testing tools. The original section was filled with dreams of tools we were just building or yearned to build—code coverage analyzers, static and dynamic structure analyzers, test databases, specification languages, test har- nesses and drivers—and now they are all available, in profusion, along with oth- ers we didn’t even dare dream about back then. Yet the penetration of these tools into actual development work is quite minor—certainly in the light of the overall risks of poor code quality when we don’t use them. What remains today is the problem of ego-full programming and the social-status problem of profes- sional testers. Evidently, the solutions to these problems are not to be found through improved testing tools, but through improved management. The emergence of the personal computer blurred the distinction between the programming language and the operating environment. Thus, many of the main points in these two sections of the original chapter are now seen to be matters of

Silver Anniversary Notes convenience, not matters of primary psychological importance. Still, jobs that are inconvenient to perform are more readily left undone. Although we’ve come a long way toward integration of software tasks, much of the integration is still done by individuals themselves acting as the operating systems of their PCs— perhaps using scripting languages to transform themselves into meta-operating- system designers. Despite what | wrote in 1971, we still have a long way to go in providing soft- ware developers with totally integrated development/testing/maintenance envi- ronments. If the perspective of these two editions is any guide, we probably don’t even see where opportunities for integration exist. There’s no way to fore- see what behaviors will emerge as integration proceeds, except to predict that what we think of today as essential programming work will be seen tomorrow as inessential housekeeping. What justifies this prediction? The past of program- ming has always demonstrated the transformation of essential to inessential. We can also predict that most of today’s programmers won't enjoy changing the definition of their fundamental work—we know because we didn’t enjoy it. Hey, there was something really satisfying about carrying your program deck in both hands, the cards securely fastened with two rubber bands; carefully remov- ing the rubber bands and placing the deck in the card reader; pressing the start button and watching the cards stack up in the tray; pressing end-of-file to run out the last of them; removing them; riffling out any chad; squaring them up on the special platform; replacing the rubber bands; and carrying that solid deck back to your desk! That was true integration, and some of us miss the feeling of control. Well, actually, | don’t miss cards too much (and | miss paper tape even less!), but I’m enough of a reprobate to yearn for some of the forced delays—delays that gave me time to think about what | was doing. Whatever the future of pro- gramming may bring, | firmly believe it will still reward thinking over mindless- ness. Does that mark me as hopelessly old-fashioned? Or should we be doing more to support the programmer who wants time to reflect—and a lot more to support the one who doesn’t? For most of us, time-sharing has become a moribund issue, mercifully killed by technology. What it did for us back then was raise the issue of interface design, an issue Tom Gilb and | explored in Humanized Input. Although that pioneering book was rapidly made obsolete by technological events, interface design, of course, is still a live issue—and is the one area subject to the kind of psychologi- cal research that was my interest in writing Psychology originally. We now have journals and professional societies devoted to the computer-human interface, though none specifically directed at programming or other psychological aspects of programming beyond the interface. It seems to me that interface research results have had little direct influence on the job of the programmer. For one thing, tool designers and process design-

Silver Anniversary Notes [RK ers seem to work on the basis of intuition, not on scientific results. For another, this research has shown that people are so adaptable, they are able to work with almost any interface—and to consider it user-friendly. Finally, if there is a differ- ence among these intuitively designed interfaces, it's not big enough to be a major factor in the complex job of programming. And then there is the subject of documentation. This book provides a kind of documentation, and after twenty-five years, I’ve forgotten most of the troubles | had in first getting it published. | do recall the good times after it became popu- lar, but even if | didn’t, I’m still glad | wrote it. Regardless of any other value to documentation, it’s nice to be able to remember where you’ve been, in order to anticipate where you might be going. As far as software documentation is concerned, it doesn’t seem that we’ve come very far in twenty-five years. Sure, we have some neat new tools, like code formatters and on-line documentation and hypertext, but content still counts most, and few tools have really enhanced content. There’s not much point in having easier access to information you didn’t want in the first place—as the Internet so often demonstrates. The biggest influence on content is the institutionalization of document production as a specialized task in the software business. Many commercial software houses simply don’t require developers to write documentation—certainly not user docu- mentation. They don’t consider documentation to be part of the developer’s job. As | wrote back then, “. .. when we speak about computer program documentation, we are not speaking about the psychology of computer programming at all—except insofar as programmers have the illusion that anyone can do a good job of docu- mentation, provided he is not smart enough to be a programmer’ (p. 270).