63KB
The Psychology of Computer Programming — Chapter-06: The Programming Project
- Source: /library/The Psychology of Computer Programming/source-file.pdf
- PDF pages: 139–162
- Pages without text: none
- Note: Includes the Silver Anniversary comments that immediately follow the original chapter. Text comes from the scan's OCR layer and may contain recognition errors.
THE PROGRAMMING PROJECT hen two teams must work together to accomplish some pro- gramming end, a coordination function emerges just as it did when two programmers had to work together. New types of social relations emerge, just as they did when teams were formed from individuals. For example, two programmers now interact both as individuals and as members of different teams—so that now their team goals may differ, even if their individual goals are very similar. The success of a team member is not only measured relative to other members of his team, but also as part of his team which is measured relative to other teams. Coordination among teams now requires a second level—or higher—of leadership, and so emerges the phenomenon of leaders whose followers are not engaged in the basic work of the project, but are leaders themselves. Many aspects of project management from a management point of view
96 Programming as a Social Activity have been discussed in published books and articles, and we shall not attempt to survey the entire field. What we shall discuss in the next few sections are a few selected aspects—aspects selected for their amenity to psychological treatment and perhaps somewhat as a corrective to cer- tain schools of programming project management thought. STABILITY THROUGH CHANGE One of the most intriguing characteristics of large organizations is their ability to survive over periods of time which are longer than the time that any one member remains. This characteristic is obvious in such organiza- tions as the United States of America, for no person is known to have lived for 200 years. Thus, the United States of America cannot take its continuing identity from some person, no matter how great. Neither Washington, nor Lincoln, nor Jefferson, nor even Joe McCarthy make up the entire story of the United States of America; yet, as obvious as this seems, many managers are unable to apply the same reasoning to their programming project. Even in a programming team, we sometimes see the survival of the team when not a single original member remains. Indeed, the ability to survive its members can be one of the strong points of a team organiza- tion, for the work to be done is not set aside if one person happens to be removed from the scene. This ability comes, of course, from the inter- action among team members, because this interaction permits the goals and accomplishments of the team to be transmitted to new members and so remain when old members depart. In a sense, a programming project or team is like a river which remains the same river even though its water is undergoing constant change. Many project managers are unable to grasp this view of a project. Their view of the project’s structure is, instead, much like that of a house—a structure that might collapse should one of the beams be removed. Their actions with respect to the people in the project—especially so-called “key” people—reflect this static view, often with disastrous consequences. Two cases should illustrate some of the consequences of this view. In one instance, Mark S. had designed the control program for an on-line system to control an industrial process. He had then been asked to stay on to implement the control program, inasmuch as he was the only person who understood the way the system was to work. Mark was not terribly happy about this assignment, for to him the design was tantamount to the implementation. He had already implemented two systems like this and did not see much challenge in implementing a third. There were sufficient differences to keep him moderately interested for a while, but this small
The Programming Project 97 interest was further eroded by the necessity to work in a rather remote spot—at the plant. Mark was a city boy, yet the type of process control system he was developing had twice before required him to work in a remote location for long periods. He was, quite simply, bored. The symptoms of his boredom became manifest when he began requesting that Bob A., a programmer of rather extensive experience, be transferred from another section of the project—where he was writing certain analytical routines—to his team, which then consisted of only himself and two trainees. Bob had expressed an interest in the control program, and this had given Mark the idea to train Bob to take over his functions so that he might move back to the city. The project manager, however, had a different view of things. He was having difficulty getting experienced people to come out in the wilds and work on the project, so he was not about to sacrifice one of his more experienced programmers to satisfy Mark’s evident desire for company. Instead of adding Bob A. to the control program team, he gave Mark another trainee. The trainee was probably the last thing Mark wanted, for he was already doing most of the work himself and carrying the other two trainees aS so much excess baggage. He wanted to get done in a hurry or get replaced, and the new man seemed only to slow him down. He renewed his request for Bob A., who was also unhappy about not getting the chance he desired; but the manager was not going to give in. He recognized that Mark was a “key” man in his project and that he was un- happy, but he didn’t want to go to the trouble of finding someone to replace Bob A. In order to keep Mark happy, he called him in one day and gave him a substantial raise. But before Mark had a chance to draw a single week’s pay at the new rate, he had quit and gone back to the city. Managers are quite frequently surprised to find that a programmer quits a project very soon—sometimes immediately—after getting a solid raise. It is Surprising that they should be so surprised, for there are at least two good psychological reasons for this behavior. Sometimes, the programmer interprets the raise as putting extra responsibility on him, and not everyone wants extra responsibility. The programmer reasons that if he is getting more money, more work is going to be expected of him. He is already pushing himself harder than he wants to, and the raise symbolizes the straw that would break his back. More frequently, however, the raise is interpreted as an attempt to substitute money for something the programmer really wants. Perhaps the programmer was expecting to be named team leader, but the raise indicates to him that a money reward is being given in anticipation of the
98 Programming as a Social Activity team leader spot being given to someone else. In the case of Mark S., there was a correct interpretation that the raise was an attempt to avoid giving him the qualified assistant he wanted, an attempt to bind him more closely to the project. He already felt he was staying on as a favor to the manager, and since the manager had once more refused his quite reasonable request—not for immediate removal, but merely for the chance to train a replacement—he gave up. Notice how the manager in this case, by trying to bind Mark to the project, only succeeded in pushing him out more quickly, and more expensively, than would have been necessary. After Mark left, Bob A. was in fact chosen to replace him; but as he had no training for the job, and nobody with experience to teach him, the project foundered. Finally, more than a year after the scheduled completion date, a patchwork sys- tem was put into operation, but it proved too slow to control all the variables which had been foreseen, due to inefficiencies in the control program. As a result, the anticipated savings from the system were not realized, and after about six months of operation, the entire system was discontinued. A similar case with a different outcome should be instructive as an illustration of the concept of stability through constant change. Henry F. played a similar role to Mark S. in a project connected with the space effort. He had designed the control program and had been made team leader of a somewhat larger and better qualified team. As he began to tire of the implementation work, he filled his time by training his two best qualified team members, Mary M. and Sid B., so that they would know as much about the system as he did. The training was made simpler by the democratic organization of the team and their practice of egoless pro- gramming, but it was not encouraged by higher management. One of the duties that Henry had often to perform was making presenta- tions to visiting dignitaries, for this was a highly visible, high-prestige project. Henry’s presentations were always well received, and the man- agement had twice refused him to allow Mary M. to take his place on the rostrum. Finally, he told Mary to prepare a talk and then arranged to be “sick” on the day of the visit. Management was in a panic, but they had no choice but to try Mary, and she made a superb performance. Now the managers were willing to accept Mary as a substitute leader— in fact, they began to request her services for making presentations—and Henry decided it was time to ask for a transfer to other duties. In spite of Mary’s evident competence, however, Henry’s manager viewed the prospect of his leaving with horror. She was good—he acknowledged that—but Henry had been the original creator of the control program concept. Besides, Mary was a woman, and though the manager had seen
The Programming Project 99 her make outstanding presentations, he could not believe that a woman was capable of leading the group. Henry had been more or less prepared for this reaction, so he now brought up the name of Sid B. Unhappily, the very success of Mary M. had made it hard for the manager to believe that Sid was also competent to lead the group. Henry proposed that Sid be given a chance to make a presentation, but now the manager insisted that nobody but Mary would do. So, Henry arranged for Mary to be “sick,’’ with the result that management finally saw the light. Henry was ready to go, and both Mary and Sid were ready—and willing—to step into his shoes. A final offer was made to give Henry a higher management position within the project, but he had his eye on something else, something which chal- lenged his programming imagination, so he left. But, in spite of the residual fears of the management, the project went on, and to a success- ful conclusion. The differences between these two cases are instructive. No single factor can account for the success of the second as against the failure of the first. Henry was more experienced in dealing with people than Mark; he was fortunate in having a team consisting of other than trainees so that a natural succession was possible without management intervention; the managers in Henry’s project were more enlightened, less rigid, and perhaps under less pressure for success; and, finally, Henry was more patient and understanding of the problem as it would be seen by the management. For Mark’s manager to have been successful, he would not only have had to give Mark the assistant he wanted; but he would have had to help Mark in giving that assistant the training he needed—for Mark rather enjoyed the role of indispensable man. The lessons to be drawn from these two cases and hundreds of others like them are clear. As a project moves forward, people learn things that make them less content with limited responsibilities or—let us call a spade a spade—are simply bored with duties which they feel are beneath their capabilities. Not every programmer wants to advance to more challenging work—some will leave if pushed too fast—but for the most part, as programmers learn they become discontented unless they can apply their knowledge. Thus, to achieve a stable project over a long period of time, a manager must encourage the project to function as a sort of programmer processing plant—with a fresh supply of trainees coming in one end and a stream of experienced leaders coming out the other, with people in the middle being bumped up one position at a time by the flow. One other lesson needs to be drawn explicitly. A project is not a house of cards which collapses when a single “key” person is removed. At
100 Programming as a Social Activity least, it should not be; but often, when management thinks it is, the prophecy becames self-fulfilling. That was one of the lessons in Mark’s case, for if a little less reliance had been placed upon Mark, his manager might have populated his team with other than trainees. After all, even if the ‘‘key’’ man is as contented as a cow with his work, people are sometimes inconsiderate enough of their managers to get sick, to get drafted, or to die. No, if a manager wants to run a stable project, he would do well to follow this simple maxim: If a programmer is indispensable, get rid of him as quickly as possible. MEASURING PERFORMANCE One consequence of the size of a large project is that performance becomes more difficult to measure. Part of this difficulty stems from the scale of things, which makes it impossible for a single mind to be making the judgment of all the parts. Since progress on a program is such a subjective matter, opinions would differ about the amount of progress a particular program had made, even if it were not for other effects. But we never get the opportunity to study the different views on one program. Instead, we compare and combine different people’s views of different programs into some sort of picture of project progress. In making this combination, there are numerous possibilities for psycho- logical mischief. The numerous stages can produce interesting effects, as a result of filtering practices in a large project. An extreme example was found in a military project that involved not only programming but creation of a worldwide communication network. The programming project itself con- sisted of about 75 first-level people organized into twelve teams, with the twelve team leaders organized into three groups and the three group leaders reporting to one programming project manager. Within the com- pany doing the programming, there were also projects to design and build the central computers and special hardware, so these three project managers were organized into a team under a single company project manager. The company project manager participated in a management team consisting of the project managers from the other companies and headed by an overall project manager. | Each month, by the requirements of the contract, a progress report had to be submitted to the government. Naturally, since this was an expensive project, the report had to be printed in an impressive full- color format. This meant that the final copy for the report had to be in the hands of the printer twelve days before the report deadline—the tenth of the month following the month of the report. Thus, for example,
The Programming Project 101 the September report had to be in the hands of the printer by September 28th—and possibly earlier if this fell on a week-end. In order for the overall project manager to have time to review and amend the report, he had to have each company report five working days before the printer’s deadline. Allowing for mailing, this meant that the September company report would have to be finished by, say, the 20th. In this particular company, the company manager needed four working days for review of his three project managers’ reports. Thus, the report of the programming project had to be in by about the 15th. Working backwards like this, we eventually reach the individual pro- gramming team, whose report had to be made about four days before the end of the month preceding the month of the report. Therefore, what the individual team was reporting was not progress for the month, but a prediction for the coming month. What came out the other end, however, was labeled as progress reporting, and nobody seemed to worry about the difference. So far, none of this is particularly psychological, except for two minor factors—the willingness of people to believe something that cannot possibly be what it pretends to be and the interesting relationship between the amount of time needed for reviewing reports, the level of the reviewer, and the amount of work actually contributed. The “‘higher”’ the reviewer, the longer he insisted he must have the reports and the less he actually did with them. Not that nothing was done—dquite the contrary. At each stage of the consolidation, a certain smoothing-out was made, regardless of the content of the report. The reasoning at each stage went something like this. If a subgroup reported an abnormally high amount of progress, the reviewer would shave the amount a trifle under the assumption that it wouldn’t hurt to hold a little in reserve in case their luck changed next month. If, on the other hand, little progress was reported, the reviewer would step it up by a point or two—not wanting to call attention to any weakness and resolving to look into the trouble if things persisted. Similarly, if the list of specific problems was too long, he shortened it a bit, leaving out the least important. If it was too short, he amplified some problem or other into two separate problems. The net result of six or seven stages of such filtering was a report that monthly presented a consistent forward progress, a few areas slightly behind or slightly ahead, a few problems solved from last month, a few new problems, and a few problems still open. There was, in short, no measurable relationship between what had been reported at the bottom and what came out the top. Of course, what went in the bottom was only a prediction of progress anyway, so perhaps it didn’t matter what was done to it on the way up.
102 Programming as a Social Activity In fact, when one of the team leaders in the programming project hap- pened to get hold of a final report, he saw what had happened to his information and decided not to waste his time trying to be accurate. From that point on, instead of bothering his programmers with requests for progress predictions, he just made up a set of nice looking figures in five minutes and passed it on up the line. Within a few months, the same practice had spread to the other programming teams. And so, progress reporting went on with a minimum disturbance, or relation, to actual progress. Where did this project go wrong in its progress reporting? We can identify a number of areas. First, there is the emphasis on reporting, rather than on measuring. Even if the entire movement up the hierarchy had been speeded to five days, there still would have been no check on the meaningfulness of the initial information. And regardless of what some “management information systems” promise, garbage in always yields garbage out—though the converse is not necessarily true. Further- more, the people who were supplying the intial information—the team leaders—could see that there was no check on the meaning of the information they gave. Indeed, the only time anyone ever was called to question on his predictions was when he chose some very large or small figure. In order to minimize the number of such sessions, the leaders learned to adjust their own figures to eliminate extremes. It is a well-known psychological principle that in order to maximize the rate of learning, the subject must be fed back information on how well or poorly he is doing. What is perhaps not so well known is that people who feel that their performance is being judged but who have no adequate information on how well they are doing will test the system by trying certain variations. In a reporting system like this, if the people at the bottom are not fed back information based on their input, they will start to vary the input in arbitrary ways to see the effect—to get some feedback, even at the risk of it being a poor evaluation. In this case, the team leaders were already well on the way to learning that the only important factor in their reports was that they contain noth- ing striking. Then, one of them chanced to see a final report, which only completed the picture that the entire reporting scheme was an exercise in futility. One additional piece of news only confirmed an already held suspicion—and at least had the advantage of saving everyone a lot of time. From an information-theoretic point of view, this reporting system can be seen as a series of filters—each with a certain delay and each with a certain loss of information. Information, in this sense, is carried by a report to the extent that it carries surprising news. If we read in the paper that the sun came up yesterday, that conveys little information;
The Programming Project 103 but if we read that the sun did not come up, a lot of information is trans- mitted to us because of the unexpectedness of the event. Similarly, in a progress report, the most information is carried in those items which indicate very much or very little progress. But what happens to those items? In psychological testing, when subjects are asked to make judgments along some linear scale, very few subjects will ever choose the end points of the scale. In fact, if a discrimination of, say, five levels is desired, a seven or nine point scale will usually be used and the end points—if they are chosen at all—are lumped in with extremes closer to the center. This effect is probably related to what we see in the filtering of the report at each stage, with each editor softening the harsh extremes left over by the last. The extremes in this case, however, are relative to what has been left in by the previous editor, so the report moves more and more toward the center in each of its figures. In doing so, however, it is filtering out precisely the information it is supposedly designed to reveal. Is there not some way this filtering can be defeated? To answer that question, we must identify the psychological source of the tendency to move the extremes toward the middle. One possibility is cognitive dissonance. To the extent that a manager believes that good management creates a smoothly running operation, he is likely to find fluctuations in performance inconsistent with this managerial image. From there it is a small step to believing that the fluctuations are simply misjudgments on the part of the people who are ‘“‘too close to the situa- tion to get the big picture’. Another possibility—not excluding the cognitive dissonance idea—is implied or inferred pressure from higher up. Pressure from higher up is classically recognized by management manuals as both the way to get work done and the way to destroy a reporting system. What a manager must ijearn to do is to motivate people to modify the way they work or the rate at which they work, not motivate them to conceal what they are doing. Therefore, he has to reward an accurate reporting of good work. Since the person doing the reporting is usually the one responsible for the work being reported, it is difficult, if not impossible, to reward him for accurate reporting at the same time he is being rep- rimanded for poor performance. Inasmuch as the hierarchical system of organization requires this confusion of goals for its managers, we generally see that reporting systems in this type of organization move further and further away from meaningfulness as time goes on. Even where the project is not organized hierarchically, pressure to modify judgments of progress can come from other sources, particularly from colleagues. In a classical series of experiments, Asch demon-
104 Programming as a Social Activity strated how people could be made to modify their judgment of the relative lengths of two lines simply by the pressure of knowing the opinions of others. In a typical experiment, six “subjects” sit behind a table and are asked to state which of two straight lines on a card is the longer. The only true subject is seated perhaps in the second-to-last position, and the others are told secretly what answers to give. After a few rounds of correct answers, all the “subjects” preceding the true subject say that the shorter line is the longer. This creates a pressure on the subject, who can see that they are wrong but would have to contradict them all if he is to assert the facts of the matter. Some people, it is found, never yield to these pressures, but most do. As we might expect, more yield when the lines are close in length—but some even yield when the longer is three times as long as the shorter. Moreover, when interviewed, they believe in the judgments they made, as we might expect from dissonance theory. There are many variants to these social pressure experiments, and it would pay the programming manager to study them. For our present purposes, however, it is sufficient to notice how opinions on concrete matters can be easily influenced by the announced opinions of others, and how the influence becomes more effective as the judgment becomes more difficult. Under the circumstances, it is easy to see how such a difficult and subjective judgment as the progress on a programming project could come under the influence of one’s peers—either in a meeting or through reading their reports. One other relevant result of these experiments is the effect on judgment when the pressure for the wrong selection is not unanimous. If, for example, the second “subject” is instructed to make the correct choice when all the others choose the wrong one, almost all influence of the wrong group on the true subject disappears. In other words, the presence of even a single “ally” gives many a man the courage to see things as they really are. A striking example of this effect occurred in one project where the system testing of six subsystems together was to begin the following week on a machine that was located across the country—so that a crew of eight people would have to be supported on expenses while the testing was going on. At the final meeting of the six team leaders with their manager before system testing, a review was made of progress. Every- one agreed that all was on schedule and that the system testing crew could be sent out and set up in their hotel. After discussing various tech- nicalities and logistic problems, the manager prepared to wind up the meeting. “Just one more check,” he said, “before we adjourn. Does anyone see any delay in having his part done by next week?” — a
The Programming Project 105 There was a silence, which the manager allowed to prolong itself for a full sixty seconds. Finally, one of the team leaders made a slight movement of his hand, almost as if he didn’t wish it to be noticed. But the manager picked it up. ‘George. Do you have a problem?” George squirmed. “Just a little one.” “How little?” “A slight delay.” “How long?” “Umm. Maybe six weeks.” The room exploded. “Six weeks?” they all shouted at once. “How can you sit there through the whole meeting and not say anything when we're getting ready for system test and you’re six weeks behind sched- ule?” The manager calmed down the other team leaders and congratulated George on having the courage to admit his difficulties before they had gone to the expense of setting up their system test crew in a hotel. After some discussion, he persuaded George to try and get his part ready in four weeks and set a new system test schedule. Then, the manager was about to adjourn once more, but once more he asked for any further problems. “Well,”” one of the other leaders put in reluctantly. “If George gets four extra weeks, | think my group should too.” “You mean,” asked the manager, ‘“‘that you’re not ready either.” “Not exactly.” “How much not exactly?” “I! guess about six weeks. But we’ll try to do it in four.” Eventually, with the floodgates thus opened, it turned out that every one of the six parts was behind schedule. And yet, if George hadn't taken the first step to admitting what they all knew, the meeting might have closed without a single hint of the trouble, and a fruitless system testing effort would have begun at enormous expense. Recognizing that even a single ally can provide a safety valve to re- lease the social pressure to produce conforming opinions, some pro- gramming projects have established a ‘“‘devil’s advocate”’ system for their management meetings. In a typical situation, a technical staff assistant to the project manager assumes this role. In every meeting, it is his duty to raise all possible negative points to opinions which all the others seem to share. By thus expressing disagreement with the group, he provides an anchor point for anyone whose lurking doubts were being restrained by the difficulty of being one against many. Because the role of devil’s advocate is institutionalized, the pressure on him is less than it is on any other member of the group who is first to express a dissenting opinion. Nevertheless, the official devil’s advocate
106 Programming as a Social Activity may find himself under increasing social pressure as time goes on, for the others will have difficulty in remembering that he is acting officially and is not necessarily against the others all the time. If this happens, his ability to play the role effectively may diminish just at the time when it should be increasing. Some projects attempt to avoid this personalizing of the devil’s advocate role by rotating the position among the various leaders from one meeting or one report to the next. In this way, they each have the opportunity to learn that taking the negative point of view is not necessarily a negative activity. PROJECT STRUCTURE If a programming project is going to overcome the psychological pit- falls inherent in progress evaluation, some sort of separation between doing the work and evaluating it is essential. Probably the most serious weakness of the hierarchical organization lies in its inability to achieve this separation, because the lines of control for doing the work are simply the lines of progress reporting run in reverse. Such a system, as we have seen, generally leads to a deterioration in the quality of in- formation communicated through it. In order to combat this deterioration, deviations from a simple hierarchic structure begin to emerge. Many projects have a standards group whose principal duty is not production, but evaluation of the production of other groups. When the testing stage is reached, the standards group—or part of it—may become responsible for system testing, or a system testing group may have existed from the beginning. Special groups also arise for functions other than the evaluation of progress—a library group, a hardware group, a group of documentation specialists, or a systems programming group responsible for all the service programs used by the project. One type of group, which is particularly useful in projects of a high degree of visibility, is the public relations group, whose activities shield the other groups from daily interference from inquisitive outsiders. Finally, various groups may be set up for special projects of limited duration. For instance, if there is a bug in the system which ordinary methods do not find, a special task force may be set up to root it out. To be able to establish such temporary groups, the project manager must preserve a certain amount of ‘“‘slack”’ in the project, so that people can be extracted and used to plug the leak. No project, no matter how carefully planned, can anticipate all contingencies and set up groups in advance to meet them. Thus, the typical project may expect to experience several reorgan- izations throughout its lifetime. When a reorganization is made, it sometimes reveals more worms than ae ee ee
The Programming Project 107 lifting a large flat rock. People refuse to move from one group to another, to work with a certain person or on a certain job. Others insist that they must continue working with a certain person, or on a certain job, or that they must go to a different group. Essentially, all these problems stem from a division of loyalties, much like the problems arising from an in- dividual going to work on a team. Each person working on a team within a project has come to accept certain of the team goals as his own. But when the team is specialized, its goals do not necessarily coincide with the overall goals of the project, or with the goals of another team. These conflicts are invariably manifest in some sort of social relations. For example, certain group tasks are considered less prestigious, more menial, than others. A documentation group, for example, is often looked down upon by mem- bers of working programming teams—with the result that the documenters become defensive and clannish, avoiding contacts with other team members as much as possible. But, unfortunately, avoidance of others is precisely what we do not want in a documentation team. Managers are not guiltless in encouraging this type of division in a project. In the first place, they may simply have their own prejudices and permit them to become reified in the project structure. In one large project, the manager was telling some visitors about the training pro- gram they had, in order to demonstrate that he understood the concept of stability through change. He showed the visitors a list of the new trainees arranged by their final evaluation scores, and called their atten- tion to the fine scores they ‘“‘all” had achieved. “But,” asked one of the visitors, “what about these three at the bottom? Their scores seem to be way below the others.” “Oh, that,” shrugged the manager. ‘‘That’s nothing. They’re going into the documentation group, so it’s nothing to worry about.” Can we be surprised if the other groups hold a low opinion of the documenters and do not want to cooperate with them? Any testing group is also put into a difficult relationship with other groups, because it is their job to criticize. One of our students who was working in a program testing group did a study of social interactions between members of her group and of the group whose programs they were testing. Although their offices were intermingled, there was almost no social interaction between them. Members of each group did not lunch or take coffee with members of the other, nor did they do even such simple things as borrow pencils or paper clips from one another. In con- versations among members of a group, members of the other group were typically characterized as unintelligent, unfriendly, and unfair. To some extent, this attitude of mutual dislike was encouraged by the management, for when this student tried to arrange a party for the two
108 Programming as a Social Activity groups, she met with strong management resistance on the grounds that it was not “healthy” for these two groups to become very close. When she explained that she wanted the party just to test this assumption for her class in the psychology of computer programming, she was reluctantly given permission. The party was held under the pretext of a celebration of the successful completion of testing of a major piece of the project. In the week before the project, social contacts increased between the groups, and the party itself was a great success. Afterward, according to her observations, the two groups seemed to be able to carry out their mutual work with more personal contact and less traffic in recriminating memoranda. We have no way of measuring whether or not this improved social climate led to better or worse work between these two groups. However, we do know through our experiences with egoless programming that there is no particular reason why your friend cannot also be your sternest critic. We could certainly use controlled experiments on this question. In the meantime, it probably is a good idea for a project manager not to foster conflict between groups whose work brings them into some sort of natural conflict. Another natural conflict which arises in projects cuts horizontally in- stead of vertically—tending to separate the management from the working programmers. This conflict arises over the setting of goals, for project goals are not simply the sum of individual team goals. Typically, goals are set from the top down, with major decisions having been made before most of the teams have even been put together. Such a structure does not encourage democratic goal-setting within the teams, and it may there- fore be difficult to get the programmers to have loyalty to the project as well as to their own group. Many of these problems can be overcome if the management of the project can see the operation as a machine for getting the project done, rather than as a pyramid to be climbed for the satisfaction of their personal ambitions. In an engine, for example, the valves are not the “boss” of the cylinders, nor is the crankshaft the ‘‘boss” of the valves. The hierarchical organization, which so many of our projects seem to emulate, comes to us not from the observation of successful machines or natural systems, but from the nineteenth century successes of the Austrian Army. Yet it would be difficult to imagine two groups which differ more than a bunch of privates and a bunch of programmers. In the army—old-fashioned style—every footsoldier was considered interchangeable with every other. The hierarchical organization, then, was conceived as the structure that could give the fastest and most direct coordination between these interchangeable parts. But a programming project is not a battle, regardless of appearances. There is no need for a
The Programming Project 109 quite the speed of communication which is necessary under field condi- tions, nor are the things to be communicated so simple that they can be barked over a two-way radio with shells bursting in the background. What is needed in a programming project is slow, careful communication among teams of people doing very different, highly specialized tasks. Moreover, the programmer is not drafted into his job, and he is not subject to courts-martial if his program does not meet specifications. And finally, there is no danger to the officers of being hit by enemy fire or being shot in the back by their own men, so there is really no excuse for not getting down in the trenches once in a while with the troops. COMMON SOCIAL PROBLEMS OF LARGE PROJECTS The remoteness of the project leadership from the workers is the source of many social problems in large projects. There are, within a programming team, only two roles to fill—the programmer and the leader of programmers. Within the project, however, there emerges a third role, the leader of leaders, or nth-level manager. The differences between first- level and -nth-level managers are significant, for the first-level manager maintains at least some contact with the actual work being done, and the nth-level manager only sees the work indirectly, through other managers. Even though the nth-level. manager may have originally been a pro- grammer, which is rare enough, he has almost certainly lost any program- ming ability by the time he reaches these heights. Given the rapid change of programming techniques and hardware, the higher-level manager be- comes obsolete even if he retains the skills he originally had and keeps them sharply honed, for those languages and those machines are no longer in use. In a small survey of working programmers, only 15 percent of the first-line managers were thought to be as skillful as the pro- grammers themselves, and none of the higher-level managers. Some of the programmers commented that a higher-level manager sometimes came around and looked at what they were doing, but that his attempts to look knowing were perfectly transparent to the programmers. Nobody, of course, was going to risk challenging his manager’s manager, but after he left, all would have a good chuckle at his expense. Whenever a supervisor is responsible for work he does not understand, he begins to reward workers not for work, but for the appearance of work. Programmers who arrive early in the morning are thought to be better programmers than ones who are seen to arrive after official start- ing time. Programmers who work late, however, may not be rewarded
110 Programming as a Social Activity because the manager is not likely to see that they are working late. Programmers who are observed talking to others are not considered to be working, because the manager has an image that programming work involves the solitary thinker scratching out secret messages to the computer. But what else can a manager do? Since the results of programming work will not be available for many months, he cannot just wait to see whose programs run and whose don’t—even if he could tell the difference. Probably the best thing the manager can do is to rely on his first-level managers, and to insist that they actually observe the work—yes, even going so far as reading the programs. Lurking around and dropping in on the programmers to give them the impression that he is watching isn’t going to fool anybody. One programmer we interviewed told us that he had “earned” a fat raise by preparing a program that produced some output that the project manager thought he understood. When the project manager dropped into his office, the programmer took out this program to show him, and asked his advice on how to handle a certain situation. The manager gave him some advice—which would have been worthless even if the program had been part of the system they were producing—and left beaming. A few days later, the programmer made a point of thanking the manager in the lunch room, in front of his lower-level managers; and a few days after that, he got his raise. Being thanked for some technical advice must have elevated the status of the project manager in the eyes of his fellows, for such direct per- formances are rare for managers. At the working level, status is largely determined by the ability to perform technical tasks—such as legendary feats of debugging. But for the manager, such avenues to earning status are not normally open, though he may often bore everybody by recalling the good old days when they were. As he climbs the management ladder, it becomes less and less easy for him to establish his technical superiority, so he comes to rely more and more on symbols of status to give him his authority. Status symbols are always amusing—to anyone outside the system. Within programming, some special status symbols not found in other professions are worth mentioning even though the subject is well-treated elsewhere. For instance, we have a card file in the office. At certain low levels, the card file is a positive status symbol over ordinary programmers, because it implies that its owner cannot leave his important programs unguarded. At higher levels, of course, the card file must be removed, lest the observer should think that one was too close to the actual pro- gramming. In shops which run a closed machine room, anyone with per- mission to enter the machine room is obviously more important than anyone who does not—though usually it is safest to give such permission
The Programming Project 111 only to those high enough so that they really have no business being there. Where access to the machine room is not the issue, priority of running jobs may be, especially if each person’s priority is listed—in descending order—in a conspicuous place. It must be tremendously satisfying to the project manager to know that his job would be run ahead of everyone else’s—if he ever had one. Lately, the terminal has become the number one status symbol for programming managers. No matter how short of terminals the poor workers may be, the project manager must have his terminal by his side for all his guests to see. What he uses it for is rarely known, or at least is a well-guarded secret. What the casual visitor usually sees is a hoked- up demonstration or a game of Nim or Spacewar, depending on the type of terminal. One of the most beautiful of such demonstrations con- sisted of displaying a PERT chart of some project function on the CRT. When the manager pushed a button, the critical path in the chart was illuminated with greater brightness than the other paths. Then, by point- ing the light pen at one of the nodes, a deletion was made and the new chart was displayed. One more button push and the new critical path was illuminated. This demonstration never failed to impress visitors with the great control the manager had over his project. Of course, it was a complete fake—all four patterns were prestored in the system, and they were the only four patterns available. One of the classic status symbols from which programming managers are not exempt is the secretary or the administrative assistant. The differ- ence between these two titles is that secretaries are always female, and administrative assistants may be of either gender. The prettiest secretary may be a status symbol—though an ugly one may be taken as indicating that one’s status is so high that one rates a competent secretary. This association of women with the menial tasks of an office and of good looks with poor brains is the source of one of the more serious social problems in large projects—the sex problem. One of the women in our class undertook to study the attitudes about women programmers of men in her project. The picture that emerged from this survey was of men (especially the older ones) who plastered over their insecurities by belittling the women who were involved in professional work. Women working as secretaries did not come in for this abuse—presumably, they “knew their place.” But anything the women programmers did was in- terpreted in the worst possible light. For example, if a woman was seen talking to another woman, the tendency was to call it “gossip.” If a woman talked to a man, it was “flirting”. But when men talked to men, there was no special name—it was just “business,” even when the business was baseball, basketball, or bowling. In one section of the project, there were only two women, and they had been put into the same team because the manager thought
112 Programming as a Social Activity they would “like each other’s company.”’ Whenever the manager saw the team leader, he would ask how “they” were doing. Everyone understood who “they” were. Most men, of course, will dismiss these observations as not being serious. “Of course men talk about women in those ways, but a little friendly joking never hurt anybody.”’ Even the women who are not pro- fessionally involved tend to support the system by taking every opportunity to express disbelief that other women could ‘“‘do such complicated things” —meaning, ‘“‘compete with men.” But to the women involved, the matter is deadly serious—though they are not about to let any curious male know about it. In many projects, women are systematically excluded from management positions, or management positions above the first level. Thus, a woman who has such aspirations knows that her future is limited as long as she stays on this project—not a very healthy way to inspire participation. To be sure, male managers can offer all sorts of rationalizations for such policies, but such rationalizations accompany any prejudice. In- dividual cases can be called upon to support each erroneous belief, but that is true of any prejudice, too. Anybody who thinks Scotsmen are cheap can point to an example of a Scot who saved little pieces of string; anyone who thinks Sicilians are gangsters can show you a newspaper clipping to ‘prove’ his case; and any manager who won't promote a woman can point to a case where a woman was promoted and then left to have a baby or to follow her husband to his new job. And, of course, if a woman doesn’t have babies or follow her husband, he says, “What kind of a woman is she?” Each prejudice has its price. In a programming project, the exclusion of anyone from any position on any basis besides lack of competence robs the project of the best possible performance. Moreover, once one faction begins to feel that they are being judged differently from others, they will begin to act differently. Although prejudices against other groups can also be serious matters, the prejudice against women is so common in programming that it merits special attention. Possibly the greatest single action to relieve the shortage of programming and programming manage- ment talent would be to start treating women as true equals—if indeed they are only that. SUMMARY At the end of 1969, the second NATO conference on Software Engineer- ing was held, and the tone was already much changed from the one held the previous year. People were beginning to come into the picture,
The Programming Project 113 and the reason is not hard to find in the proceedings. We may quote this comment from Joel Aron as most clearly representing the problem: We made a study of about a dozen projects, though not in a very formal manner. However, our results were convincing enough to us to set up a course on programming systems management. The nature of the study was ‘‘Why do our projects succeed or fail?” We took as “successful” a project that met its requirements on schedule within the budgeted dollars and satisfied the customer. On this basis, out of 10 or 12 proj- ects that we examined, we had one success and a whole lot of failures. We analyzed the reasons for failure, as given to us by the project managers, and by people who had performed previous evaluations of the projects. They gave various reasons behind the failure of the projects, virtually all of which were essentially management failures. We ran into problems because we didn’t know how to manage what we had, not because we lacked the techniques themselves. (Emphasis added.) Over the years, | have had the opportunity to work with Joel Aron as he has crusaded to get people to believe what he says about programming projects. The sad part of it is that most of the computing fraternity re- mains unconvinced. On the very next page of the proceedings, C. A. R. Hoare, a very brilliant man, says: Basically all problems are technical. If you know what you want to do and you have the necessary technical background, there is no point in making a great management problem out of it. Obviously a certain amount of resource control and personnel work have to go on but that’s all. Hoare is right, in a certain sense, but it doesn’t work out that way. People are trying to cope with problems they don’t understand, and those they don’t know they don’t understand—in itself a failure of man- agement. We have tried to show how a project can go down the drain even when it does have the technical capability—for if people remain as unconvinced as Mr. Hoare, there will be no fruitful study in this area. The picture is only sketched in broad outline with a few anecdotal details, but it must be filled in, or we must stop trying to do programming projects until it is. QUESTIONS For Managers
- How long have you been in charge of your present group? How many of the original people remain? Make a list of the people who have left and give the reasons for their departure. What sort of provisions do you make for this kind of turnover?
114 10. Programming as a Social Activity Have you ever had someone quit soon after a promotion or a raise? Was the promotion or raise a substitute for something you would not or could not give that person? What reasons were given for quitting, and what do you think the real reasons were? Do you have an indispensable woman or man working for you? If so, what would you do if she or he got killed or was sick for six months? In what ways do you modify information as it passes you on its way up through your organization? What does your manager do to reward you or punish you for accurate information? Which do you reward most, accurate information or pleasing in- formation? Do your programmers know what the information you require is used for? Do they see the final reports which are the destination of their information? If not, why not? At your next management meeting, try to play devil’s advocate on some point on which there is otherwise common agreement. Record the pressures you feel, and the reactions of those trying to bring you into line by other than rational arguments. Do you favor certain of your teams as being more important? Do you see certain of the teams as more prestigious than others? Which of your teams have the greatest morale problems? Do you encourage competition between groups with conflicting in- terests in order to stimulate production? How can you measure whether or not this strategy is working? . Look around your office and make a list of the status symbols you find there. If you can’t find any, go to some other manager’s office and make a list of his status symbols. Then go back to your office and try to make another list. Are women in your organization treated any differently than men? In what situations, and for what reasons? Are the women in your organ- ization contributing less than the men, or don’t you know? For Programmers 1. 3. 4. Have you ever felt that you were being forced into staying in a posi- tion that you wanted to leave? What methods were used to force you? How did it affect your work in that position? Have you ever been given money as a substitute for something else you wanted, such as a different job, more help, better access to the machine, or different working hours? Did the money satisfy you so that you no longer wanted the other thing? Are you indispensable in your position? If so, what are you doing about it, making yourself more or less indispensable? Do you know what happens to your progress reports or time sheets? Do you ever wonder? Do you ever ask?
The Programming Project 115 5. Have you ever yielded to group or manageriat pressure against your best technical judgment? Describe the situation and the conse- quences. 6. From the point of view of prestige, how do you rate the following types of assignments, and why? Documentation Program library work System testing . Progress reporting . Standards work . Diagnostic programming g. Education ~29290 5p 7. What status symbols do you have in your office? See Manager’s question 9. 8. What is the attitude toward women in your organization? If you are a woman, what does that attitude do to you and your work? BIBLIOGRAPHY Naur, Peter, and Brian Randell, eds., Software Engineering, NATO Science Com- mittee Report, January 1969. Buxton, J. N., and Brian Randell, eds., Software Engineering Techniques, NATO Science Commitee Report, April 1970. These two conferences brought together some of the leaders of the international programming community to discuss problems of controlling software projects. Al- though the reports cannot be expected to capture the full vitality of the confer- ences, the editors have done a fine job, and there is much information on software engineering to be gleaned from them—although there is not nearly enough from the point of view of the behavioral sciences. Asch, S. E., Studies of Independence and Submission to Group Pressure: I. A Minority of One Against a Unanimous Majority, Psychological Monographs, 7, Series No. 416, 1956. This is the classical and original work on opinions and social pressure, and it should be required reading for all technical people. Kantowitz, Leo, Women and the Law: The Unfinished Revolution, Albuquerque, University of New Mexico Press, 1969. A fine antidote for those who feel that women are treated equally with men in the United States, as seen through the codification of laws over time. Friedan, Betty, The Feminine Mystique, New York, Norton, 1963. One of the first of the new “women’s liberation” books, giving the way our world looks to a woman who has her wits about her and her consciousness raised. Gagne, Robert M., ed., Psychological Principles in System Development, New York, Holt, Rinehart, and Winston, 1962. This book is a good job documentation of the “Systems Development’ movement of the fifties, which was largely a baby of the military, through such agencies as
116 Programming as a Social Activity RAND and SDC, with some participation from the universities. It is not clear that the ‘‘movement’”’ was ever very successful at accomplishing anything but getting large grants of money from the government, and certainly its influence on non- military work has so far been rather small. The level of work in the collection is uneven, ranging from superficial platitudes to detailed system diagrams and photographs, and there is much overlap. The “psychological” aspect is largely concerned with how to view people as components in complex system—the human engineering approach and the personnel approach. There is really nothing of value on computer programming as such, and the main interest in this book is discovering why this approach does not succeed in programming. Meyer, Marshall W., Automation and Bureaucratic Structure, American Journal of Sociology 74, 3(1968), pp. 256-264. This study, which is a survey of 254 departments of finance at state, county, and city levels, shows the differences in structure between the data-processing sections and the other parts of the organization. Among other things, first-line supervisors supervise more people, and higher-level ones fewer people, giving a narrow or- ganization with a wide bottom. This expansion of the span of contro! at the lowest level and extension of the number of levels to reach the top is just what we don’t need in computer programming, where first-line managers already don’t know enough about what their people are doing, and communication up the tree is already too slow and unreliable.
Silver Anniversary Notes Pea hae | Comments on Chapter 6: The Programming Project After twenty-five years, a new generation of managers still believes project loyalty can be bought with money—by using salary increases or bonuses when they can’t or won’t reward developers with other things they value. Sadly, the idea persists because to some extent it’s true. But the longer a project lasts, the less binding the money. Moreover, the same people who can be bound by money can be released by more money. Some managers have learned this lesson and now look for other ways to reward people for staying on their projects. Some even construct their projects in small, transferrable units of work, so they don’t need to bind people—or want to. Over the years, I’ve been thanked countless times by managers for the wake- up call of this section, still true today: If a programmer is indispensable, get rid of him as quickly as possible. (p. 100) I’ve expanded the important and timely topic of measurement into Volume 2 of the Quality Software Management series. Not much has changed in the psy- chology of measurement. People still play the games described in this chap- ter—schedule fiction and schedule chicken—because people are still pressured to give the answers management wants, rather than giving information about the true state of their projects.
SS Silver Anniversary Notes After a quarter-century of doing software projects, | can confirm that the Austrian Army model still prevails—at least in most managers’ minds. The reali- ties of who-talks-to-whom may be quite different today, as I’m sure they were dif- ferent in the Austrian Army. Still, it’s encouraging to find that it's now permissible to speak or write about other models, especially those emphasizing teamwork as the essential kernel of a successful project.1 Projects come and go, but the same social problems remain. | now recognize that the root of many of these problems is low self-esteem. The cure lies in an environment that boosts self-esteem, not one that destroys it. After a generation, many higher-level managers are still clueless about all. aspects of software. At least now we have a thriving software industry that gives software developers a chance to show what they can do in upper management. Some are doing quite well, while others are demonstrating that their managerial problem is not lack of software experience. At the same time, these years have shown quite clearly that people with no software experience can successfully manage large software projects. So, it seems, the lack of software skill never was the problem. Lack of managerial skill was always the problem; lack of software skill was merely a convenient excuse for those managers who lacked esteem. The particular status symbols that were so much in evidence twenty-five years ago have changed as hardware has changed, but there are still status symbols. This shows that it’s not the particular symbols that matter, but, rather, the need for status symbols, a need that also arises from low self-esteem. By and large, professional women in projects are no longer treated as a differ- ent species, but they do seem to be viewed by some as an inferior form of the same species. | suppose that’s a kind of progress, but after a generation, you’d think we could have done better than that. Perhaps we would have, too, if so many men didn’t feel so inadequate. Way back then, | wrote, “People are trying to cope with problems they don’t understand, and those they don’t know they don’t understand—in itself a failure of management. . . . The picture is only sketched in broad outline with a few anecdo- tal details, but it must be filled in, or we must stop trying to do programming pro- jects until it is” (p. 113). Fortunately, we didn’t stop to wait for research, so our experience actually doing projects has become our research. Today, we have lots and lots of successful projects under our belts, and lots of unsuccessful ones— more than enough data to fill in the picture of what it takes to succeed in software. For myself, the biggest missing detail was failure to understand the emotional dimensions of project management. I’ve spent these years filling in those dimensions, mostly in the Quality Software Management series, and now my success rate as a consultant is much, much higher. 1 See, for example, Jim McCarthy, Dynamics of Software Development (Redmond, Wash.: Microsoft Press, 1995). a