History of CP/CMS

This article covers the History of CP/CMS — the historical context in which this important IBM time-sharing virtual machine operating system was built. It provides details to support the main CP/CMS and History of IBM articles, drawing on source material that is not readily available on-line.

CP/CMS development occurred in a complex political and technical milieu. To understand the system's history, it is necessary to examine these broader forces. The following material summarizes major issues and events of the day from the perspective of CP/CMS development a perspective that is somewhat different from (and orthogonal to) other ways of viewing the period. A key theme is the politics of a long running feud at IBM, between time-sharing and batch processing exponents.

Historical notes, below, provides supporting quotes and citations from first-hand observers.

Early 60s: CTSS, early time-sharing, and Project MAC

The seminal first-generation time-sharing system was CTSS, first demonstrated at MIT in 1961 and in production use from 1964–74.[1] It paved the way for Multics, CP/CMS, and all other time-sharing environments. Time-sharing concepts were first articulated in the late 50s, particularly as a way to meet the needs of scientific computing. At the time, computers were primarily used for batch processing — where jobs were submitted on punch cards, and run in sequence. Time-sharing let users interact directly with a computer, so that calculation and simulation results could be seen immediately.

Scientific users quickly embraced the concept of time-sharing, and pressured computer vendors such as IBM for improved time-sharing capabilities. MIT researchers spearheaded this effort, launching Project MAC, which was intended to develop the next generation of time-sharing technology and which would ultimately build Multics, an extremely feature-rich time-sharing system that would later inspire the initial development of UNIX.[2] This high-profile team of leading computer scientists formed very specific technical recommendations and requirements, seeking an appropriate hardware platform for their new system. The technical problems were awesome. Most early time-sharing systems sidestepped these problems by giving users new or modified languages, such as Dartmouth BASIC, which were accessed through interpreters or restricted execution contexts. But the Project MAC vision was for shared, unrestricted access to general-purpose computing.

Along with other vendors, IBM submitted a proposal to Project MAC. However, IBM's proposal was not well received: To IBM's surprise, MIT chose GE as the Multics system vendor. The fallout from this event led directly to CP/CMS.

Further information: IBM mainframe operating systems and CTSS

IBM and the System/360

In the early 60s, IBM was struggling to define its technical direction. The company had identified a problem with its past computer offerings: incompatibility between the many IBM products and product lines. Each new product family, and each new generation of technology, forced customers to wrestle with an entirely new set of technical specifications. IBM products incorporated a wide variety of processor designs, memory architectures, instruction sets, input/output strategies, etc. This was not, of course, unique to IBM. All computer vendors seemed to begin each new system with a "clean sheet" design. IBM saw this as both a problem and an opportunity. The cost of software migration was an increasing barrier to hardware sales. Customers could not afford to upgrade their computers, and IBM wanted to change this.

IBM embarked on a very risky undertaking: the System/360. This product line was intended to replace IBM's diverse earlier offerings, including the IBM 7000 series, the canceled IBM 8000 series, the IBM 1130 series, and various other specialized machines used for scientific and other applications. The System/360 would span an unprecedented range of processing power, memory size, device support, and cost; and more important, it was based on a pledge of backward compatibility, such that any customer could move software to a new system without modification. In today's world of standard interfaces and portable systems, this may not seem such a radical goal; but at the time, it was revolutionary. Before the System/360, each computer model often had its own specific devices and programs that could not be used with other systems. Buying a bigger CPU also meant buying new printers, card readers, tape drives, etc. In addition, customers would have to rewrite their programs to run on the new CPU, something customers often balked at. With the S/360, IBM wanted to offer a huge range of computer systems, all sharing a single processor architecture, instruction set, I/O interface, and operating system. Customers would be able to "mix and match" to meet current needs; and they could confidently upgrade their systems in the future, without the need to rewrite all their software applications. IBM's focus remained on its traditional customer base: large organizations doing administrative and business data processing.

At the start of System/360 project, IBM did not fully appreciate the amount of risk involved. The System/360 ultimately gave IBM total dominance over the computer industry; but at first, it nearly put IBM out of business. IBM took on one of the largest and most ambitious engineering projects in history, and in the process discovered diseconomies of scale and the mythical man-month. Extensive literature on the period, such as that by Fred Brooks, illustrate the pitfalls.

It was during the period of System/360 panic that Project MAC asked IBM to provide computers with extensive time-sharing capabilities. This was not the direction the System/360 project was going. Time-sharing wasn't seen as important to IBM's main customer base; batch processing was key. Moreover, time-sharing was new ground. Many of the concepts involved, such as virtual memory, remained unproven. For example: At the time, nobody could explain why the troubled Manchester/Ferranti Atlas virtual memory "didn't work".[3] This was later explained as due to thrashing, based on CP/CMS and M44/44X research. As a result, IBM's System/360 announcement in April 1964 did not include key elements sought by the time-sharing advocates, particularly virtual memory capabilities. Project MAC researchers were crushed and angered by this decision. The System/360 design team met with Project MAC researchers, and listened to their objections; but IBM chose another path.

In February 1964, at the height of these events, IBM had launched its Cambridge Scientific Center (CSC), headed by Norm Rassmussen. CSC was to serve as the link between MIT researchers and the IBM labs, and was located in the same building with Project MAC. IBM fully expected to win the Project MAC competition, and to retain its perceived lead in scientific computing and time-sharing.

One of CSC's first projects was to submit IBM's Project MAC proposal. IBM had received intelligence that MIT was leaning toward the GE proposal, which was for a modified 600-series computer with virtual memory hardware and other enhancements; this would eventually become the GE 645. IBM proposed a modified S/360 that would include a virtual memory device called the "Blaauw Box" a component that had been designed for, but not included in, the S/360. The MIT team rejected IBM's proposal. The modified S/360 was seen as too different from the rest of the S/360 line; MIT did not want to use a customized or special-purpose computer for MULTICS, but sought hardware that would be widely available. GE was prepared to make a large commitment to time-sharing, while IBM was seen as obstructive. Bell Laboratories, another important IBM customer, soon made the same decision, and rejected the S/360 for time-sharing.

Further information: IBM System/360

1964–67: CP-40, S/360-67, and TSS

The loss of Project MAC was devastating for CSC, which essentially lost its reason for existence. Rasmussen remained committed to time-sharing, and wanted to earn back the confidence of MIT and other researchers. To do this, he made a bold decision: The now-idle CSC team would build a time-sharing operating system for the S/360. Robert Creasy left Project MAC to lead the CSC team, which promptly began work on what was to become CP-40, the first successful virtual machine operating system based on fully virtualized hardware.[4]

IBM's loss of Project MAC and Bell Laboratories caused repercussions elsewhere at IBM.

CSC soon submitted a successful proposal to MIT's Lincoln Laboratory for a S/360-67, marking an improvement in IBM's credibility at MIT. By committing to a "real" time-sharing product, rather than a customized RPQ system, IBM was showing the kind of commitment MIT found with GE.

CSC also continued work on CP-40, ostensibly to provide research input to the S/360-67 team — but also because the CSC team had grown skeptical about the TSS project, which faced a very aggressive schedule and lofty goals. Since a S/360-67 would not be available to CSC for some time, the team conceived an ingenious stopgap measure: building their own virtual-memory S/360. They designed a set of custom hardware and microcode changes that could be implemented on a S/360-40, providing a comparable platform capable of supporting CP-40's virtual machine architecture. Actual CP-40 and CMS development began in mid 1965, even before the arrival of their modified S/360-40. First production use of CP-40 was in January 1967.

The TSS project, in the meantime, was running late and struggling with problems. CSC personnel became increasingly convinced that CP/CMS was the correct architecture for S/360 time-sharing.

Further information: CP-40 historical notes, S/360-67, and TSS/360

1967–68: CP-67

In September 1966, CSC staff began the conversion of CP-40 and CMS to run on the S/360-67. CP-67 was a significant reimplementation of CP-40; Varian reports that the design was "generalized substantially, to allow a variable number of virtual machines, with larger virtual memories", that new data structures replaced "the control blocks describing the virtual machines [which] had been a hard-coded part of the nucleus", that CP-67 added "the concept of free storage, so that control blocks could be allocated dynamically", and that "the inter-module linkage was also reworked, and the code was made re-entrant." Since CSC's -67 would not arrive for some time, CSC further modified the microcode on its own customized S/360-40 to simulate the S/360-67 particularly its different approach to virtual memory.[5] CSC repeatedly and successfully used simulation to work around the absence of hardware: when waiting for its modified S/360-40, for its S/360-67, and later for the first S/370 prototypes. This can be seen as a logical outgrowth of "virtual machine" thinking. During this period, early testing of CP-67 was also done at sites where S/360-67 hardware was available notably IBM's Yorktown Heights lab and MIT's Lincoln Laboratory.

Observers of CP-67 at Lincoln Labs, already frustrated with severe TSS problems, were very impressed by CP-67. They insisted that IBM provide them a copy of CP/CMS. According to Varian, this demand "rocked the whole company", which had invested so heavily in TSS. However, because the site was so critical, IBM complied. Varian and others speculate that this chain of events could have been "engineered" by Rasmussen, as a "subterfuge" to motivate IBM's continued funding CSC's work on the "counter-strategic" CP/CMS, which he "was told several times to kill".[6]

By April 1967 just a few months after CP-40 went into production CP/CMS was already in daily use at Lincoln Labs. Lincoln Lab personnel worked closely with CSC in improving CP/CMS. They "began enhancing CP and CMS as soon as they were delivered. The Lincoln and Cambridge people worked together closely and exchanged code on a regular basis", beginning a tradition of code sharing and mutual support that would continue for years. At around the same time, Union Carbide, another influential IBM customer, followed the same path deciding to run CP/CMS, to send personnel to work with CSC, and to contribute to the CP/CMS development effort.[7]

CP-40, CP-67, and CMS were essentially research systems at the time, built away from IBM's mainstream product organizations, with active involvement of outside researchers. Experimenting was both an important goal and a constant activity. Robert Creasy, the CP-40 project leader, later wrote:

The design of CP/CMS [was] by a small and varied software research and development group for its own use and support… [and] for experimenting with time-sharing system design.... Schedules and budgets, plans and performance goals did not have to be met.… We also expected to redo the system at least once after we got it going. For most of the group, it was meant to be a learning experience. Efficiency was specifically excluded as a software design goal, although it was always considered. We did not know if the system would be of practical use.... In January, 1965, after starting work on the system, it became apparent from presentations to outside groups that the system would be controversial.[8]

TSS, in the meantime, described by Varian as an "elegant and very ambitious system," was exhibiting "serious stability and performance problems, for it had been snatched from its nest too young."[9] In February 1968, at the time of SHARE 30, there were eighteen S/360-67 sites attempting to run TSS. During the conference, IBM announced via "blue letter" that TSS was being decommitted — a great blow to the time-sharing community. This decision would be temporarily reversed, and TSS would not be scrapped until 1971. However, CP/CMS soon began gaining attention as a viable alternative.

Further information: CP-67

1968–72: CP/CMS releases

At the time of the S/360-67, software was "bundled" into computer hardware purchases; see "IBM's unbundling of software and services". In particular, IBM operating systems were available without additional charge to IBM customers. CP/CMS was unusual in that it was delivered as unsupported Type-III software in source code form meaning that CP/CMS sites ran an unsupported operating system. The need for self-support and community support helped lead to the creation of a strong S/360-67 and CP/CMS user communities, precursors to the open source movement.

In the summer of 1970, the CP/CMS team had begun work on a System/370 version of CP/CMS; this would become VM/370. CP-370 proved vital to the S/370 project, by providing a usable simulation of a S/370 on S/360-67 hardware a reprise of CSC's earlier hardware simulation strategies. This approach enabled S/370 development and testing before S/370 hardware was available. A shortage of prototype S/370s was causing critical delays for the MVS project, in particular. This remarkable technical feat transformed MVS development, won an award for the CP-370 developers, and probably rescued the CP project from extinction, despite aggressive efforts to cancel the project.

August 1972 marked the end of CP/CMS, with IBM's "System/370 Advanced Function" announcement. This included: the new S/370-158 and -168; address relocation hardware on all S/370s; and four new operating systems: DOS/VS (DOS with virtual storage), OS/VS1 (OS/MFT with virtual storage), OS/VS2 (OS/MVT with virtual storage, which would grow into SVS and MVS), and VM/370 — the re-implemented CP/CMS. By this time the VM and CP/CMS development team had swelled to 110 people, including documenters. VM/370 was now a real IBM system, no longer part of the IBM Type-III Library. Source code distribution continued for several releases, however; see CP/CMS as free software for details.

Further information: VM/370 and System/370

1968–86?: VP/CSS

Main articles: VP/CSS and National CSS

In 1968, the principals of a small consulting firm in Connecticut called Computer Software Systems had the audacious idea of leasing an IBM System/360-67 to run CP/CMS and reselling computer time. It was audacious because IBM would not typically lease its $50–100K/month systems to a two-person startup. The third and fourth employees were Dick Orenstein, one of the authors of CTSS, and Dick Bayles, from CSC the primary architect of CP-67. Other key hires from the CP/CMS world included Harold Feinleib, Mike Field, and Robert Jesurum (Bob Jay).

By late November 1968, having persuaded IBM to accept the order (no small feat) and secured initial funding, CSS took delivery on their first S/360-67. They began selling time in December 1968.

CSS quickly discovered that, by selling every available virtual machine minute at published prices, they could barely take in enough revenue to pay the machine lease. A whirlwind development program followed, ramping up CP/CMS performance to the point where it could be resold profitably. This began a fork of CP/CMS source code that evolved independently for some fifteen years. This operating system was soon renamed VP/CSS; the company went public, and was renamed National CSS.

Although VP/CSS shared much with its CP/CMS parent and its VM/370 sibling, it diverged from them in many important ways. For business reasons, the system had to run at a profit; and its users, if frustrated, could stop paying at any time simply by hanging up the phone. These forces gave a high priority to factors affecting performance, usability, and customer support. VP/CSS soon became known for routinely supporting two to three times as many interactive users as on comparable VM systems.

Early NCSS enhancements involved such areas as page migration, dispatching, file system, device support, and efficient fast-path hypervisor functions accessed via the diagnose (DIAG) instruction. Later features included a packet-switched network, FILEDEF-level (pipe) interprocess/intermachine communication, and database integration. Similar features also appeared in the VM implementation. Ultimately, the NCSS development team rivaled the size of IBM's, implementing a wide array of features. The VP/CSS platform remained in use through at least the mid-80s. NCSS was acquired by Dun & Bradstreet in 1979; renamed DBCS (Dun & Bradstreet Computer Services); increased its focus on the NOMAD product; changed its business strategy to embrace VM and other platforms; and in the process discontinued support and development of VP/CSS, probably the last non-VM fork of CP/CMS.

1964?–72?: IDC's use of CP/CMS

Interactive Data Corporation (IDC) followed a plan similar to that of National CSS, selling time-sharing services based on the CP/CMS platform. Its focus at the time was in financial services. Varian reports that IDC had "several" S/360-67 systems running CP/CMS and one of IBM's "first relocating S/370", presumably referring to the S/370-145 of 1971, with the first DAT box; but perhaps to the systems announced in 1972 with the "System/370 Advanced Function" announcement, including the S/370-158 and -168.[10]

[Further details and citations are sought on the history of IDC and CP/CMS time-sharing.]

Historical notes

The following notes provide brief quotes, primarily from Pugh and Varian [see references], illustrating the development context of CP/CMS. Direct quotes rather than paraphrases are provided here, because the authors' perspectives color their interpretations.

See also

References

Primary CP/CMS sources

  • R. J. Creasy, "The origin of the VM/370 time-sharing system", IBM Journal of Research & Development, Vol. 25, No. 5 (September 1981), pp. 483–90, PDF
    ― perspective on CP/CMS and VM history by the CP-40 project lead, also a CTSS author
  • E.W. Pugh, L.R. Johnson, and John H. Palmer, IBM's 360 and early 370 systems, MIT Press, Cambridge MA and London, ISBN 0-262-16123-0
    ― extensive (819 pp.) treatment of IBM's offerings during this period; the limited coverage of CP/CMS in such a definitive work is telling
  • Melinda Varian, VM and the VM community, past present, and future, SHARE 89 Sessions 9059–61, 1997;
    ― an outstanding source for CP/CMS and VM history

Additional CP/CMS sources

  • R. J. Adair, R. U. Bayles, L. W. Comeau, and R. J. Creasy, A Virtual Machine System for the 360/40, IBM Corporation, Cambridge Scientific Center Report No. 320‐2007 (May 1966)
    ― a seminal paper describing implementation of the virtual machine concept, with descriptions of the customized CSC S/360-40 and the CP-40 design
  • International Business Machines Corporation, CP-67/CMS, Program 360D-05.2.005, IBM Program Information Department (June 1969)
    ― IBM's reference manual
  • R. A. Meyer and L. H. Seawright, "A virtual machine time-sharing system," IBM Systems Journal, Vol. 9, No. 3, pp. 199–218 (September 1970)
    ― describes the CP-67/CMS system, outlining features and applications
  • R. P. Parmelee, T. I. Peterson, C. C. Tillman, and D. J. Hatfield, "Virtual storage and virtual machine concepts," IBM Systems Journal, Vol. 11, No. 2 (June 1972)

Background CP/CMS sources

  • F. J. Corbató, et al., The Compatible Time-Sharing System, A Programmer’s Guide, M.I.T. Press, 1963
  • F. J. Corbató, M. Merwin-Daggett, and R. C. Daley, "An Experimental Time-sharing System," Proc. Spring Joint Computer Conference (AFIPS) 21, pp. 335–44 (1962) — description of CTSS
  • F. J. Corbató and V. A. Vyssotsky, "Introduction and Overview of the MULTICS System", Proc. Fall Joint Computer Conference (AFIPS) 27, pp. 185–96 (1965)
  • P. J. Denning, "Virtual Memory", Computing Surveys Vol. 2, pp. 153–89 (1970)
  • J. B. Dennis, "Segmentation and the Design of Multi-Programmed Computer Systems," JACM Vol. 12, pp. 589–602 (1965)
    ― virtual memory requirements for Project MAC, destined for GE 645
  • C. A. R. Hoare and R. H. Perrott, Eds., Operating Systems Techniques, Academic Press, Inc., New York (1972)
  • T. Kilburn, D. B. G. Edwards, M. J. Lanigan, and F. H. Sumner, "One-Level Storage System", IRE Trans. Electron. Computers EC-11, pp. 223–35 (1962)
    ― Manchester/Ferranti Atlas
  • R. A. Nelson, "Mapping Devices and the M44 Data Processing System," Research Report RC 1303, IBM Thomas J. Watson Research Center (1964)
    ― about the IBM M44/44X
  • R. P. Parmelee, T. I. Peterson, C. C. Tillman, and D. J. Hatfield, "Virtual Storage and Virtual Machine Concepts", IBM Systems Journal, Vol. 11, pp. 99–130 (1972)

Additional on-line CP/CMS resources

Background information

  • Peter J. Denning, "Performance Modeling: Experimental Computer Science at its Best", Communications of the ACM, President's Letter (November 1981)
    ― Cites the following papers relating to the IBM M44/44X:
  • L. Belady, "A study of replacement algorithms for virtual storage computers," IBM Systems Journal Vol. 5, No. 2 (1966), pp. 78–101.
  • L. Belady and C. J. Kuehner, "Dynamic space sharing in computer systems," Communications of ACM Vol. 12 No. 5 (May 1969), pp. 282–88.
  • L. Belady, R. A. Nelson, and G. S. Shedler, "An anomaly in the space-time characteristics of certain programs running in paging machines," Communications of ACM Vol. 12, No. 6 (June 1969), pp. 349–53.
  • L.W. Comeau, "CP-40, the Origin of VM/370," Proceedings of SEAS AM82 (September 1982)
    ― description of CP-40, cited in Varian [above]
  • Harold Feinleib, "A technical history of National CSS", Computer History Museum (March 2005). PDF.
  • W. O'Neill, "Experience using a time sharing multiprogramming system with dynamic address relocation hardware," Proc. AFIPS Computer Conference 30 (Spring Joint Computer Conference, 1967), pp. 611–21
    ― [Describes the experimental IBM M44/44X, reports performance measurements related to memory and paging, and discusses performance impact of multiprogramming and time-sharing.]
  • Dick Orenstein, "From the very beginning… from my vantage point", Computer History Museum (January 2005). PDF
    ― early history of National CSS
  • D. Sayre, On Virtual Systems, IBM Thomas J. Watson Research Center (April 15, 1966)
    ― An early virtual machine paper, describing multiprogramming with the IBM M44/44X, an experimental paging system.

Citations

  1. Creasy, op. cit., p. 485 — CTSS dates
  2. cf. Ritchie, "The Evolution of the Unix Time-sharing System", AT&T Bell Laboratories Technical Journal, Vol. 63 No. 6 Part 2, October 1984, pp. 1577-93
  3. Comeau, op. cit., p. 40 — Atlas "didn't work"
  4. Varian, op. cit., p. 10, Note 26 — the experimental IBM M44/44X paging system used virtual machines that did not fully virtualize the 7044: "close enough to a virtual machine system to show that 'close enough' did not count"
  5. Varian, op. cit., p. 18 — CP-40 reimplemented as CP-67
  6. Varian, op. cit., pp. 19–20 — subterfuge
  7. Varian, op. cit., pp. 20-21 collaboration on CP/CMS
  8. Creasy, op. cit., p.' 470 — CP/CMS as experiment
  9. Varian, op. cit., p. 17 — TSS "snatched from its nest too young"
  10. Varian, op. cit., pp. 24, Note 76 IDC systems (quoting Dick Bayles)
  11. C. Strachey, "Time Sharing in Large Fast Computers," Proceedings of the International Conference on Information Processing, Paper B.2.19, UNESCO, New York, June, 1959 [cited in Varian, op. cit., p. 3].
  12. Varian, op. cit., pp. 3–4
  13. F.J. Corbató, M.M. Daggett, R.C. Daley, R.J. Creasy, J.D. Hellwig, R.H. Orenstein, and L.K. Korn, The Compatible Time-Sharing System: A Programmer’s Guide, The MIT Press, Cambridge, MA, 1963 [cited in Varian, op. cit., p. 3].
  14. Creasy, op. cit., pp. 485–87 — influence of CTSS.
  15. Pugh et al., op. cit., pp. 355–56.
  16. Varian, op. cit., p. 4.
  17. T.J. Watson, Jr., Father, Son, and Co.: My Life at IBM and Beyond, Bantam Books, New York, 1990, pp. 244–5 [cited in Varian, op. cit., p. 4]
  18. Varian, op. cit., p. 7
  19. Varian, op. cit., p. 6
  20. Pugh et al., op. cit., p. 169
  21. Varian, op. cit., pp. 7, 9
  22. Pugh et al., op. cit., pp. 360–61
  23. Creasy, op. cit., p. 486 — CSC view of S/360
  24. Pugh et al., op. cit., pp. 360–61.
  25. Varian, op. cit., pp. 8, 17
  26. Varian, op. cit., pp. 14–5.
  27. J. J. Glauthier, "Computer time sharing, its origins and development," Computers and automation, October 1967, pp. 23–7.
  28. Pugh et al., op. cit., pp. 595–97.
  29. Pugh et al., op. cit., pp. 741–42 (note 219 to Ch. 6)
  30. Pugh et al., op. cit., p. 598
  31. Pugh et al., op. cit., p. 633
  32. Creasy, op. cit., p. 489 — disappointed expectation of building an OS incubator

Family tree

 CTSS 
> IBM M44/44X
>> CP-40/CMS CP[-67]/CMS  VM/370 → VM/SE versions → VM/SP versions → VM/XA versions → VM/ESAz/VM
VP/CSS
> TSS/360
> TSO for MVT → for OS/VS2 → for MVS → ... → for z/OS
>> MULTICS and most other time-sharing platforms
This article is issued from Wikipedia - version of the 7/31/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.