LII:Are You a Laboratory Automation Engineer?
Title: Are You a Laboratory Automation Engineer?
Author for citation: Joe Liscouski, with editorial modifications by Shawn Douglas
License for content: Creative Commons Attribution 4.0 International
Publication date: 2006
The technology and techniques that are used in automating laboratory work have been documented for over 40 years. Work done under the heading of “laboratory automation” has progressed from pioneering work in data acquisition and instrument control, to instrumentation with integral computing and communications. If the field is to move forward, we need to organize the practices and education of those applying automation and computing technologies to laboratory work. This note[a] is an opening to a dialog in the definition and development of the field of "laboratory automation engineering” (LAE).
In this article the following points will be considered:
- Definition of laboratory automation engineering (LAE)
- Laboratory automation as an engineering discipline
- Developments in laboratory automation
- Benefits of formalizing laboratory automation engineering
- Sub-disciplines within LAE
- Skills required for LAE
- What comes next?
There are many of us working on the use of automation technologies in scientific disciplines such as chemistry, high-throughput screening, physics, quality control, electronics, oceanography, and materials testing. Yet, given the diversity of applications, we have many characteristics in common. There are enough common traits and skills that we can declare a field of laboratory automation engineering (LAE). The establishment of this field of work benefits both the practitioner and those in need of their skills. In addition, we may finally come to the full realization of the rewards we expect from automation systems.
Laboratory automation engineering can be defined as the application of automation, information, computing, and networking technologies to solve problems in, or improve the practice of, a scientific discipline. As such, it fits the accepted definition of "engineering": the discipline dealing with the art or science of applying scientific knowledge to practical problems.
The common characteristics of LAE are:
- It is practiced in facilities where the focus is scientific in nature, including research and development, testing, and quality control activities.
- The end product of the work consists of information (e.g., initial and processed test data) and/or systems for managing and improving people’s ability to work with information (intermediate steps result in prepared samples or specimens).
- The automation practice draws upon robotics, mechanical and electrical engineering, data and information handling and management, information and networking technology, and the science that the automation is addressing.
- It requires the practitioners to be knowledgeable in both automation technologies and the discipline in which they are being applied.
LAE differs from other forms of automation engineering found in manufacturing and product production in several ways. First, modern manufacturing and production facilities are designed with automation as an integral part of the process. Laboratory automation is a replacement process: manual techniques, once they mature, are replaced with automated systems. Economics, and the need to improve process uniformity, drive the replacement. If a testing or quality control lab could be designed initially as a fully automated facility, it would be difficult to distinguish the automated facility from any other manufacturing facility aside from the point that its “product” is data rather than tangible materials. The second difference is the point just made: the results are ultimately data.[b] LAE would result in systems and methods (e.g., high-throughput screening and combinatorial methods) that enable science to be pursued in ways that could not be done without automation.
Developments in laboratory automation
Laboratory automation was initiated by scientists with a need to do things differently. Whether it was to improve the efficiency of the work, to substitute intelligent control and acquisition systems for manual labor, or because it was the best or only way to implement an experimental system, automation eventually moved into the laboratory. Those doing the work had to be as well-versed in computing hardware and software as they were in their science. The choice of hardware and software, as well the development of the system, was in their hands.
The first generation of laboratory automation systems was concerned with interfacing instrument, sensors, and controllers to computers. Then the control of experiments through robotics and direct interfacing with computer systems arrived. A variety of experimental control software packages became available, as did laboratory information management systems (LIMS) and, in the last few years, electronic laboratory notebooks (ELNs).
If you have visited a conference in any discipline with a discussion on automation, it would appear that automation has seriously taken hold and most things are already automated or soon will be. One problem with this product array is the ability to integrate products from different vendors. Individual procedures are being addressed, but the movement of data and information within the lab and your ability to work with it is not as robust as it could be. Some vendors answer this need with comprehensive ELN systems in which their products are the hub of a lab's operations. Data is collected from instruments with local computing capability (e.g., the simple data translation of balances or similar devices) into their systems for storage, cataloging, analysis, etc. The movement is primarily one-way, but this is only the current generation of systems. The capabilities of today’s products generate new possibilities requiring upgrades to, or replacement of, these products.
The early automation models were driven by limitations in computing power, data storage, and communications. There was one experimental setup, with one system to provide automation. The next generation of laboratory automation has to replace previous models, and develop implementations that can be gracefully upgraded rather than replaced. This will require increased modularity and less dependence upon local-to-the-instrument systems for automation. Achieving it requires engineering systems, not just building products.
Today, data communication is readily available thanks to the development of communication standards. Storage is cheap and there is an abundance of computing capability available. Data analysis does not have to be done at the instrument. Instead, the data file with instrument parameters can be sent to a central system for cataloging, storage, analysis, and reporting, while still making the raw data readily available for further analysis as new data treatment techniques are developed. The dots in the automation map will be connected with bi-directional communications. The field needs people who are trained to do the work and to take advantage of the increased capabilities available to them.
Beyond the issues of experimental work lay the requirements of corporate information architectures. Laboratory systems are part of the corporate network, whether it is a for-profit company, a university, or a research organization. Laboratory computing does not stop at the door. The purpose of a laboratory is to create knowledge, information, and data. The purpose of laboratory automation is to assist in that development, making the process more successful, effective, and cost-efficient. This can be accomplished by engineering systems that have security built-in to protect them, while at the same time giving scientists access to their data wherever they need it. It also means designing systems that permit a graceful evolution and continual improvement without disrupting the lab workings.
Too often we find corporate information technology (IT) departments at odds with those using computers in laboratory settings. On one side you hear “IT doesn’t understand our needs,” and on the other “we’re responsible for all corporate computing.” Both sides have a point. IT departments are responsible for corporate computing, and labs are part of the corporation. However, IT departments usually do not understand the computing needs of scientists; they do not speak the same language. IT-speak and science-speak do not readily connect and the issue will worsen as more instruments become network-ready. Specialized software packages used in laboratory work raise support issues for IT departments. IT would be concerned about who was going to support these programs and the effect they may have on the security of the corporation’s information network. LAE’s have the ability to bridge that gap.
Benefits of formalizing laboratory automation engineering
People have been successful for several decades doing lab automation work, so why do we need a change in the way it is done? While there certainly have been successful projects, there have also been projects that have ranged from less-than-complete successes, or even cancellations. In the end, the potential for laboratory automation to assist or even revolutionize laboratory work is too great to have projects be hit-or-miss successes. Even initially apparent successes may lead to long-term problems when the choice of software platform leads to an eventual dead-end, or the lab finds itself locked into a product set that does not give it the flexibility needed as lab requirements change over time.
To date, laboratory automation has moved along a developmental path that has been traveled in other areas of technology applications. In commercial chemical synthesis, aerospace, computers, civil engineering, and other fields, progress has been made first by individuals, followed by small groups practicing within a particular field. As the need to apply these technologies on a broader scale grew, training became formalized, as did methods of design and development. The methodology shifted from an “art” to actual engineering. The end result has been the ability to create buildings, bridges, aircraft, etc. on a scale that independent groups working on their own could not hope to achieve.
In the fields noted in the previous paragraph, and in others, “engineering” a project means that trained people have analyzed, understood, and defined it. It also means that plans have been laid out to meet the project goals, and risks have been identified and evaluated. It is a deliberate, confident movement from requirements to completion. This is what is needed to advance the application of automation technologies to scientific and laboratory problems. Hence, a movement to an engineering disciple for laboratory automation is necessary.
The benefits[c] of formalizing LAE to the individual practitioner, laboratory management, and the field are outlined below.
Benefits to the individual:
- provides a thorough and systematic education in a broadly practiced field;
- informs of past activities regarding what has worked and what has failed;
- provides evidence of relevant knowledge and training through a degree or certificate program; and
- lends a sense of identity and community with others on the same career path.
Benefits to laboratory management:
- provides a basis for evaluating people’s credentials and ability to work on laboratory automation projects;
- provides a basis for employee evaluation;
- limits the need for on-the-job training, reducing the impact on budgets;
- lends to faster project implementation with a higher expectation of success; and
- lends expertise to the design of laboratories using automation as part of their initial blueprint, rather than using it to replace existing manual procedures, while reducing the cost and improving the efficiency of a laboratory’s operations.
Benefits to the field of laboratory automation:
- creates a foundation of documented knowledge that LAEs can use for learning and to improve the effectiveness of the profession;
- encourages a community of people that can drive the organized development of systems and technologies that will provide advancements to the practice of science, while creating re-useable resources instead of re-inventing systems from scratch on similar projects;
- provides a basis for research into new technologies that can significantly improve scientist’s ability to do their work, encouring a move from incremental advancement in automation systems to leaps in advancement; and
- promotes a community of like-minded individuals that can discuss, and where appropriate, develop positions on key issues in the field (e.g., the impact of regulatory requirements and standards development) and develop position papers and guidelines on those points as warranted.
Sub-disciplines within LAE
Laboratory automation has the ability to allow us to do science that would be both physically and economically prohibitive without it. As noted above, high-throughput screening and combinatorial methods depend on automation. In physics, the data rates would be impossible to handle. In most industrial analytical chemistry laboratories, automation keeps budgets and testing schedules workable. As the discipline develops, we are going to become more dependent on automation systems for data access, data mining, and visualization.
Given the level of sophistication of existing systems, and the demands that will be placed on them in the future, there will be a need for specialization, even within the field of LAE. Sub-disciplines within LAE include:
- Sample handling, experiments, and testing – Automation is applied to the movement, manipulation, and analysis of samples and objects of interest. A common type of automation is robotics, which includes special-purpose devices such as autosamplers, as well as user-defined configurable systems.
- Data acquisition and analysis – This includes any sensor-based data entry, with subsequent data evaluation.
- Data, information, and knowledge management - This includes managing the access to and storage of thos objects, as well as understanding the uses of LIMS and ELN.
These sub-disciplines are not mutually exclusive but have considerable overlap (Figure 1) and share underlying technologies such as computing (which includes programming), networking, and communications. In some applications it may be difficult to separate “data acquisition” from experiment control; however, that separation could lead to insights in system design. The sub-disciplines also have distinctly different sets of skill requirements and, as you move from left to right in the Figure 1, less involvement with the actual laboratory science.
A systems approach to laboratory automation is essential in the implementation of projects in modern laboratories; without it, we are going to continually repeat the “islands of automation” practices so common in the past. Project designers must look beyond the immediate needs and make provision for integration with other systems, in particular bi-directional communication with knowledge, information, and data management systems.
Skills required for LAE
In the mid-1960s, “programming” instruments was a mechanical problem handled with needle-nosed pliers, screwdrivers, and a stopwatch, as you adjusted cams and micro-switches.[d] This process is more complicated today. Automation systems are built around computer components, therefore, a strong background in computer science and programming would be necessary to be successful. In addition, an engineering background along with management, regulatory, and strong organizational skills would be expected. An understanding of the basic science in which automation is being applied is also crucial. As noted below under communications, it is up to the LAE to understand the science and the scientists in order to convert their needs into a functional requirements document and then implement working systems.
Those working in LAE should have the following basic skills in their backgrounds. These would be common across all sub-disciplines:
- project and change management
- strong communications skills
- strong understanding of regulatory issues (e.g. FDA, ISO)
- general systems theory and beyond
- process analysis and development
Project and change management
In addition to the skills you might expect to be covered by project management, the topic of change management also deserves to be noted. Change management is included to emphasize their mutual importance. Until we get to the point where we can anticipate automation issues within a lab and build them into the lab as it is constructed, laboratory automation is going to be both a replacement for an existing process (a project) and a new set of processes imposed on those working in that lab (some change). Even when we get to a stage where pre-built automation systems exist, there will be both step-wise and radical changes in the applied technologies as new techniques are developed.
The LAE practitioner will have to deal with the normal issues of budgets, schedules, and documentation common to projects. The need for thorough documentation cannot be stressed enough. Well-written documentation of the project goals, structure, milestones, the reasoning behind choices of materials, equipment, alternative ways of doing things, and final acceptance criteria are necessary to provide a basis for regulatory review. This will lead to the development of a supportable system that can be upgraded, and provide an objective reference for determining when the project is completed. As part of their practice, LAEs should adopt practices in other engineering disciplines such as the development of a functional requirements specification (FRS), user requirements specification (URS), and design specifications prior to system development. The LAE should also be familiar with the concept of project life cycle management found in a number of industries, paying particular attention to software life cycle management.
More interestingly, the LAE will have to be able to respond to statements like “I know we’ve never done anything like this before but I still need to know how much it will cost and when it will be done” and questions like “ok, maybe you don’t know why it isn’t working, but can you tell me when it will be fixed?”. The politics of managing changes, adding requirements to projects (with appropriate documentation, change orders, budget, and scheduling adjustment) are part of professional work.
Change management also addresses “people issues,” and they can be among the more stressful. Moving from an existing process of doing work to an automated one means that people’s jobs will change. For some it will be a minor adjustment to the way they do work, while for others it will be the possibility of their job dramatically changing (and change raises people’s anxiety levels). Those working in LAE are going to have to be able to face that reality head-on because that will affect their ability to get work done. If those working in the lab believe that your project will cause them to lose their job, or have to make a significant change (particularly if it is viewed as a negative change) they may limit your ability to complete the project. The fact that this point deals with people does not mean that it cannot affect what may be viewed as a technology project, or that the issue can be viewed as a problem to be solved by lab managers.
Paying attention to “people issues” can:
- point to deficiencies in project planning,
- avoid delays in the final acceptance of the project, and
- help you uncover important factors in designing a system.
Regarding the first point, one of the lessons learned in the installation of early LIMS was the need for both training and the availability of practice databases so that those working with the system could familiarize themselves with a non-production database before using the live system. Having the project plan reviewed by those in the lab who will be affected by the system can lead to the discovery of issues that need to be addressed in training, scheduling, and potential conflicts in the lab’s operations. By extension, if deficiencies are caught early, delays in final project acceptance may be avoided.
As for the final point, laboratory work contains elements of art as well as science. People’s familiarity with the samples they are processing can lead to undocumented shortcuts and adjustments to procedures, which can make the difference between your implementation of an automated process working or failing. In the late 1980s, Shoshana Zuboff documented the problems that occurred in the automation of pulp mills when the plant personnel’s experience was ignored. The parallel here is if you ignore peoples experience in doing the work you are about to automate, you run a significant risk of failing.
Project and change management also has to provide planning for the stresses that develop during a project's implementation. As noted earlier, laboratory automation today is largely a replacement of existing manual procedures. LIMS are installed to facilitate the bookkeeping of service laboratories, improve the lab’s ability to respond to questions about test results, and increase the efficiency of the lab’s operations. Robotic sample preparation systems are introduced into a lab because the existing manual processes are no longer economical, and there is a need for better data precision, or higher sample throughput.
These replacement activities are going to introduce stressors on the lab, which need to be taken into account in the planning process. First, there will be some stress on the lab workers and budget during the development of the system. That stress will come in the form of the cost of running the lab while the development is taking place, people’s attitudes if they feel their jobs will be adversely affected, potential disruption of lab operations, and the problems associated with installing systems in labs that are usually short on space. Second, once the automation system has been developed, it will have to be evaluated against the existing process to show that it meets the design criteria. Once that has been successfully accomplished, a change-over process has to be designed to switch from one method of operation to another. Both of these activities will increase the workload in the lab since, for a time, both the automated system and the process it is replacing will run in parallel.
Strong communications skills
The previous section mentioned managing people issues, and effective communication is part of that. The strong communication is also useful in making sure people understand what work is being done, what the benefits are, and what the costs are, as well as declaring any schedules, risks, etc. Those aspects, as well as the ramifications of any changes, have to be clearly understood. The lack of clear communications is often the basis of misdirection, delays, and frustration around projects. Communication is a two-way street. The LAE needs to be able to discuss the project, as well as listen to and understand the needs of those working in the lab.
Strong understanding of regulatory issues
No matter what industry in which you are working, meeting regulatory requirements is a fact of life. Companies are under increasing pressure from regulatory agencies to have their internal practices conform to a standard. This is no longer a specific regulatory- or standards-based issue that is focused on manufacturing systems, but rather an organization-wide set of activities encompassing financial and accounting practices, human resources, etc. Name a department and it will be affected by a regulation.
While the initial reaction is that this adds one more thing to do; for the most part, what the regulations require is part of a good system design process. The requirements for the validation of a system, for example, call for the designer to show that the system works, that it is supportable, well-documented, and uses equipment suited to the designer’s purpose and from reliable vendors. Tinkering in order to get a system together is no longer an accepted practice; it may work for prototyping, but not for production use. The point of any applicable regulations and standards is to ensure that any implemented system can stand up to long-term use, and that it fits a well-defined, documented need. In short, regulations and standards help ensure a system is well-engineered.
General systems theory and beyond
Frank Zenie (one of the founders of Zymark Corporation[e]) usually made the statement that “you can only automate a process, you can’t automate a thing” as part of his introductory comments for Zymark’s robotics courses. Recognizing that a process exists and that it can be automated is the initial step in defining a project. General systems theory[f] provides the tools for documenting a process, as well as the triggers and the state changes that occur as the process develops. It is particularly useful in working with inter-related systems.
Recognizing that a process exists and can be well described is one part of the problem. Another is its ability to be automated. Labs and lab equipment are designed for people. Using that same equipment and tools with automated systems, including robotics, may require a significant amount of re-engineering, and bring into question the economics and perceived benefits of a project.
Process engineering should also include statistical process control, statistical quality control, and productivity measurements. Robotics systems are small-scale manufacturing systems even if the end result is data. As portions of laboratory work become automated and integrated, the systems will behave like manufacturing processes and should fall under the same types of control. Productivity measurements are also important. Automation systems are put in place to achieve a number of goals, including a focus on improving efficiency of operations. Productivity measurements test the system’s ability to meet those goals, and if successful, will lead to the development of additional projects.
Much of what has been written so far could easily be recognized as part of an engineering or engineering management program, and that is exactly the point: laboratory automation is an engineering activity. The elements that differentiate it from other engineering activities include that:
- it's a laboratory or scientific environment, as noted earlier;
- automation is an enabling technology opening the door to new scientific methodologies, including discovery-based science[g];
- automation is usually a replacement for existing manual operations, which will require the LAE to be sensitive to change management issues;
- the scope of activities can include materials handling (robotics), data acquisition, analysis, reporting, and integration with database systems; and
- LAEs need to be knowledgeable in automation technologies and their application, and have strong backgrounds in science practiced in the lab.
This last point is significant; the LAE cannot expect the lab personnel to describe their requirements in engineering terms. Yes, lab personnel can explain what they would like to accomplish; however, it is the LAE’s responsibility to determine how to do it and understand the ramifications in system design and implementation. The LAE functions in part as a translator, converting a set of needs of scientists into a project plan, and ultimately into a functioning system. In some respects this is similar to traditional software engineering.
All the work associated with the three sub-disciplines reflects the transition of working with “things” and materials, making measurements, and creating a description of those items, and then managing and working with the resulting knowledge, information, and data. The lists found in Figure 2 are a summary of skills needed in addition to those noted above. Software engineering is repeated in the lists for additional emphasis.
What comes next?
There are two major tasks that need to be undertaken, both under the heading of education. First is the development of a curriculum for laboratory automation engineering at the university level. This will require support from both organizations like the Association for Laboratory Automation (ALA) and from industry. A university is not going to create a program, even one that can be assembled to a large extent from existing programs, unless it has some assurance that its graduates will find jobs. The availability of employment opportunities would be used to attract students, and the process would move on from there. One issue that needs to be addressed is how much laboratory science knowledge an LAE has to have in order to be effective. Another closely tied issue is whether we are talking about an undergraduate program with particular science specializations or a graduate program.
Undergraduate and graduate science programs need to address the inclusion of automation in their courses. The point is not to teach them engineering but to acquaint them with how the work of modern science is done. Just as computer literacy is part of high-school curriculums, automation literacy should be part of the course of science studies. James Sterling’s 2004 journal article on laboratory automation curriculum provides one example of what can be done. The ALA can take a lead and develop materials (e.g., course outlines, source material, etc.) to assist instructors who want to include this material in their programs.
While a university could address a long-term need, there is the need to provide a means of having those already working in the field to augment their backgrounds. Expanding the short-course structure already put in place by the ALA could satisfy that need. Another possibility is the development of certification programs similar to those used in computer science, working in conjunction with short- courses and the development of an “Institute for Laboratory Automation” with an organized summer program.
The second task is the organization of a “body of knowledge” about laboratory automation engineering that would include compilations of relevant texts, web sites, knowledge bases, etc., while encouraging those working in the field to contribute material to its development. The initial step would be the development of a framework to organize material and then to begin populating it. Once a framework is in place, publications like the Journal of the Association for Laboratory Automation could have authors key their papers to that structure. The details of the framework need input from others in the field and more depth of evaluation than this introductory piece can afford.
In the development of this framework (Figure 3), the following points should be considered. First, there are fundamental techniques, skills, and technologies in laboratory automation that cut across scientific disciplines pretty much intact, including analog data acquisition, fundamentals of robotics, interfacing techniques, project management, etc. While there may be application-specific caveats (e.g., “Considerations in analog interfacing in secure biohazard facilities”), for the most part they are common elements and can be treated as a common block of knowledge. Second, once we get beyond basics and into applications of techniques in scientific disciplines, we may see the same basic outline structure duplicated with parallel sections. Robotics in chemistry may be similar to robotics in another discipline but with differences in equipment details and implementations. The outline will appear similar, with differences in the details. The framework should consider linkages between parallel outlines where similar needs generate similar systems development. In short, we should make it easy to recognize cross-fertilization between similar technologies in different disciplines.
Laboratory automation is still in the early phases of its development. Some may say that significant progress has been made, but in comparison to the rate of development of automation and information technologies in other areas, our progress has been slow and incremental (the internet, for example, is a little more than a decade old).
What can and must be done is getting the user community to embrace the establishment and development of a discipline focused on envisioning, creating, and improving the tools and techniques of laboratory automation. That in turn will allow us to realize the promise that proponents of automation have long held: giving people the opportunity to do better science. Laboratory automation needs to be driven by people who want to do good work and are trained to do it.
An LAE’s employment will be driven by market demand. However, the skill sets should be transferable. Just as computer science professionals have flexibility in where their skills are applied, LAEs should enjoy the same ability to move from one scientific application to another. The major differences will be learning the underlying science.
Abbreviations, acronyms, and initialisms
ELN: Electronic laboratory notebook
FDA: Food and Drug Administration
FRS: Functional requirements specification
IT: Information technology
ISO: International Organization for Standardization
LAE: Laboratory automation engineering (or engineer)
LIMS: Laboratory information management system
URS: User requirements specification
- A portion of this material was published as a guest editorial in the Journal of the Association for Laboratory Automation (now SLAS Technology), June 2006, volume 11, number 3, pages 157-162.
- As a point of fact, and as noted later in this piece, the results of laboratory work are knowledge, information, and data. “Data” as used here is a short-hand for all three. For a full discussion of this point from the author's point of view, please refer to Laboratory and Scientific Computing: A Strategic Approach, J. Liscouski, Wiley Interscience, 1995.
- I’d like to thank Mark Russo, executive editor of the JALA, for his comments and, in this section in particular, for his insights and the material he contributed.
- For example, process chromatographs made by Mine Safety Appliances and Fisher Control used cams and micro-switches to control sampling and back-flush valves.
- Zymark was acquired by Caliper Life Sciences in 2003.
- See "What Is Systems Theory?" for an introduction. In particular, review the work of George Klir, including An Approach to General Systems Theory and Facets of Systems Science.
- As suggested by a reviewer.
I’d like to thank Mark Russo (Bristol-Myers Squibb) for his comments and support in the development of this article.
Initially educated as a chemist, author Joe Liscouski (joe dot liscouski at gmail dot com) is an experienced laboratory automation/computing professional with over forty years of experience in the field, including the design and development of automation systems (both custom and commercial systems), LIMS, robotics and data interchange standards. He also consults on the use of computing in laboratory work. He has held symposia on validation and presented technical material and short courses on laboratory automation and computing in the U.S., Europe, and Japan. He has worked/consulted in pharmaceutical, biotech, polymer, medical, and government laboratories. His current work centers on working with companies to establish planning programs for lab systems, developing effective support groups, and helping people with the application of automation and information technologies in research and quality control environments.
- Hallock, N. (2005). "Creative Combustion: A History of the Association for Laboratory Automation". SLAS Technology 10 (6): 423–31. doi:10.1016/j.jala.2005.10.001.
- Liscouski, J. (1985). "Laboratory Automation". JCOMP 25 (3): 288–92.
- Matey, J.R. (1999). "History of Laboratory Automation - Session BA01.05, Cent. Symposium: 20th Century Developments in Instrumentation & Measurements". Centennial Meeting of the American Physical Society. http://flux.aps.org/meetings/YR99/CENT99/abs/S350.html#SBA01.001.
- "Project Management Life Cycle". Method123. https://www.method123.com/project-lifecycle.php.
- "Systems development life cycle". Wikipedia. https://en.wikipedia.org/wiki/Systems_development_life_cycle.
- "Lesson 2.2 Project Management". Adobe Web Tech Curriculum. 2005. Archived from the original on 10 January 2006. https://web.archive.org/web/20060110072101/http://www.adobe.com/education/webtech/CS/unit_planning1/pm_home_id.htm.
- Zuboff, S. (1988). In the Age of the Smart Machine. Butterworth–Heinemann. ISBN 0434924865.
- Sterling, J.D. (2004). "Laboratory Automation Curriculum at Keck Graduate Institute". SLAS Technology 9 (5): 331–35. doi:10.1016/j.jala.2004.07.005.