Book:Choosing and Implementing a Cloud-based Service for Your Laboratory/What is cloud computing?/History and evolution

-----Return to the beginning of this guide-----

1. What is cloud computing?

Figure 1. A basic visualization of cloud computing architecture layers and some of the activities that take place on those layers.

If you were alive in the late 2000s and doing most anything related to computers and the internet, you were bound to encounter the latest internet buzzword: cloud computing. A certain mysticism was seemingly attached to the concept, that your files and applications could reside on the internet, "out there in the 'cloud.'" "But what is this 'cloud'?" many would ask. A plethora of media articles, journal articles, blogs, and company websites were published to give practically everyone's take on what the cloud was and wasn't meant to be. However, the then growing consensus of cloud computing as networked and scalable architecture meant to rapidly provide application and infrastructure services at reasonable prices to internet users largely matches up with today's definition. Pulling from both The Institution of Engineering and Technology and Amazon Web Services, we come up with cloud computing as:

an internet-based computing paradigm in which standardized and virtualized resources are used to rapidly, elastically, and cost-effectively provide a variety of globally available, "always-on" computing services to users on a continuous or as-needed basis

Of course, those computing services come in a variety of flavors, the most common being software, platform, and infrastructure "as a service" (SaaS, PaaS, and IaaS, respectively). These conveniently correspond to the underlying architectural layers of the services, with infrastructure at the base, platform on top of that, and software (or application) on top of that.

Figure 1 portrays a simplified visualization of cloud computing architecture layers, as well as examples of activities that happen on those layers. This concept has also been visualized by others using pyramids and pancake stacks of layers, but the concept remains the same. At the base is the computing infrastructure, including the physical data centers and their networking equipment, servers, hypervisors, application programming interfaces (APIs), and operating systems. This infrastructure is the foundation that supports not only applications users want to run but also that acts as the developmental foundation of users not wanting to implement their own infrastructure. On top of all that can be found platforms or middleware, which serve as software development and deployment environments (that include databases, web servers, load balancers, etc.) or connectivity tools for analytics, workflow management, system integration, and security management. And on top of that are applications, typically designed to run optimally in cloud environments and accessed via web browsers or apps using internet—i.e. networking—connectivity and computing devices.

Customers who require application hosting, internet-hosted software development platforms, or underlying computing infrastructure (e.g., data storage, computational time, etc.)—particularly when they can't or don't want to invest in their own hardware—are increasingly turning to the cloud computing paradigm. Even before a worldwide COVID-19 pandemic started to take shape in late 2019, the global cloud services market was expected to reach $266.4 billion by the end of 2020, with Gartner expecting that to represent a 17 percent increase from 2019. As work-from-home practices expanded significantly in 2020 due to the pandemic, expectations that the trend would last post-pandemic pushed estimates of overall cloud-based workloads moving from physical work offices to the cloud to 55 percent by 2022, with the cloud services market reaching $600 billion in 2023 and $1 trillion by 2030. This growing migration to cloud computing has many implications for organizations of all types, including laboratories.

1.1 History and evolution

Cloud computing has its strongest origins in the "web services" phase of internet development. In November 2000, Mind Electric CEO and distributed computing visionary Graham Glass, writing for IBM, described web services as "building blocks for creating open distributed systems" that "allow companies and individuals to quickly and cheaply make their digital assets available worldwide," while prognosticating that web services "will catalyze a shift from client-server to peer-to-peer architectures." At that point, the likes of Microsoft and IBM were already developing toolkits for creating and deploying web services, with IBM releasing an initial high-level report in May 2001 on IBM's web services architecture approach. In that paper, web services were described by its author Heather Kreger as allowing "companies to reduce the cost of doing e-business, to deploy solutions faster, and to open up new opportunities," while also allowing "applications to be integrated more rapidly, easily, and less expensively than ever before."

Here's a recap of thinking on web services at the turn of the century:

• "[act as] building blocks for creating open distributed systems"
• "quickly and cheaply make ... digital assets available worldwide"
• "catalyze a shift from client-server to peer-to-peer architectures"
• "reduce the cost of doing e-business, to deploy solutions faster, and to open up new opportunities"
• "[allow] applications to be integrated more rapidly, easily, and less expensively than ever before"

We'll come back to that. For the next stop, however, we have to consider the case of Amazon and how they viewed web services at that time. Leading up to the twenty-first century, Amazon was beginning to expand beyond its book selling roots, opening up its marketplace to other third parties (affiliates) to sell their own goods on Amazon's platform. That effort required an expansion of IT infrastructure to support web-scale third-party selling, but as it turned out, a lot of that IT infrastructure, while reliable and cost-effective, had been previously added piecemeal, with many components getting "tangled" along the way. Amazon project leads and external partners were clamoring for better infrastructure services. This required untangling the IT and associated provider data into an internally scalable, centralized infrastructure that allowed for smoother communication and data management using well-documented APIs. By 2003, the company was indirectly acting as a services industry to its partners. "Why not act upon this strength?" was the sentiment that quickly developed that year, with Amazon choosing to use its internal compute, storage, and database infrastructure and related expertise to its advantage.

At that point, the paradigm of web services expanded to include infrastructure as a service or IaaS, with compute, storage, and database services running over the internet for web developers to utilize. "If you believe developers will build applications from scratch using web services as primitive building blocks, then the operating system becomes the internet,” noted AWS CEO Andy Jassy in a 2015 retrospective interview. From that concept evolved the idea of determining what it would take to allow any entity to run their technology applications over their web-service-based IaaS platform. In August 2006, Amazon introduced its Amazon Elastic Compute Cloud (Amazon EC2), "a web service that provides resizable compute capacity in the cloud." This quickly prompted others in academic and scientific fields to continue the conversation of turning IT and its infrastructure into a service. In turn, conversations changed, discussing the opportunities inherent to "cloud computing," including Google and IBM partnering to virtualize computers on new data centers for boosting academic research and teaching new computer science students, IBM releasing a white paper on cloud computing and announcing its Blue Cloud initiative, and Google doubling down on its cloud-based software offerings in competition with Microsoft.

In IBM's 2007 white paper, they described cloud computing as a "pool of virtualized computer resources" that can:

• "host a variety of different workloads, including batch-style back-end jobs and interactive, user-facing applications";
• "allow workloads to be deployed and scaled-out quickly through the rapid provisioning of virtual machines or physical machines";
• "support redundant, self-recovering, highly scalable programming models that allow workloads to recover from many unavoidable hardware/software failures";
• "monitor resource use in real time to enable rebalancing of allocations when needed"; and
• "be a cost efficient model for delivering information services, reducing IT management complexity, promoting innovation, and increasing responsiveness through real-time workload balancing."

In 2011, the National Institute of Standards and Technology (NIST) came up with a more standards-based definition to cloud computing. They described it as "a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction." They went on to highlight the five essential characteristics further:

• On-demand self-service: The unilateral provision of computing resources should be an automatic or nearly automatic process.
• Broad network access: Thin- or thick-client platforms, both hardwired and mobile, should allow for standardized, networkable access to those computing resources.
• Resource pooling: A multi-tenant model requires the provisioning of resources to serve a wide customer base, with a layer of abstraction that gives the user a sense of location independence from those resources.
• Rapid elasticity: The platform's resources should be readily and/or automatically scalable commensurate with demand, such that the user sees no negative impact in their activities.
• Measured service: The resources should be automatically controlled and optimized by a measured service or metering system, transparently providing accurate and timely information about resource usage.

When we compare these 2007 and 2011 definitions of cloud computing with the comments on web services by Glass and Kreger at the turn of the century (as well as our own derived definition prior), we can't help but see how the early vision for cloud computing has taken shape today. First, web services can indeed be paired with other technologies to form a distributed system, in this case a centralized and scalable computing infrastructure that can be used by practically anyone to run software, develop applications, and "host a variety of different workloads." Second, those workloads can be quickly deployed worldwide, wherever there is internet access, and typically at a fair price, when compared to the costs of on-premises data management. Third, new opportunities are indeed developing for organizations seeking to tap into the on-demand, rapid, scalable, and cost-efficient nature of cloud computing. And finally, benefits are being seen in the integration of applications via the cloud, particularly as more options for multicloud and hybrid cloud integration develop. The early vision that perhaps hasn't been realized is found in Glass' "shift from client-server to peer-to-peer architectures," though discussions about the promise of peer-to-peer cloud computing have occurred since.

Though clearly linked to web services and the early vision of cloud computing in the 2000s, the cloud computing of the 2020s is a remarkably more advanced and continually evolving technology. However, it's still not without its challenges today. The data security, privacy, and governance of computing in general, and cloud computing in particular, will continue to require more rigorous approaches, as will reducing remaining data silos in organizations with pivots to hybrid cloud, multicloud, and serverless cloud implementations. But what is "hybrid cloud"? "Serverless cloud?" The next section goes into further detail.