IT and Co-location
In the distant past of software development - by which I mean, ten years ago - software companies made a clear distinction between Application development and Infrastructure management. Most companies had a dedicated IT team to configure, provision, install, manage, maintain and update the hardware infrastructure needed to run their software applications; separately, the software engineering team focused on building the software platform and applications. The hardware was typically hosted in a co-location facility. The IT team essentially “owned” this hardware, with responsibility for all the functions associated with it.
Over the past five years, the rise of the DevOps movement in startups and in large cutting-edge technology companies such as Google and Netflix, especially in Silicon Valley, created the first bridge between these two areas of software applications and infrastructure.
According to the DevOps philosophy, configuration and management of the underlying infrastructure services was treated more like software development, using higher-level languages like Ruby and Python, and standard software application methodologies and tools like agile development, source code control (e.g. git), versioning and branches, bug tracking (e.g. JIRA) and test automation. DevOps as a function came increasingly within the purview of the software engineering team rather than that of traditional IT.
The rise of Cloud computing and “Infrastructure/Platform as a Service” offered by vendors like Amazon Web Services (AWS) has further accelerated this trend. For example, AWS has a built-in command-line tool called AWS CLI, as well as software API bindings for popular languages like Java, PHP and Python, that allow a computer program to perform actions for the entire Infrastructure life-cycle: from provisioning new hardware, to scaling it, managing and monitoring it, through the final tear-down when it is no longer needed.
Underneath the covers of course is an actual data center, with real servers, racks, power and networking equipment, security devices and policies, NOCs, and all the other “traditional IT” functions. But this layer is completely “virtualized” or hidden from the application developers, who can treat it as yet another software API or service.
The public cloud computing vendors, and private cloud data centers within large companies, still undertake all the classic IT projects and perform the same tasks as before. What is different however, is that from the perspective of a typical software startup or mid-sized SAAS application provider, a separate Infrastructure function has ceased to exist; instead it has been absorbed into the software engineering team, whose members are now expected to possess skills for managing cloud computing APIs, just as they are typically expected to understand how to interact with an operating system, or with a web application server or a handheld device API. This will not only affect the composition and operating practices of software engineering teams, but will also enable them to develop code at an increasingly faster pace in the future.
OpsWorks and Docker
A great example of this new approach of treating infrastructure provisioning, management and monitoring as software code is embodied by the AWS OpsWorks service, which uses an open source framework called Chef for application management. Chef programs, appropriately named “recipes”, can be used to configure servers, implement auto-scaling, apply security policies and monitor the resulting infrastructure stack.
The most recent step in this evolution is the increasing popularity of deployment containers, especially Docker (but also Rocket and LXC). In an explicit analogy to shipping containers which provide for all types of disparate goods to be handled in a uniform way by different modes of transportation, Docker enables application developers to interact with their infrastructure environment in a uniform way, regardless of whether the container is hosted on a laptop or on a data center server with high computing power. Both Amazon’s OpsWorks and their newly offered Container Service support container deployment.
Implications for Development Teams
This trend will have some interesting implications for the software development team in a typical small to medium-sized software-as-a-service (SAAS) company:
- Team composition: Software developers in application development teams will need to understand both infrastructure (at the API level) and traditional application logic (algorithms + data structures). In larger development teams, this may be structured as separate sub-teams, each with a different individual focus.
- Faster turnaround: Since the PAAS vendors allow computing resources to be added or removed with the click of a button or with a single API call, dev teams can now react much faster to the needs of customers or individual developers. This enables auto-scaling based on customer demand and individual development sandboxes for experimentation, which will change how developers work on their code.
- Cost: Auto-scaling removes the need to provision servers up-front for peak demand. Along with economies of scale at the big data centers, this makes infrastructure resources significantly cheaper. At the same time, unless scaled-down appropriately, unused resources continue to incur periodic costs even when they are idle which can get very expensive. Due to these two competing forces, managing virtual infrastructure cost will be an important consideration for software architecture, code and development processes in the future.
- Approach: Treating infrastructure configuration and management in a similar way to source code is a fundamentally different approach, lending itself well to the standard tools, processes and abstractions of software engineering as described previously. This will enable software development teams to take an agile, iterative approach to infrastructure development, analogous to standard software “scrum” practices.
- Leaky abstractions: One challenge with the new approach is that software developers will be more aware of the details about the underlying virtual infrastructure. This could lead to “leaky” abstractions where they take advantage of local quirks within their particular infrastructure. Introducing containers like Docker will go a long way in mitigating this issue.
The Future: Cluster Management
As applications increasingly adopt a container-centric approach for deployment, two additional types of services and tools will become very important:
- Container management: This type of tool enables container deployment, orchestration, management and scheduling. A great example is the open source Kubernetes project from Google, others include Lattice and Marathon.
- Cluster management: This is the concept of providing a higher level of abstraction across the entire data center, i.e. an entire cluster of servers is treated as a single operating system kernel capable of running multiple projects simultaneously. Initially proposed by Google for their internal Borg project, this is now available to everyone through the open source Apache project Mesos. The new open source operating system CoreOS is designed from the ground up to run Docker containers across a cluster.
As these types of tools become mainstream in the future, developers will only need to focus on building out their application components inside standard containers and specifying the needs and requirements of those containers; it will be up to the underlying infrastructure services framework to optimize the required resources and cost to provide the application with everything it needs.