Introduction to Web Scale It

Web scale IT is a relative newcomer to IT infrastructure and architecture, but it changes everything we know about scale, agility, and flexibility for companies.

In this article, I’ll tackle web scale IT, including:

• Concepts & components
• Web scale vs hyper-converged
• How & why to switch to web scale IT

What is web scale?

Web scale IT, or web scale infrastructure, is the technology of converged architectures with very flexible, scalable, fault-tolerant software that can run on standard x86 hardware. This combination of characteristics allows you to integrate various infrastructure components—computation, storage, virtualization, and networking—into one platform or appliance. By aggregating resources and centralizing management, you will:

• Increase efficiency and flexibility
• Minimize maintenance

Global enterprises working across large scales have implemented (essentially created) web scale IT to change the way their IT teams work and maintain speed, agility, and scalability.

The best examples are web giants like Google, Facebook, and Amazon, but smaller players are starting to get in on the action, too. Gartner first introduced the term in 2013 with high hopes for its rapid growth within five years. Although the uptake has been slower than anticipated, its steady growth will likely continue into the future.

Web scale characteristics

Web scale doesn’t refer to one single technology; rather, it represents an infrastructure formed by a set of technologies and capabilities that large companies have successfully implemented at global scales.

The methodology and approach of web scale infrastructure are unique in that they primarily only work in large environments, fostering different ways of thinking compared to more traditional approaches. Driving the methodology are properties and components that focus on:

• Convergence
• Distribution
• Automation
• Error recovery and prevention

Methodology & approach

The web scale architectural approach focuses on designing, building, and managing data center and software infrastructure in a way that tailors systems to global computing and large environments.

The nature of the internet and global computing means modern architectures tend to grow at very fast rates. This, in turn, creates bottlenecking. To avoid scale limitations, web scale methods prioritize the efficient scalability of your infrastructure for these key components:

• Speed and agility
• Consistency
• Versioning
• Tolerance

Tolerance, in particular, is key to helping operators identify and fix issues more quickly, thus preventing bottlenecking and enabling faster deployment. Web scale’s open environment fosters tolerance by facilitating:

• Standardizing protocols
• Identifying issues
• Creating a unified stack for efficient communication

Making improvements in these areas often requires some customization to help suit specific business requirements. Doing so allows you to:

• Tailor your system to your needs and budget
• Prevent getting locked-in with vendors

Properties & components

Web scale infrastructures are software-based, meaning that everything is on software running on standard x86 hardware without any accompanying specialized hardware performing single tasks.

This pairs well with the need for being able to expand while functioning as a cohesive unit—rather than relying on several deployments of multiple units that are not individually scalable.

That said, web scale cannot upgrade all components in one go due to the immense size of the system. That means that certain features are crucial for web scale systems:

• Self-defining and versioned objects
• Self-describing and version-aware services

This allows the encoding and serialization of structured data as well as communication between the various parts of the distributed system, all without the expectation of upgrading all components at once.

The huge size of web scale architecture also leads to the human-to-machine ratio heavily skewing in the robots’ favor. Such disparities mean you should implement analytics and automation software to reduce human responsibilities and interactions.

To aid this automation, web scale architecture should include programmatic interfaces working off HTTP-based services. These interfaces should use latency, loss-tolerant protocols, and asynchronous request responses to grant complete control and automation.

And finally, to protect against single points of failure and bottlenecks, the architecture should include failure tolerance considerations that can address problems as quickly as possible. Some techniques for accomplishing this goal include:

• Consensus algorithms
• Rate limiting
• Multiple replicas
• Two-phase commit