People Break Stuff!

Now we are into the New Year, I’ve had some time to...
by JAY SMITH | February 13, 2018

Now we are into the New Year, I’ve had some time to reflect on 2017 and our customers’ experiences, especially on Black Friday, Cyber Monday and through the holidays. There was a common trend that jumped out to us all as we reviewed where they had difficulty, and it was the fact that people break stuff! We used a different word, but “stuff” will do. I’m not talking about people making bad decisions like not applying security patches as they come out and then wondering why they are suddenly neck deep in an attack. No, I’m referring to intelligent people, with the best of intentions, but with a few impulse control issues – let me explain.

Their livelihood depends on their site functioning properly and they care a lot about the stability of their applications. Perhaps too much though, because if they see something that doesn’t look perfect, they feel the need to touch it and figure out why. This often results in an untracked production change that can spell disaster for their storefront. But I get it, when you care so much about the outcome, it takes tremendous discipline to not touch something, and to not want to fix it right that second.

I’m not suggesting that these otherwise intelligent people should somehow shutdown and become somebody else. Quite the contrary, I believe strongly that their need to make things better, and their urgency to get it done, is to be encouraged. I’m more concerned with the environment that allows them to make an untracked hot fix in production in the first place. I think the bigger problem is that they are missing some key tools to help them apply changes safely, without having to endure the usual slow change management process.

So the real problem is a bit more subtle than the obvious risk associated with making a hot fix in production. More typically, I see customers making production changes or deployments with little to no testing in a production like environment. Often, the only testing they do is on their local box or a dev server that is wildly different from production. This is a problem that I can do something about. Even though we are all delivering our services from the cloud, sometimes we miss the obvious advantage that that affords us.

Last year, my support team completed the transition to a software defined delivery model where we no longer hand build any infrastructure. This means that we can push button deploy the distributed system, that is their modern cloud application, by compiling the software description of their system into a running application on whatever cloud provider they choose. We later added the ability to use this same toolset to deploy past versions of the application from backup. This means a staging site is simply a redeployment of the most recent backup of the production system. See where I’m going?

My goal for 2018 is to put this control in the hands of our customers. I want my customers to leverage our software defined infrastructure in their deployments so that they can do dry runs on a copy of production, eliminating risk. Now, with an old hardware-based mindset, this sounds like a lot of work. But we’re in the cloud and it’s easy given the cloud-native software we have already developed.

Our next iteration of dry run deployments will be driven from a webhook in the Webscale portal. That means testing their deployment is as simple as clicking a menu item in our portal. With that click, we will deploy a copy of production, install their latest code release, and execute a set of automated tests aggregating the results and presenting them back for inspection.

This tool is the first of many coming from Webscale to help eliminate risky deployments and changes from our production environments, while still helping our customers realize their urgency to fix things, while maintaining 100% uptime.

Popular posts

by Jose Kunnappally | August 22, 2022

Ecommerce Holiday Shopping 2022: What to Expect and...

by Jose Kunnappally | April 18, 2022

The Global Ecommerce Security Report 2022

by Jose Kunnappally | January 12, 2022

How a CDN can boost your Core Web...

by PAUL BRISCOE | January 11, 2022

3 Ways to Write Better Caching Modules in...

Stay up to date with Webscale
by signing up for our blog subscription

Recent Posts

by Daniel Bartholomew | September 25, 2023

Supercharging LLMs with Supercloud

Supercloud, characterized by a decentralized and distributed architecture, has the potential to revolutionize cloud computing. This paradigm shift could have far-reaching implications for Large Language Models (LLMs), such as ChatGPT,...
by Daniel Bartholomew | August 27, 2023

Key Kubernetes and Edge Trends to Watch

Daniel Bartholomew, Webscale's Chief Product Officer, has shared his insights on four noteworthy trends to monitor within the realms of Kubernetes, container orchestration, and the expanding landscape of edge computing....
by Daniel Bartholomew | July 31, 2023

Prometheus Querying – Breaking Down PromQL

Prometheus has its own language specifically dedicated to queries called PromQL. It is a powerful functional expression language, which lets you filter with Prometheus’ multi-dimensional time-series labels. The result of each...