DevOps: The Genesis
First of all, it’s all about agile.
The DevOps methodology was created on top of agile methods, to deliver a higher value inside software releases, automating feature release through pipelines, that can test hypothesis faster allowing higher adaptability using “fail-fast” approaches. Those changes are more cultural than technical, so it’s normal to see DevOps being called culture.
The implementation of DevOps happens through processes automation, having a strong sense of processes re-engineering inside the company. Comparing to the cultural change, the technical is easy to implement. Therefore the role that a “DevOps Engineer/Analyst” performs is very confusing, which enables many SysAdmins and Infra Analysts assuming the role of “DevOps.”
Lean is the basis of Agile
Reality is not as happy as it sounds. After World War II, Japan was destroyed and under-resourced after losing the war. With a limited amount of resources, the country needed to reinvent itself and survive after a time of severe depression. During that time two guys gained attention inside a company that later gave its name after the methodology.
Those guys were Eiji Toyoda and Taiichi Ohno, inside Toyota Motor Corporation. They’re the founders of the “Toyota production model” also known as Toyotism.
Toyota gave birth to Lean
Lean teaches how to optimize the end-to-end process, focusing on processes that create value for customers. Bottlenecks in the process must be removed, and wasteful activities need to be identified and avoided. Both explained and defined by LEAN 3M: Muda, Mura, and Muri.
Also teaches to improve yourself day after day and always focus on quality through Kaizen (continuous improvement).
Japanese culture truly believes that quality is the main objective to deliver value to customers since quality is what brings your clients back.
A mindset that helps to look at each part of the process exclusively and think about the improvements. Involving the people who are part of the process, encourage the inclusion of these people in the decisions of change, since:
- It is much easier to accept a change when it is not imposed (top-down);
- There is a greater absorption of change by people when they’re included in the planning;
- The people who are involved in the process bring their concerns and suggestions, which contribute positively to the evolution of the change, making the idea more robust.
The process of defining improvements through Kaizen happens (usually) in the following order:
- Define data-driven objectives;
- Review the current state and develop an improvement plan;
- Implement improvement;
- Review the implementation and improve what does not work;
- Report the results and determine the items to be monitored.
This process is also called PDCA: Plain-Do-Control-Act, which is summarized in:
- Plan (develop the hypothesis);
- Do (experiment);
- Check (validate results);
- Act (refine the experiment and start over).
3M: Muda, Mura, Muri
Any activity that consumes time without adding value to the final consumer. e.g.:
- idle time in the process;
- products with a defect.
It’s important to remember that there are different levels of Muda that can be removed quickly or not, and the classification depends on the time for removal.
An example of a more time-consuming Muda is the discontinuation of legacy software that ends up with longer release cycles, causing teams to be idle, followed by an often long or manual test routine.
Unevenness in operation, caused by activities that are very changeable and unpredictable, generating different results in all executions. e.g., the execution of tasks that were not well planned and ended up arriving with strict deadlines. The team runs in the rush, generating exhaustion, despair, and moreover, when finished leaves the people who have performed these tasks waiting (for feedback, or confirmation that it is completed).
Overburdening equipment or operators by requiring them to run at a higher or harder pace beyond the limit, to achieve some goal or expectation, causing fatigue and consequently failures during the process. These failures are usually human errors caused by fatigue during overwork.
Back to Agile
In 2000 a group of 17 people met at a resort in Oregon to talk about ideas that could improve the flow of software development. After a year of mature ideas, these people met again and published the ideas, which we now know as Agile Manifesto.
Main points are:
Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan
I will restrict the explanation of these points with the DevOps point of view, keeping on track (now).
Individuals and interactions
over processes and tools
First comes the individuals, they should receive the necessary tooling to work with, and then be empowered to do their jobs. Interactions between people are greatly encouraged, for sharing knowledge and also for facilitating creative flow within development teams.
An excellent example of interaction encouraged through DevOps is the code review habit. Considering that small parts of the software will be iterated and approved in the pipeline passing through different environments, automatically, the best way to prevent defects is through code review.
This habit brings benefits such as:
- Knowledge sharing;
- Observation of the problem from a different point of view;
- Team engagement;
- Lesser bugs.
over comprehensive documentation
Here’s a trick in “working software,” software that works is not code that compiles. The software that works is what meets the requirements of the user; i.e., the software that solves the problem and the pains of the user.
As the market is very dynamic, and evolves with high speed, often during the software development project the requirements change due to external factors. Therefore, knowing that it is not possible to predict all the elements, many “workarounds” are made during development and documented. Passing the responsibility to the user to handle the faults, and perform the workarounds, expending more effort than would be required to perform the tasks using the software.
Deliver a working software frequently, ranging from a few weeks to a few months, considering shorter time-scale. - Agile Manifesto
Encouraging as many deployments as possible, so that failures happen as early as possible, thus allowing their impact to be much less.
Failures are understood and encouraged because it’s part of the mindset. Because:
- Only those who do make mistakes;
- Failures are the best opportunity for learning and evolving;
- Shit happens.
Nothing like quoting Murphy’s law to contextualize
“Anything that can possibly go wrong, does.”
Therefore, it’s best for failures to occur early, while the cost of correction is still low. Failing a controlled testing environment allows the fix to be much faster (and cheaper) than it would if the fix were already in production.
For this approach to succeed, there is a premise that environments are production copies, or at least as close as possible. Otherwise, there will be behavioral changes in the software between the environments, making the test environment unfeasible.
If the environments are divergent, the promotion of bugs for production will be very frequent, causing late failures, which are expensive failures.
over contract negotiation
Know your client! Including it in the process is the best approach to have working software. After iterating over deliverables, it’s essential to create a positive feedback loop with your client, bringing it as close as possible to the development of the tools that he/she is going to use.
We can describe this situation with:
- From point A it is possible to see only point B;
- From point B it is possible to see point C;
Therefore there is a great incentive for the software to be delivered in parts, continuously. Thus gathering user feedback on the next steps, following the concepts of evolutionary prototyping, which were widely publicized through The Lean Startup.
This point contrasts sharply with the previous one about continuous release, so that it is possible to present the prototype and evolve it throughout the project.
Learn who your customer/consumer/user is, and whom you are making the software for, as this is the only way you can deliver value to that customer. An essential part of the software development process is to be empathic with user problems, and to truly understand what the problem is to be solved, and the result of the impact on software development (value creation for the user).
Responding to change
over following a plan
Redesigning the requirements overtime is part of the job, and a necessary step to success. If you want to build something useful that is going to grow and have absorption, it’s a key feature to include your client in the implementation process.
It will be the only way to bring all the problems of the user to the table and create the best solution for all these problems because the user is the only person that knows the real challenges he faces in their routine dealing with software.
With continuous delivery of software along with monitoring results, the process of collecting feedback is much simpler and faster.
DevOps, DevOps, DevOps
With the popularization of DevOps, a lot of disagreement came out there followed by a significant confusion about the subject. It is very common to come across different interpretations of what is DevOps. There is a lot of euphemism in the area, and gourmetization on LinkedIn, with many SysAdmins calling themselves DevOps since they learned to code shell script inside Python.
Do you want to keep reading? Here are the benefits of adopting DevOps techniques.