A Walk Through Time: Understanding DevOps

To grasp DevOps, we journey back to the era of the programmers.

As the Tao tells us:

The programmers of old were figures of mystery and wisdom. Their thoughts elude our understanding, so we can only speak of their presence:

• Present, like a fox navigating a stream
• Vigilant, like a commander on the battlefield
• Gracious, like a host welcoming guests
• Uncomplicated, like unshaped blocks of wood
• Inscrutable, like dark pools in hidden caverns

The Programmer brought forth machine language. Machine language brought forth the assembler. The assembler brought forth the compiler. Now, languages abound, thousands upon thousands. Every language serves a purpose, however modest. Each embodies the balance of Yin and Yang in software. Each holds its own within the tapestry of software.

Back then, software development spaces were often called labs, and programmers were considered scientists. To craft a successful application, programmers needed a deep understanding of the problem it aimed to solve. They had to consider where it would be used and the type of system running it. Essentially, programmers possessed complete knowledge of their application, from initial concept to final deployment and ongoing support.

However, human nature intervened, and our desires grew. We craved more speed, more features, more users - more of everything.

Being naturally modest, humble, and peace-loving individuals, programmers struggled to withstand this surge of demanding users. To survive, they adapted, evolving into specialized roles. Soon, programmers as a single entity faded, giving rise to new breeds: developers, software engineers, network administrators, database developers, web developers, system architects, QA engineers, and countless others. This rapid evolution and adaptation to external pressures became ingrained in their very being. New specializations could emerge in a matter of weeks. Web developers branched into back-end developers, front-end developers, PHP developers, Ruby developers, Angular developers… it was a chaotic explosion of growth.

In this rush, they forgot their shared origin - the Programmer, a simple and peaceful seeker of a better world. They began to clash, each group claiming to be the true heir of “The Programmer,” their lineage purer than the rest.

Communication dwindled to a minimum, reserved only for absolute necessity. They ceased celebrating the achievements of their distant kin and eventually stopped even the simplest gestures of connection.

Little did they know, the spark of the Programmer still flickered within them, waiting to ignite and bring harmony to their world.

Blinded by their own ambitions, the descendants of the Programmer lost sight of their core purpose - solving problems for their clients. Clients, in turn, felt the consequences as projects became delayed, exceeded budgets, or outright failed.

Occasionally, a beacon of hope emerged. Inspired individuals would attempt to bridge the divides between these factions. They introduced Waterfall, a seemingly brilliant concept that acknowledged the minimal communication between developer groups. Once a group completed their task, they passed it along to the next, never looking back.

This approach worked temporarily, but as always, humans (Clients) desired more. They wanted active involvement in the software creation process, frequent opportunities to provide feedback, and the ability to request changes even late in the game.

Software projects became so prone to failure that it was accepted as the norm. Statistics indicated that over half of all projects failed, and there seemed to be no solution in sight (ZDNet, CNet).

Yet, every generation had its visionaries who understood that these developer groups had to find a way to collaborate, communicate, and adapt to ensure the best possible solutions for their clients. Traces of such efforts existed as early as 1957, spearheaded by colleagues of the esteemed John Von Neumann. However, it wasn’t until the early 2001 that a revolution began, ignited by a group of forward-thinkers who crafted the Agile Manifesto.

The Agile Manifesto rests on twelve key principles:

1. Prioritize customer satisfaction through early and continuous delivery of valuable software
2. Embrace changing requirements, even late in development
3. Deliver working software frequently (weeks instead of months)
4. Foster close, daily collaboration between business stakeholders and developers
5. Build projects around motivated individuals, trusting them to get the job done
6. Prioritize face-to-face communication (co-location)
7. Measure progress primarily through working software
8. Maintain a sustainable development pace
9. Pay continuous attention to technical excellence and good design
10. Embrace simplicity—maximizing the amount of work not done
11. Empower self-organizing teams
12. Regularly adapt to changing circumstances

This manifesto was a giant leap towards restoring harmony and balance. For the first time in a long time, connecting all stakeholders in the software development process emphasized cultural and human elements as much as procedural and technical ones. People started talking again, meeting regularly, and exchanging ideas and feedback constantly. They rediscovered their common ground, and clients transformed from external entities into integral team members.

Challenges remained, but the future appeared brighter. Being agile meant embracing change and adapting constantly. However, excessive change made it challenging to stay focused on the ultimate objective - delivery. And that’s how Lean Software Development emerged.

Inspired by LSD (pun intended) and risking ostracism, some descendants sought solutions beyond their specialization and the software industry. They found inspiration in the practices of a prominent car manufacturer. Toyota Production System was strikingly simple, and its lean manufacturing could be readily applied to software development.

Lean hinges on seven principles:

1. Eliminate Waste
2. Build Quality In
3. Create Knowledge (Amplify learning)
4. Defer Commitment (Decide as late as possible)
5. Deliver as fast as possible
6. Respect People (Empower the team)
7. Optimize the Whole

Layered upon Agile, Lean principles reinforced the mindset and focus on effectiveness while maintaining flexibility throughout the process.

With Agile and Lean embraced by software development teams, it was a natural progression to apply these principles to IT as a whole - ultimately leading us to DevOps!

Enter DevOps: A Three-Lane Highway

The traditional view of software development involved distinct roles: business analysts, system architects, front-end developers, back-end developers, testers, and so on. Optimizing this process, incorporating agile and lean principles, primarily focused on these groups. Once the application was deemed “production-ready,” it was handed off to “Operations,” consisting of systems engineers, release engineers, DBAs, network engineers, security professionals, and others. DevOps emerged as the driving force to dismantle this wall between Dev and Ops.

DevOps embodies the implementation of lean principles across the entire IT value stream, extending development into production and uniting all ‘distant relatives’ descended from the original Programmer.

For a compelling explanation of DevOps solutions, look no further than Gene Kim. If you haven’t read The Phoenix Project, I highly recommend it.

Remember, you shouldn’t hire a “DevOps engineer” or establish a separate DevOps department. DevOps is a culture, a mindset, an integral part of IT as a whole. No single tool or level of automation can magically transform your IT into a DevOps organization or empower your teams to deliver maximum value to your clients.

DevOps is often depicted as a three-way method, but I envision it as a three-lane highway. You begin in lane one, gain momentum, shift to lane two, and eventually find yourself cruising in the fast lane - lane three.

• Lane one - Prioritize the performance of the entire system over individual components

• Lane two - Establish a continuous feedback loop flowing upstream, ensuring it’s heard and acted upon

• Lane three - Foster a culture of experimentation, continuous improvement, and rapid learning from failures

Lane One: Gaining Momentum

Understanding the significance of the whole system and prioritizing tasks effectively is paramount for any IT organization embracing DevOps. Bottlenecks created by individuals or teams hinder the flow of work and are unacceptable.

Ensuring an uninterrupted workflow becomes the collective goal. Regardless of their specific role, everyone involved strives to achieve profound understanding of the system. This mindset directly impacts quality, as defects are never passed downstream to avoid creating bottlenecks.

But maintaining flow isn’t enough. A productive organization continuously seeks to enhance it. Numerous methodologies exist for this purpose, whether it’s Theory Of Constraints, Six Sigma, Lean, or the Toyota Production System. Feel free to choose what works best for you, create your own approach, or blend different methods.

DevOps principles transcend individual roles and team affiliations. Whether you’re a System Architect, DBA, QA, or network administrator, the same rules apply. Everyone is expected to adhere to two fundamental principles:

• Maintain an uninterrupted system flow
• Continuously optimize and enhance workflow

Lane Two: Shifting Gears

Uninterrupted system flow is unidirectional, typically from development to operations. Ideally, software is developed rapidly and with high quality, deployed to production, and delivering value to clients.

However, DevOps isn’t a utopia. If unidirectional delivery were sufficient, the waterfall method would suffice. Evaluating deliverables and communicating feedback upstream are crucial for quality assurance. The first “upstream” communication channel to establish is from Ops to Dev.

Celebrating successes is easy, but seeking and providing feedback offer a clearer picture. Every small step downstream should be met with instant upstream confirmation.

The method of establishing this feedback loop is flexible. Invite developers to support team meetings or include network administrators in weekly sprint planning. As long as a feedback mechanism exists, and comments are addressed promptly, your organization is progressing on its DevOps journey.

Lane Three: Reaching Warp Speed

The DevOps fast lane is not for the faint of heart. It demands a mature organization ready to embrace risk and learn from failures, continuously experiment, and accept that mastery requires repetition and practice. The term Kata is frequently used for a reason. Consistent training and repetition transform complex actions into routines.

However, before attempting complex maneuvers, mastering each individual step is crucial.

“What is appropriate for the Master is not appropriate for the novice. You must understand Tao before transcending structure.”

The DevOps Third Way/Fast Lane involves dedicating time for daily experimentation, rewarding risk-taking, and intentionally introducing faults into the system to enhance resilience.

To prevent organizational chaos when implementing these measures, establish frequent feedback loops between all teams while ensuring clear bottlenecks and uninterrupted system flow.

Embracing these practices keeps your organization alert and adaptable, capable of responding to challenges swiftly and effectively.

Summary: The DevOps Organization Checklist

This checklist helps assess the level of DevOps enablement within your IT organization. Feel free to add your own points and share your thoughts.

• The wall between development and operations is nonexistent. Both are integral parts of a unified process.
• Work transferred between teams is consistently verified and of the highest quality.
• Work never piles up; bottlenecks are addressed proactively.
• Development doesn’t hijack Operations’ time; deployment and maintenance fall within the same timeframe.
• Development avoids delivering code for deployment late on Fridays, leaving Operations to clean up over the weekend.
• Development environments are standardized, easily reproducible and scalable to production by Operations.
• Development can deliver new versions at their discretion, and Operations can deploy them seamlessly.
• Clear communication channels exist between all teams.
• All team members have dedicated time for experimentation and system improvement.
• Defects are intentionally introduced (or simulated) and addressed regularly. Lessons learned from each experiment are documented and disseminated to relevant personnel. Incident handling is routine and follows established procedures.

Conclusion

Utilizing modern DevOps Tools such as Chef, Docker, Ansible, Packer, Troposphere, Consul, Jenkins, SonarQube, AWS, etc., doesn’t automatically equate to applying DevOps principles. DevOps is a way of thinking. It’s about recognizing that everyone is part of the same process, working together to deliver value. Every individual involved in software delivery can either accelerate or hinder the entire system. A bug introduced during development can be just as detrimental as an incorrect firewall configuration.

DevOps encompasses everyone. Once your organization internalizes this understanding, the right tools and technology stack will naturally follow.