RSS
Логотип
Баннер в шапке 1
Баннер в шапке 2
2020/07/19 10:00:00

Igor Nikiforov: GitOps is an incredibly powerful workflow template to control the modern cloud infrastructure

Igor Nikiforov, Lead DevOps Engineer, on the GitOps methodology and its use in the Kubernetes context.

Igor
Nikiforov
GitOps provides a self-service model for developers, allowing them to write, integrate, and deploy the code

Over the past few years, GitOps has become increasingly prevalent in the IT field, which is, generally speaking, a process that serves for a continuous deployment of a cloud application. This model is especially popular among DevOps engineers using Kubernetes. We asked Igor Nikiforov what GitOps is, its advantages and disadvantages, and who uses it. Igor Nikiforov is a lead DevOps engineer in a US company and one of the top Russian IT experts with more than a dozen IT certifications (ITIL, CKA, CKAD, 5x AWS, 2x HashiCorp, 2x Confluent).

Please tell us what GitOpsis and what does it do?

IGOR NIKIFOROV: The GitOps term doesn’t have a specific, well-established definition. Still, I would describe it as a methodology that uses advanced DevOps principles and applies them to address infrastructure management automation problems. Remarkably, GitOps cannot be considered as some kind of product or platform since all its processes are based on existing tools.

GitOps originated from the principles of IaCwith the purpose to fill in the main gaps in this approach, i.e. allow to skip the versioning of the code and approval of changes, as well as the ability to roll out changes by running the code manually. The single source of truth about the desired state of the system is the Git repository, where all changes are fully monitored and tracked, and thus are easily auditable. Here, GitOps simply implements a similar set of functions when working with an infrastructure or application configuration. Observability is an important feature of GitOps as it allows to ensure that the desired state and the actual state are fully identical.

What is your personal view of GitOps, and how do teams put it into practice?

IGOR NIKIFOROV: GitOps processes help teams manage their IT infrastructure using the processes they are already using for applications development. GitOps itself is in fact a combination of three main components: infrastructure as code (IaC), pull requests, and CI/CD automation on top of it all.

First of all, GitOps, for me, is an evolution of the pre-existing approach, that is, infrastructure as code (IaC). This approach also uses Git as a version control system for infrastructure configurations. IaC follows mostly a declarative approach to infrastructure management, identifying the desired state of the system and tracking its current state. GitOps, in the same way as IaC, requires a declarative description of the desired state of the system. When it comes to GitOps, the main component of IaC is the version control system, which acts as a single source of truth for describing the infrastructure.

Pull requests are used by GitOps as a mechanism for making and approving changes for all infrastructure updates. Pull requests provide the team members interaction through reviews and comments, and most importantly, the final approvals of the changes are made there. After the pull request is approved, all changes are recorded in the main branch of the repository and serve as an audit trail.

The closing stage of this process is the CI/CD pipeline, it’s the final step in the GitOps process. All new changes in the code occur through it, as well as the start of the automated testing process and the approved changes implementing. Any configuration drift in this case, such as manual changes or errors, is overwritten by GitOps automation, so the environment is always in the desired state as defined in Git.

Everything you describe sounds a lot like DevOps. How do they differ?

IGOR NIKIFOROV: First of all, it should be noted that GitOps and DevOps share the principles and goals. At the same time, DevOps is a cultural change first of all, when the development teams work together with the operations team. GitOps provides the tools and platform for implementing DevOps practices, such as collaboration, CI/CD, and version control, including Git, to automate infrastructure and deploy applications. This allows developers to work with pre-existing Git repositories, while the operations team can keep other related parts of infrastructure or application configuration separately.

What benefits is GitOps able to bring to developers and companies?

IGOR NIKIFOROV: One of the major advantages is the simple implementation process using familiar tools. This is because Git is the actual standard for version control systems and is the most common tool currently used in software development by most developers. The version control system allows the team to keep track of all modifications of the system configuration. It provides a single source of truth and serves as the main checker in case of breakdowns or unexpected behavior of system elements. This is accomplished by the option to view the Git history and observe the history of the changes that have led to the error. In addition, the Git history can be used as a source when verifying the audit or security compliance.

When it comes to development teams, GitOps provides a self-service model for developers, allowing them to write, integrate, and deploy the code. It also significantly reduces the time period from writing code to rolling it out into the production environment. This allows developers to start working more independently and productively. GitOps gives developers responsibility for their own configurations, while the operations team manages platform configurations for security and compliance by ensuring that confinement is in place. Thus, the operations team role is to focus totally on the development teams’ support and maintain the security and compliance of the infrastructure and related processes. All of this allows experimenting on the new infrastructure configurations promptly. If the new changes don’t play as expected, the team can use the Git history to roll back all system changes to its recent running state. This tool is incredibly powerful as it provides the familiar `undo` functionality, including git revert, in a complex infrastructure.

Moreover, GitOps take a lot of the hard work out of the Kubernetes development and control. This way, it helps organizations improve their productivity and simplify processes. Storing all of your configuration data in Git, you can deploy identical Kubernetes clusters in sequence across different environments. And Kubernetes is just one example because, in fact, any kind of infrastructure scaling becomes easier, faster, more reliable, and ergo less error-prone.

GitOps is often discussed in the context of Kubernetes. Is this its main use, or can it be applied to absolutely any kind of infrastructure?

IGOR NIKIFOROV: Sure, indeed, GitOps is mentioned quite often in the context of Kubernetes, but its scope isn’t limited only to Kubernetes. Basically, a company can use absolutely any infrastructure that can be described declaratively with the IaC tools available for this. But right now, due to the spike in the Kubernetes popularity, most of the GitOps solutions are implemented with Kubernetes functionality in mind, of course.

When using GitOps with Kubernetes, it is common practice to implement all the functionality through the so-called operators. In this case, an operator is a controller that extends the Kubernetes API to control GitOps processes through a declarative description of its essence. It compares the desired state in the repository to the actual state of the deployed infrastructure and updates the infrastructure whenever a difference is noticed between the states. The operator can also manage the repository of the containers images where the applications have been packaged and apply updates in the same way as usual deployment of new images.

Sounds great! Does it mean GitOps has no issues?

IGOR NIKIFOROV: GitOps is a process change, first of all, and that will require discipline from everyone involved. This is because the approving changes process through pull requests might seem too tedious and unusual, especially for those engineers who are used to making quick changes manually. Therefore, it is extremely important for absolutely everyone on the team to document all changes being made in the new pull request and what issues may arise from them. It will be really difficult not to be tempted to edit something directly in production environment or to make changes manually, but the less you do this, the better GitOps performs.

Another particular issue is the multiplication of Git repositories. This is typical for the IaC approach as well, but with GitOps they may grow in numbers even more. It depends on how you build the hierarchy of infrastructure environments and applications in Git. Sure, this will be followed by another issue, the configuration management of each of these repositories, as well as the agents that synchronize the repository state and the environment. The issue can be solved for sure by reducing the repositories number. However, due to this, all sorts of access control issues will arise, including the pull requests control.

Another side effect is the transparency loss, as it becomes complicated to find answers to the questions among the huge number of repositories and configuration files. This has a lot to do with the fact that it’s really difficult to check changes in Git against application deployments or infrastructure changes. Some of them may be related only to the applications configuration, while others may lead to the deployment of several applications at once. This issue can be partially solved by establishing some kind of standard for describing commits and pull requests, but the search in the Git history itself can take a long time to find the right answer.

Now, the last point I would like to focus on especially. GitOps does not solve the issue of secrets, although it inevitably occurs when using it. On the one hand, Git is not a place to store secrets, as they need to be encrypted and decrypted somehow so as not to be stored in open view. However, even if it becomes necessary to keep secrets encrypted, because of the rise in repositories, it will be really difficult to find a place where one or another secret should be updated. On the other hand, there are many options for solving this issue. For example, HashiCorp Vault or cloud solutions like AWS Secrets Manager or Azure Vault help manage secrets. Here is the issue, GitOps does not help in any way to link the configuration stored in Git with the secrets associated with it, which may be stored in any of the secret managers. Even popular GitOps solutions like ArgoCD or Flux still lack the support for secret management solutions. Therefore, you also have to look for third-party solutions or some other approaches that can cover the company needs.

Despite the foregoing, I don't mean that GitOps is a faulty approach. Just never forget the possible issues when you apply it in your projects.

What is the reason for the popularity and advantage of GitOps over other methodologies?

IGOR NIKIFOROV: I guess it’s mainly because of its basic principles that can be applied to absolutely any type of infrastructure automation, including virtual machines or containers. It also works really well for teams looking to manage their infrastructure by Kubernetes.

If most tools and methodologies promise faster deployment and smoother code and infrastructure management, GitOps is primarily distinguished by its experience and focus on developers. Infrastructure management with GitOps carries out in the same version control system as application development allowing all teams to use a single entry point for absolutely all changes, from applications to infrastructure, using familiar Git feature set.

Can you provide real-world examples of this approach in the field of infrastructure and problem-solving in case of errors?

IGOR NIKIFOROV: Let’s take, for example, a scenario of deploying an application to some kind of cloud environment. A spike in traffic occurs at some point, causing the application performance issue and following by the errors on-premises. The operations team decides to commit more resources to solve this issue.

Using GitOps, the team makes changes to the Git repository. They can be quickly checked by other team members further for configuration errors and approved for later deployment. The changes made will start the CI/CD pipeline where the git operator compares the current state and the desired state. If the infrastructure itself is free of issues, it will be automatically configured in accordance with the desired state.

If there are any errors due to the new configuration, the GitOps approach will allow the operations team to roll back to the previous infrastructure state smoothly and fast. They can once again go through all the configuration changes that caused the issue and re-adjust the configuration to rerun the CI/CD pipeline to deploy the changes further.

Depending on your architecture complexity level, along with the changes made, rolling back to a previous state of configurations can be very difficult without GitOps, especially if the changes have not been documented properly. Therefore, such a declarative approach and a version control system make the complex and time-consuming task of infrastructure management simple. This approach improves the system visibility, along with reliability and stability at the same time.

GitOps is an incredibly powerful workflow framework for managing modern cloud infrastructure. While it’s primarily used in Kubernetes clusters management, GitOps is also applicable to other non-Kubernetes systems. At the same time, GitOps does not introduce anything new and does not reinvent the wheel but only applies the accumulated experience in a new field.

Alex Lomov