Once upon a time, the concept of “workflow” in Kubernetes might have seemed a bit antithetical.

Consider this older definition from the business process management world, via Wikipedia: “A workflow consists of an orchestrated and repeatable pattern of activity, enabled by the systematic organization of resources into processes that transform materials, provide services, or process information.”

“Orchestrated and repeatable” sure sound relevant to Kubernetes, but that definition (and many of its variants) also suggests state – something Kubernetes and containers in general weren’t thought to be good at in their initial phases. In the early days, a term like “stateful application” would have been viewed as a no-go for Kubernetes, says Ravi Lachhman, evangelist at Harness.

“Since containerized workloads are ephemeral and meant to terminate quickly and gracefully, this would not be conducive to a workflow that would be long-lived,” Lachhman. “Inherently, workflows are stateful [and] need to live to give decisions or move forward workloads.”

[ Kubernetes 101: An introduction to containers, Kubernetes, and OpenShift: Watch the on-demand Kubernetes 101 webinar.]

As Kubernetes and its ecosystem have evolved, however, automating certain workloads and processes – including for stateful applications – has become much more achievable. Lachhman points to Kubernetes Operators as a particular boon for workflow automation – we’ll discuss this in detail at the end of this post.

Workflows you can automate with Kubernetes

In the meantime, we asked Raghu Kishore Vempati, director of technology, research, and innovation at Altran, to take us through some key examples or workflows or processes that can be automated with Kubernetes. Gary Duan, CTO at NeuVector, also shares some thoughts from a security automation standpoint.

1. App setup/Installation

For large solutions that comprise several (or more) applications, the setup and installation process almost inevitably requires automation to reduce the operational burden, Vempati notes. Cluster management can otherwise become significantly complex, says Vempati, especially as various installation units are further versioned.

“Adopting a DevOps approach, Continuous Delivery of applications and their configuration on a Kubernetes cluster can be completely automated,” Vempati says. “A single installation/setup for a particular logical app plane could consist of several resources that could include deployments, services, secrets, stateful sets, etc. Several such installation units belonging to a single application/solution can be orchestrated in an order to be set up on the K8s cluster.”

[ Kubernetes terminology, demystified: Get our Kubernetes glossary cheat sheet for IT and business leaders. ]

2. Pod and node scaling

Dynamic pod scaling is considered one of Kubernetes’ particularly important features. Vempati notes that the Horizontal Pod Autoscaler, or HPA, enables the system to scale pods based on common system metrics (e.g. CPU utilization) or custom app metrics (e.g. number of requests), which are configurable for the cluster.

“However, the configuration itself could be subject to change for various resources on the cluster,” Vempati says. “This process of configuring the scalability behavior of the pods itself can be automated and can be subject to various factors and dependencies within and outside the application/solution.”

There is a corollary here with another key concept in Kubernetes architecture: nodes.

“K8s by itself doesn’t provide for node scaling,” Vempati says. “It [does], however, have the capability where nodes can be added or removed from the cluster without impacting the execution of applications.”

Various cloud platforms offer automation of node scaling as part of the platforms or services, according to Vempati, including for hybrid or on-premises environments. “As with pod scaling, while the node scaling can be configured, the process of configuring the scalability of the nodes itself can be automated based on factors internal and external to the application/solution,” Vempati says.

[ Read also: 5 open source projects that make Kubernetes even better. ]

3. Persistent storage management

Stateful applications such as databases are no longer taboo in Kubernetes environments – far from it.

“K8s has many features that help run stateful applications,” Vempati says. “For example, it provides the ability to dynamically provision storage volumes on-demand.”

Vempati also points to the ability to clone persistent volumes for storage systems that implement the Kubernetes CSI spec, as well as the ability to capture snapshots of the volumes that are accessed by applications, as key features on this front.

“For large applications running in production, these capabilities are very useful,” Vempati says. “For applications that require high availability of data, having the latest snapshots of data and the time to restoration of access to the data is very critical. Automation of the above-mentioned capabilities and their associated configurations for the applications can help achieve the same.”

Let’s examine three more important examples of what you can automate:

4. Chaos testing

“When large solutions are deployed on K8s, it is difficult to assess the overall resilience of the solution,” Vempati says.

He notes that some teams turn to the chaos testing approach – also known as chaos engineering, a term made famous by Netflix’s internal technology teams – to vet the resilience of a given cluster-wide deployment. This can be automated, too.

“In this approach, chaos (or instability) is induced into the solution deployed to systematically verify for weaknesses,” Vempati says. “On a K8s cluster, this entire process of inducing chaos can be automated and run several times periodically or on-demand.”

5. Deployment and versioning of Custom Resource Definitions (CRDs)

“For developers and organizations that work on developing customized solutions/platforms based on Kubernetes, extending [its] capabilities by building Custom Resource Definitions (CRDs) and deploying them is a very common approach,” Vempati notes.

“Deploying the CRDs could include creating new namespaces, defining specific roles and associated access controls, [and so on]. Using automated routines, deployment of CRDs and their custom configurations can be achieved for various scenarios. This is a capability/feature that is observed in many platforms/products that are built targeting enterprises.”

[ Read also: OpenShift and Kubernetes: What’s the difference? ]

6. Container and Kubernetes security

Duan from NeuVector notes that automation is increasingly vital for security throughout the CI/CD pipeline and Kubernetes itself. “Approaches to security that rely on manual configurations can also slow down the pipeline, or even bring it to a standstill,” he says.

Duan points to two specific automated Kubernetes security measures as examples: End-to-end vulnerability management and implementing security policy as code.

“In the former, build, registry, admission controls, and run-time scanning can and should be automated and free of the need to manually intervene,” Duan says. “As for the latter, one of the biggest trends in Kubernetes automation is infrastructure as code, a concept that should also be applied to security automation.”

Automating security workflows – or pursuing a “security-policy-as-code” approach – is best borne out of collaboration among developers, DevOps, and security pros. CRDs again come into play here.

“Define security policies which can be declared as code early in the development process,” Duan says. “Define policy-as-code by creating CRDs, and review, test, and manage these security policies through a standard code management process. Leveraging security policies as code enables teams to define the ‘allowed behavior’ of their application in production.”

“A great leap forward” for Kubernetes automation: Operators

Again, the very term “workflow” suggests some complexity.

“Not to be confused with a quick decision or piece of data like ‘Is it hot outside?’ or ‘What is the weather in Alpharetta, Georgia?’, workflows need to wait for steps or actions to take place and orchestrate,” says Lachhman from Harness.

Kubernetes Operators have been a boon in this regard. (Get a crash course on Operators with these two articles: How to explain Kubernetes Operators in plain English, and Kubernetes Operators: 4 facts to know.)

“The great leap forward for workflows on Kubernetes is the introduction and adoption of the Operator Model,” Lachhman says. “Operators pulling together Controllers and Custom Resource Definitions allows for operations and tasks to be run in an order and be reacted to. From workflows that benefit the cluster, [such as] certificate rotation, to workflows that are application infrastructure specific, like the Kafka Operator, cluster and application infrastructure workflows are now feasible in Kubernetes.”

You can find a bunch of Operators shared by the Kubernetes community at OperatorHub.io. You can also write your own; Lachhman notes that this favors developers, especially those with Go knowledge. It also helps to have some Kubernetes experience, but the ecosystem (and the help it can provide) is growing quickly. The Operator SDK is a good example.

“The skills required to create an Operator more of a development focus, since you will be writing a Controller in Go. For software engineers who have worked in event handlers before, writing a controller is not a problem,” Lachhman says. “The surrounding ecosystem – e.g build, deploy, and test – does take some Kubernetes-specific skill sets. There are specific ways to build an Operator, so reaching for an SDK is not a problem and tooling is catching up.”

[ Get the free eBook: O’Reilly: Kubernetes Operators: Automating the Container Orchestration Platform. ]