In 2017 a top tier global financial institution was rolling out Kubernetes, but needed help with authentication in their highly regulated deployment. They turned to Tremolo Security's OpenUnison to help with their authentication, management of service accounts, and multi-tenancy using Namespace as a Service. This case study will walk through what issues the financial institution had, and how they were able to use OpenUnison to solve them.

Authentication with Active Directory

The first issue the financial institution needed to solve was integrating Kubernetes with Active Directory. Active Directory uses the LDAP protocol for most interactions and requires authenticating systems to know how to interact with Active Directory via LDAP. Kubernetes only knows how to integrate with certificates or an OpenID Connect identity provider for authentication, so they needed an additional solution. OpenUnison was able to quickly tie Kubernetes with Active Directory via LDAPS, allowing for their users to authenticate using their enterprise authentication system. This cut down on their regulatory requirements and gave developers a better user experience then having to manage separate kubectl configuration files.

In addition to securing the login with Active Directory, the financial institution didn't want to have to distribute a kubectl configuration file to hundreds of developers and administrators around the globe. Distributing a centralized configuration file can be both a security and a support nightmare. It can be a security nightmare, because when users have an issue doing things the "secure" way, they'll often look to bypass that security. It's a support nightmare because when there's a missing entry in the centralized configuration it generates additional support tickets that need to be addressed. OpenUnison's ability to generate a kubectl configuration file based on just kubectl commands for both Windows and Linux/macOS allowed them to onboard developers and administrators quickly without worrying about distributing files. An additional benefit was that users could authenticate via their local browser but easily generate configurations that can be run from jump servers in higher security environments.

They saw multiple security and compliance benefits from using OpenUnison as well:

• Short Lived Tokens - OpenUnison's tokens are scoped to one minute, making it less likely that a lost token will be used to compromise a cluster.
• Refresh Scoped to Policy - OpenUnison's token refresh timeouts are scoped to the financial institution timeout policies, making audits easier.
• User Logout - When a user logs out of OpenUnison, they're sessions are destroyed.
• Delete User Sessions for Quick Logout - Administrators can quickly logout a user by destroying their session objects via the kubernetes API or using kubectl.

Finally, while they evaluated other open source solutions, they needed a solution that provided a commercial support contract. Most of the open source solutions for Kubernetes don't have commercial support, making their use more difficult from a compliance stand point in addition to the additional work the financial institution would have needed to do in order to integrate these systems with their clusters.

Once the financial institution was able to gain control of how to manage API access to their Kubernetes clusters, they next needed to tackle access to their cluster's management web applications.

Kubernetes Cluster Management Applications

Kubernetes cluster security is more then just the API. The financial institution needed the ability to integrate their enterprise authentication into additional applications such as the Kubernetes Dashboard, Kiali, Grafana, and other applications running on their clusters. OpenUnison not only provided an SSO capability via OpenID Connect, but also provided a secure way to integrate id_token injection for applications like Kiali and the Dashboard that don't know how to use OpenID Connect but need to integrate with the API server securely.

Applications such as Kiali and the dashboard interact with the Kubernetes API server on the user's behalf. The "getting started" documentation for these projects often refer to providing high levels of capabilities for the project's service accounts, or generating tokens for individual users. This broke both their security and compliance frameworks. It also was a support headache to generate tokens, as someone with access had to respond to a ticket to generate the token and track its usage. OpenUnison combined the functionality of the Kubernetes identity provider with an OAuth2-Proxy style reverse proxy in a single package. Now dashboard applications could securely interact with the API server on behalf of the user without an additional component install. Using OpenUnison allowed the financial institution to provide a better user experience while providing additional security.

Having securely tackled their user experience, the next major hurdle towards Kubernetes adoption was managing access for their exiting pipelines.

Securing Kubernetes Pipelines

The financial institution had an extensive pipeline system that was used prior to Kubernetes. The security for these pipelines was built off of a service account system integrated with Active Directory that they had invested hundreds of thousands of dollars in to make sure their pipelines were both compliant and secure. When the financial institution used the term "service account", they didn't mean a Kubernetes ServiceAccount token. Their cluster vendor originally recommended using Kubernetes Service Accounts, but that would bypass their regulatory and security requirements. ServiceAccount tokens are not tied to an enterprise identity, making it difficult to track and requiring additional investments to create a secure token generation and distribution system. OpenUnison was able to solve this problem with a configuration update to provide the same API to Active Directory "service accounts" as users:

With OpenUnison in place, the financial institution's pipelines make a RESTful call to OpenUnison with the service account credentials. OpenUnison passes these credentials to Active Directory to validate them. Once validated, JSON is generated that includes an id_token that can be used with the cluster as well as a full kubectl configuration file that can be used with any of the Kubernetes client SDKs. The token is short lived, with the client SDKs renewing the token as needed while the pipeline runs.

OpenUnison allows pipelines to use their existing identities. This saved the financial institution months of custom development and thousands in implementation costs. We blogged previously about how to lock down pipeline access to Kubernetes without Service Account Tokens.

Multi-tenancy and Self Service

Now that the financial institution had integrated both their developers and their pipelines into their clusters using OpenUnison and had built in support for cluster management applications, the next question was how to onboard their developers. They also wanted to make use of Kubernetes' multi-tenancy features so that they could get greater density on their hardware. Using a multi-tenant approach to Kubernetes would both reduce costs directly by reducing the amount of hardware and licenses needed to run their services but also cut down on the need for each application team to have Kubernetes experts to manage clusters. The challenge was how to create individual tenants in a way that was both secure and didn't create an additional burden on management staff.

The financial institution needed to make sure that:

• Tenants were created in a consistent and auditable way.
• Once a tenant was created, the owner of the tenant would be responsible and enabled to onboard new users.
• Each tenant needed additional information for tracking chargeback and other accounting data that had to be collected and tracked at creation.
• Additional roles needed to be provided for support staff to be able to view Pods and delete them
• There needed to be an API to integrate with ongoing Internal Development Portal initiatives

Once again, OpenUnison was able to help. By adding a database to the deployment, the financial institution was able to enable OpenUnison's built in workflow capabilities and self service forms. In addition to automating the onboarding process using a workflow that would generate the namespace, roles, and bindings, they were able to collect the chargeback information needed and add a role for their business needs by updating their helm chart values file.

Once a user requested a new namespace, cluster administrators can approve the request and are then no longer needed to manage access. The requester is now able to manage access directly, removing the burden from the Kubernetes management team. New tenants are then free to do whatever work is needed.

While initially rolled out using OpenUnison's built in portal, the APIs used to execute these workflows are being integrated directly into the financial institution's internal developer platform as well without losing the needed functionality.

"Day Two" Operations

While creating tenants is important, what about "day two" operations? How will users perform tenant maintenance that would not be allowed via kubectl because they're considered cluster level operations, such as expanding on ResourceQuotas? Additionally, they use Istio and needed a way to let namespace owners control when they used upgraded Istio deployments without letting them directly edit labels on their namespaces directly.

They were able to add a form to the OpenUnison portal, along with an accompanying API, to allow namespace owners to update their namespace labels for specific options, such as setting the version of Istio they would use, and requesting new resource limits without requiring escalated privileges for namespace owners inside of a multi-tenant cluster.

Using OpenUnison as a gateway for common operational tasks without getting the operations team involved cut down on support requests and increased user satisfaction by putting the tools they needed at their fingertips.

Getting Started with OpenUnison

After seeing how OpenUnison helps the financial institution manage dozens of multi-tenant clusters across dozens more countries, you might expect that getting started takes hours. It doesn't! Getting started with OpenUnison for cluster authentication with Active Directory, Okta, or almost any other provider only takes a few minutes and a helm values file. OpenUnison will work with your on-premises clusters and your cloud managed clusters equally well, eliminating the burden of access management from your Kubernetes or cloud team. The easiest way to get started is to checkout our getting started documentation. If you need help, you can open an issue on GitHub and we'll be happy to help. Finally, we offer enterprise support too! Drop us a line to find out how Tremolo Security can help secure your infrastructure.