Handling authorization with Kafka

IAM Cloud Security AWS architecture Kafka Knowit Cloud & Architecture Authorization

Vivek Mehtani 26.03.2023

Background

Zero trust security works on the principles of never trust and always verify. Organizations are applying these principles in different areas, but there are still a few areas that require more attention in the security space to implement access control, one of them is a very popular and widely used messaging service such as Kafka where there are various solutions available for authorization. The risk footprint of this service is quite high as it is a backbone of processing huge data sets for many organizations and securing it for illegitimate access should not be ignored as the data stored in many of these topics would be highly critical and therefore should be tightly governed. There are many similar offerings provided by the cloud providers which are quite well managed and tightly integrated with secure IAM, for e.g. with AWS cloud services like Kinesis Data Streams and even the AWS managed version of Kafka the authorization is quite well integrated. So, to begin we will go through some of the ways by which we can achieve authorization in Kafka.

Kafka Authorization mechanism

After the client has been authenticated and a security principle has been identified (Kafka provides various authentication mechanisms, see the reference links below), the next step is to authorize the client request against the specified resource (topic). This could be achieved either by using the Kafka provided authorization API or by a custom created authorization API, one such implementation is using OPA (Open Policy Agent) which is also discussed below, and we could also create our own class based on how we would like to handle authorization in our own custom way.

Kafka provided Default Authorization API

The authorization API is plugged in to the broker in server.properties file with property name authorizer.class.name, the class implementing the authorization control is kafka.security.authorizer.AclAuthorizer, which looks into the ACls(Access control list) for authorization that are stored in zookeeper.

Open source offerings

OPA (Open Policy Agent) implementation

OPA is an open source and CNCF graduated project in the security & compliance space. In a nutshell OPA is a general-purpose policy engine that unifies policy enforcement across the stacks. OPA specifies policy as code with REGO language. In this specific use case of Authorization, OPA implements the plugin class similar to Kafka's provided plugin class but instead of ACL it forwards the logic of access control to OPA where policies will be defined and evaluated in REGO language for access control and verification.

Example implementation: https://www.openpolicyagent.org/docs/latest/kafka-authorization/

Kafka API plugin implementation: https://github.com/StyraInc/opa-kafka-plugin

Cloud Managed service

AWS MSK (Managed Kafka Service)

AWS IAM access control policies could be used for the authorization, by attaching the authorization control policies corresponding to the client. In an authorization policy, you specify which actions to allow or deny for the role. If your client is on an Amazon EC2 instance, associate the authorization policy with the IAM role for that Amazon EC2 instance. Alternatively, you can configure your client to use a named profile, and then you associate the authorization policy with the role.