Cloud Providers are a powerful concept in Kubernetes that provide cloud specific extensions. On AWS, Kubernetes Services of type LoadBalancer are a good example of this. We have documentation on LoadBalancer Services for the Giant Swarm platform. These should also work for most vanilla Kubernetes clusters. In this post, we’ll share some use-cases we’ve seen from our customers. This includes some advanced use-cases using Annotations that were not documented outside code comments until recently.
I want to expose a single Service
We’ll start with exposing a single service. If you’re not familiar with Kubernetes Services, see our previous blog post.
Below is a standard Kubernetes Service that exposes Pods labeled with the app helloworld. The type LoadBalancer means that an AWS ELB (Elastic LoadBalancer) will be created if the cluster has the AWS Cloud Provider enabled.
apiVersion: v1
kind: Service
metadata:
name: helloworld
labels:
app: helloworld
spec:
ports:
- port: 80
targetPort: 8080
selector:
app: helloworld
type: LoadBalancer
Describing the Service shows the ELB has been provisioned.
kubectl describe service helloworld
...
Name: helloworld
Type: LoadBalancer
LoadBalancer Ingress: a8f60781822dc11e89f4a02a815eb168-1647443905.eu-central-1.elb.amazonaws.com
...
I want my Service to be private
If your service is internal and should only be accessed within your AWS VPC (Virtual Private Cloud) you can set the scheme to internal by using an Annotation. Access from multiple VPCs is also possible if you use VPC peering.
metadata:
name: internal
labels:
app: internal
annotations:
service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
I want to use a certificate from Amazon Certificate Manager
Many services need to be encrypted and Amazon Certificate Manager integrates with ELBs. It has a couple of major benefits.
- It’s free, you only pay for the ELB resources you use.
- The certificate renewal is automated which helps keeps the service secure and reduces manual work.
Again the configuration is managed using an Annotation on the Service resource.
metadata:
annotations:
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:eu-central...
I want to have access logs for my ELB
Writing access logs to an S3 bucket is a standard feature of ELBs. For LoadBalancer Services this can also be configured using Annotations.
metadata:
annotations:
service.beta.kubernetes.io/aws-load-balancer-access-log-enabled: true
# The interval for publishing the access logs (can be 5 or 60 minutes).
service.beta.kubernetes.io/aws-load-balancer-access-log-emit-interval: 60
service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-name: my-logs-bucket
service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-prefix: logs/prod
I want to expose multiple Services cost efficiently
If you’re using a microservices architecture or have a cluster running several workloads it’s likely you’ll need to expose multiple services.
You can create a LoadBalancer Service for each of your services. However, each ELB will cost around $22 USD a month (excluding traffic costs). If you have low traffic and internal services that can quickly get expensive.
A Kubernetes Ingress is a collection of rules for inbound connections to Services. You need to run an Ingress Controller to manage your Ingress resources. The Nginx Ingress Controller is commonly used but there are other options such as Traefik.
You can use a LoadBalancer Service to expose your Ingress Controller. This is much more cost efficient as a single ELB can serve many Ingress resources. A feature of the Giant Swarm platform is we provide an Ingress Controller and ELB automatically for each cluster.
spec:
selector:
app: ingress-nginx
In this example we use the app label to expose the Ingress Controller Pods.
Advanced Use-Cases
We’ll end with some advanced use-cases we’ve seen.
I want to run multiple Ingress Controllers
As mentioned earlier using an Ingress Controller with a LoadBalancer Service is a great way of exposing multiple services. However, maybe you need to expose both external and internal services. You can use the Ingress Class Annotation on your Ingress resources and Ingress Controllers. This will route the traffic to the correct Ingress Controller and each controller can be exposed using a LoadBalancer Service.
kubernetes.io/ingress.class: "nginx-internal"
I want to use the new NLB support in Kubernetes 1.9
AWS is in the process of replacing ELBs with NLBs (Network Load Balancers) and ALBs (Application Load Balancers). NLBs have a number of benefits over “classic” ELBs including scaling to many more requests. Alpha support for NLBs was added in Kubernetes 1.9. As it’s an alpha feature it’s not yet recommended for production workloads but you can start trying it out.
metadata:
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
There is a great post from Arun Gupta and Micah Hausler on the AWS blog which goes into more detail on NLBs. One thing to watch is that older instance types including the m3 series cannot be registered with NLBs.
Conclusion
This post has shown some of the options with LoadBalancer services. You can read more in our documentation and in the Kubernetes docs. We’d also love to hear from you if you have more use cases we haven’t covered.
Photo by Uggboy / CC BY
Comments (2)