HA Kubernetes cluster with Vagrant+CoreOS+Ansible, Part 5

In the last part, we have managed to deploy the Kubernetes core components using bootbuke and be able to access to cluster using the dashboard. We’ll now use an example to demonstrate the deployment of services and expose the services using Ingress and route traffic by domain name.

In this example I’ll deploy two services using nginx, and expose one of the service using the domain nginx1.tectusdreamlab.com and the other using nginx2.tectusdreamabl.com. I’ll enable TLS and automatically redirect http to https.

Prepare the certs and keys

To get started we need have the certs and keys in order to enable TLS, we’ll use the same root CA and use the same cert for both services by adding both domains in the san.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

# START INGRESS
ingress_key = OpenSSL::PKey::RSA.new(2048)
ingress_public_key = ingress_key.public_key

ingress_cert = signTLS(is_ca:              false,
                       subject:            "/C=SG/ST=Singapore/L=Singapore/O=tds/OU=IT/CN=nginx.tectusdreamlab.com",
                       issuer_subject:     "/C=SG/ST=Singapore/L=Singapore/O=bootkube/OU=IT/CN=kube-ca",
                       issuer_cert:        kube_cert,
                       public_key:         ingress_public_key,
                       ca_private_key:     kube_key,
                       key_usage:          "digitalSignature,keyEncipherment",
                       extended_key_usage: "serverAuth,clientAuth",
                       san:                "DNS:nginx1.tectusdreamlab.com,DNS:nginx2.tectusdreamlab.com")
ingress_key_file= File.new("provisioning/roles/example/files/tls/server.key", "wb")
ingress_key_file.syswrite(ingress_key.to_pem)
ingress_key_file.close
ingress_cert_file = File.new("provisioning/roles/example/files/tls/server.crt", "wb")
ingress_cert_file.syswrite(ingress_cert.to_pem)
ingress_cert_file.close
# END INGRESS

Writing the ingress specs

In the ingress spec, we’ll enable redirect and specify domain-based routing to route requests coming with nginx1.tectusdreamlab.com to service nginx1 and nginx2.tectusdreamlab.com to service nginx2.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: tds-ingress
  namespace: tds-cloud
  annotations:
    ingress.kubernetes.io/ssl-redirect: "true"
    ingress.kubernetes.io/use-port-in-redirects: "true"
    kubernetes.io/ingress.class: "tds"
spec:
  tls:
  - hosts:
    - nginx1.tectusdreamlab.com
    secretName: nginx-ingress-tls-secret
  - hosts:
    - nginx2.tectusdreamlab.com
    secretName: nginx-ingress-tls-secret
  rules:
  - host: nginx1.tectusdreamlab.com
    http:
      paths:
      - backend:
          serviceName: nginx1
          servicePort: 80
  - host: nginx2.tectusdreamlab.com
    http:
      paths:
      - backend:
          serviceName: nginx2
          servicePort: 80

Putting things together

We need to define the deployment and service specs for the two services of course, you can find the full content of the example role on my github.

Deploy the example on the cluster by running:

1
2

cd provisioning
ansible-playbook -i ../.vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory --ssh-extra-args='-o StrictHostKeyChecking=no' example.yml -vv

Tips: You have to disable the host key checking to make this work. The inventory file is generated by vagrant automatcially.

After a while, you should see your services up and running in Kubernetes cluster and the ingress created.

Update your /etc/hosts file to add the DNS records:

1
2

172.17.5.201 nginx1.tectusdreamlab.com
172.17.5.201 nginx2.tectusdreamlab.com

You’ll find the two sites accessible from your browser.

Conclusion

As this example demonstrates, Kubernetes can be easily used as a micro-services infrastructure in which each services can be exposed by it’s domain name. The scaling of a service are automatically controlled by Kubernetes, the only thing we need to handle is the scaling of the Kubernetes cluster itself.

Comments: