Jason Soo | Software Engineer

  1. k8s cheat sheet
  2. Useful k8s troubleshooting
  3. Check application logs

What is Kubernetes

23 Oct 2022

Kubernetes (k8s), is an open source platform pioneered by Google, which started as a simple container orchestration tool but has grown into a cloud native platform.

Container orchestration is about managing the lifecycle of containers, particularly in large, dynamic environments. It automates:

  1. deployment
  2. networking
  3. scaling
  4. availability of containerised workloads and services

What is k8s by Ubuntu

Learn k8s basics

Explain k8s in plain english

What is k8s by RedHat

What is k8s by Google

k8s cheat sheet

Installing kubectl on Linux

# https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/
$ sudo snap install kubectl --classic
$ kubectl version --client

Installing kubectl on mac

# https://kubernetes.io/docs/tasks/tools/install-kubectl-macos/
$ brew install kubectl
$ kubectl version --client

Useful Kubernetes troubleshooting

schooner (Kubernetes Custom Resource Definition or CRD at our company) -> deployment -> replicaset -> pods -> containers -> services, accounts, etc.

If the pod is not up, there might be useful error info in describe replicaset or deployment

Also useful to describe pods to find dependent container logs that may show error preventing pod from starting up

// if you've changed AWS envs, this is useful to set your kube config to current env
$ aws eks --region ap-southeast-2 update-kubeconfig --name tap
$ kubectl -n digital get replicaset
// Useful for checking errors in replicaset
$ kubectl -n digital describe replicaset application-review-service
// Can use full name
$ kubectl -n digital describe replicaset application-review-service-xdfae
$ kubectl -n digital get deployments
// Useful for checking errors in deployments
$ kubectl -n digital describe deployments application-review-service
// schooler = CRD (custom resource definitions)
$ kubectl -n digital get schooner
$ kubectl -n digital describe schooner
$ kubectl -n digital describe schooner application-review-service
$ kubectl -n digital get pods
// Useful to find container logs that may show the error preventing pod from starting up
$ kubectl -n digital describe pods application-review-service
$ kubectl -n digital describe pods external-data-lookup
// Eg where application-review-service was failing due to party-db-cluster issue
$ kubectl -n digital logs -f application-review-service-5d9875d667-8dhjb -c digital-party-db-cluster-init
// Eg where external-data-lookup was failing as it could not login to vault using role, invalid role name
$ kubectl -n digital logs -f external-data-lookup-7bdc9b6ff6-78qll -c vault-static-secrets
$ kubectl -n digital logs -f -lapp=external-data-lookup -c vault-static-secrets
// shutdown pod (or application-review-avoka-integration)
$ kubectl scale deploy application-review-service --replicas=0
// restart pod
$ kubectl scale deploy application-review-service --replicas=2
$ kubectl -n digital get jobs
// Delete job (need admin access)
$ kubectl -n digital delete job {job-name}
// Login to pod
$ kubectl -n digital exec -it external-data-lookup-6f74f7c874-8cgdc /bin/sh
// Check Prometheus alerting of artefact
$ kubectl -n digital describe PrometheusRule auth0-tyro-customer-stream-subscribe

Check application logs

$ kubectl -n=digital logs -f -lapp=existing-customer-apply-bff -c existing-customer-apply-bff
$ kubectl -n=digital logs -f -lapp=application-review-service -c application-review-service
$ kubectl -n=digital logs -f -lapp=application-review-avoka-integration -c application-review-avoka-integration
// ARS database schema migration
$ kubectl -n=digital logs -f -lapp=application-review-service-schema-migration -c application-review-service-schema-migration
// Check specific log
kubectl -n=digital logs -f application-review-service-db499b44f-l7wvw -c application-review-service