Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Slides are attached: Kubernetes.pptx

Kubernetes

You can either run all this locally on your machine or on k8s-shell. If you ssh, please setup port forwarding. Please use a random port since others might be trying to use the same port as well. The port has to be greater than 1024 and less than 65535. Just pick something random and remember it

Code Block
ssh -L <port>:localhost:<port> username@k8s-shell

If you see bind [127.0.0.1]:<port>: Address already in use  this means you have already something running on your local machine on that port and you will need to change it to something else.

Kubernetes Access

Access to the cluster is controlled using SUSE Rancher. You can login with your NCSA credentials. You should see a list of all the clusters you have access, you want to download the config for k8s-class

Code Block
languagebash
titleSetup KubeConfig
export KUBECONFIG=~/k8s-class.yaml
kubectl version --short
Flag --short has been deprecated, and will be removed in the future. The --short output will become the default.
Client Version: v1.24.15
Kustomize Version: v4.5.4
Server Version: v1.24.9

...

Code Block
languagebash
kubectl

Commands we will use are:

commanddescription
applyApply a configuration to a resource by file name or stdin
createCreate a resource from a file or from stdin

delete

Delete resources by file names, stdin, resources and names, or by resources and label selector
execExecute a command in a container
getDisplay one or many resources
logsPrint the logs for a container in a pod
port-forwardForward one or more local ports to a pod
api-resourcesPrint the supported API resources on the server (also shows abbreviations)
describeShow details of a specific resource or group of resources
versionPrint the client and server version information
configModify kubeconfig files

On k8s-shell I added some aliases to minimize the typing

aliascommand
kkubectl
kakubectl apply
kgkubectl get

kd

kubectl delete

Nodes (no)

website: https://kubernetes.io/docs/concepts/architecture/nodes/

Time to see how many nodes we have. How do we get the list of the nodes

Code Block
languagebash
titlelist nodes
collapsetrue
> kubectl get nodes
NAME                        STATUS   ROLES               AGE    VERSION
k8s-class-controlplane-01   Ready    controlplane,etcd   128m   v1.24.9
k8s-class-controlplane-02   Ready    controlplane,etcd   89m    v1.24.9
k8s-class-controlplane-03   Ready    controlplane,etcd   86m    v1.24.9
k8s-class-worker-01         Ready    worker              126m   v1.24.9
k8s-class-worker-02         Ready    worker              126m   v1.24.9
k8s-class-worker-03         Ready    worker              90m    v1.24.9

Namespace (ns)

website: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/## Namespace

List all namespaces

Code Block
language

...

bash
titlelist namespaces
collapsetrue
> kubectl get namespace
NAME                          STATUS   AGE
cattle-fleet-system           Active   128m

...


cattle-impersonation-system   Active   129m

...


cattle-system                 Active   129m

...


cinder-csi                    Active   125m

...


default                       Active   130m

...


kube-node-lease               Active   130m

...


kube-public                   Active   130m

...


kube-system                   Active   130m

...


local                         Active   128m

...


metallb-system                Active   122m

...


nfs-taiga                     Active   122m

...


traefik                       Active   122m

```Create a namespace<details>
  <summary>kubectl .......</summary>
 kubectl create namespace kooper
</details>Show namespace file<details>
  <summary>kubectl .......</summary>
 kubectl get ns kooper -o yaml 
</details>```yaml
apiVersion: v1
kind: Namespace
metadata:
  annotations:
   

Code Block
languagebash
titlecreate namespaces
collapsetrue
> kubectl create namespace kooper
namespace/kooper created

Show namespace object in yaml

Code Block
languagebash
titleshow namespaces
collapsetrue
> kubectl get namespace kooper -o yaml
apiVersion: v1
kind: Namespace
metadata:
  annotations:
    cattle.io/status: '{"Conditions":[{"Type":"ResourceQuotaInit","Status":"True","Message":"","LastUpdateTime":"2023-06-25T13:35:36Z"},{"Type":"InitialRolesPopulated","Status":"True","Message":"","LastUpdateTime":"2023-06-25T13:35:36Z"}]}'

...


    lifecycle.cattle.io/create.namespace-auth: "true"

...


  creationTimestamp: "2023-06-25T13:35:35Z"

...


  finalizers:

...


  - controller.cattle.io/namespace-auth

...


  labels:

...


    kubernetes.io/metadata.name: kooper

...


  name: kooper

...


  resourceVersion: "296036"

...


  uid: 6c4092ca-4024-4814-a2c0-5739777ea3aa

...


spec:

...


  finalizers:

...


  - kubernetes

...


status:

...


  phase: Active

...

Lets simplify this, and remove some cruft

Code Block
languageyml
titleshow namespaces
collapsetrue
apiVersion: v1
kind: Namespace
metadata:
  name: kooper

Lets make our life easier, and use this namespace by default```bash
kubectl config

Code Block
languagebash
titleuse namespace
> kubectl config set-context --current --namespace=kooper
Context "k8s-class" modified.

Pods (po)

website: https://kubernetes.io/docs/concepts/workloads/pods/

Simple yaml file for pods (make sure you delete the namespace: kooper

...

titlelist nodes

...

Content by Label
showLabelsfalse
max5
spacesNCSASoftware
showSpacefalse
sortmodified
reversetrue
typepage
cqllabel = "kb-how-to-article" and type = "page" and space = "NCSASoftware"
labelskb-how-to-article

):

Code Block
languageyml
titlemy first pod
apiVersion: v1
kind: Pod
metadata:
  name: cowsay
  namespace: kooper
spec:
  restartPolicy: Never
  containers:
  - name: cowsay
    image: rancher/cowsay

Check the output (log) of the pods:

Code Block
languagebash
titlelog from pod
collapsetrue
> kubectl logs pod/cowsay​
  _
<   >
 -
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     || 

Before we can change anything, we first need to delete that pod

Code Block
languagebash
titledelete pod
collapsetrue
> kubectl delete pod/cowsay
pod "cowsay" deleted

Make sure the cow goes Mooooo

Code Block
languageyml
titlecows say mooo
collapsetrue
apiVersion: v1
kind: Pod
metadata:
  name: cowsay
  namespace: kooper
spec:
  restartPolicy: Never
  containers:
  - name: cowsay
    image: rancher/cowsay
    command:
    - cowsay
    - Mooooo

A pod can be multiple containers

Code Block
languageyml
titlemultiple containers
collapsetrue
apiVersion: v1
kind: Pod
metadata:
  name: cowsay
  namespace: kooper
spec:
  restartPolicy: Never
  containers:
  - name: cowsay
    image: rancher/cowsay
    command:
    - cowsay
    - Mooooo
  - name: whalesay
    image: docker/whalesay
    command:
    - cowsay
    - Howdy

Listing the output of the containers

Code Block
languagebash
titlelog of multiple containers
collapsetrue
> kubectl logs cowsay
Defaulted container "cowsay" out of: cowsay, whalesay
 ________
< Mooooo >
 --------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||
> kubectl logs cowsay -c whalesay
 _______
< Howdy >
 -------
    \
     \
      \
                    ##        .
              ## ## ##       ==
           ## ## ## ##      ===
       /""""""""""""""""___/ ===
  ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ /  ===- ~~~
       \______ o          __/
        \    \        __/
          \____\______/ 

Debugging if somethings goes wrong

Code Block
languagebash
titleDebug pods
> kubectl describe pod cowsay
Name:         cowsay
Namespace:    kooper
Priority:     0
Node:         k8s-class-worker-01/192.168.2.42
Start Time:   Sun, 25 Jun 2023 09:52:44 -0500
Labels:       <none>
Annotations:  <none>
Status:       Failed
...
Events:
  Type     Reason     Age   From               Message
  ----     ------     ----  ----               -------
  Normal   Scheduled  5s    default-scheduler  Successfully assigned kooper/cowsay to k8s-class-worker-01
  Normal   Pulling    5s    kubelet            Pulling image "rancher/cowsay"
  Normal   Pulled     5s    kubelet            Successfully pulled image "rancher/cowsay" in 321.252759ms
  Normal   Created    5s    kubelet            Created container cowsay
  Warning  Failed     5s    kubelet            Error: failed to start container "cowsay": Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "Mooooo": executable file not found in $PATH: unknown

Storage

Storage Classes (sc)

website: https://kubernetes.io/docs/concepts/storage/storage-classes/

List the storage classes available in this cluster

Code Block
languagebash
titlelist storage class
collapsetrue
> kubectl get sc
NAME                   PROVISIONER                                               RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
csi-cinder-sc-delete   cinder.csi.openstack.org                                  Delete          Immediate           true                   18h
csi-cinder-sc-retain   cinder.csi.openstack.org                                  Retain          Immediate           true                   18h
nfs-taiga (default)    cluster.local/nfs-taiga-nfs-subdir-external-provisioner   Retain          Immediate           true                   17h

Persistant Volume (pv)

website: https://kubernetes.io/docs/concepts/storage/volumes/

List all volumes

Code Block
languagebash
titlelist pv
collapsetrue
> kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM             STORAGECLASS   REASON   AGE
pvc-4ff2a331-466b-41c4-9c4a-18734c70badd   128Mi      RWO            Retain           Bound    traefik/traefik   nfs-taiga               18h

Persistant Volume Claims (pvc)

website: https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims

Create a PVC

Code Block
languageyml
titlePVC
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: kooper-pvc
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi

Apply the PVC and check volumes, why is there no new volume?

Code Block
languagebash
titleapply pvc
collapsetrue
> kubectl apply -f pvc1.yaml
persistentvolumeclaim/kooper-pvc created
> kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM             STORAGECLASS   REASON   AGE
pvc-4ff2a331-466b-41c4-9c4a-18734c70badd   128Mi      RWO            Retain           Bound    traefik/traefik   nfs-taiga               18h
> kubectl delete -f pvc1.yaml
persistentvolumeclaim "kooper-pvc" deleted

Delete and modify volume claim to create a volume on nfs-taiga

Code Block
languageyml
titlePVC
collapsetrue
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: kooper-pvc
spec:
  storageClassName: nfs-taiga
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi

Mount volume in container (make sure you replace the claimName with your own claim)

Code Block
languageyml
titlePod with PVC
collapsetrue
apiVersion: v1
kind: Pod
metadata:
  name: pod-with-pvc
spec:
  restartPolicy: Never
  volumes:
  - name: data
    persistentVolumeClaim:
      claimName: kooper-pvc
  containers:
  - name: ubuntu
    image: ubuntu
    command: ["sleep", "3600"]
    volumeMounts:
    - name: data
      mountPath: /data

and edit data in volume

Code Block
languagebash
titleedit container
collapsetrue
> kubectl exec -ti pod-with-pvc -- bash
root@pod-with-pvc:/#

On k8s-shell you can go to /taiga/<folder> and see the same data

Pods cntd

website: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-conditions

A simple webserver using the previous created pvc, liveness and readiness

Code Block
languageyml
titlenginx deployment
collapsetrue
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  volumes:
  - name: data
    persistentVolumeClaim:
      claimName: kooper-pvc
  initContainers:
  - name: create-index-html
    image: busybox:1.28
    command:
      - sh
      - -c
      - echo "<html><body>Created $(date)</body></html>">/data/index.html
    volumeMounts:
    - name: data
      mountPath: /data
  containers:
  - name: nginx
    image: nginx:1.14.2
    ports:
    - containerPort: 80
    livenessProbe:
      httpGet:
        port: 80
    readinessProbe:
      httpGet:
        port: 80
      initialDelaySeconds: 5
      periodSeconds: 10
    volumeMounts:
    - name: data
      mountPath: /usr/share/nginx/html

Forward a port so we can check it out

Code Block
languagebash
titleport forward
collapsetrue
> kubectl port-forward pod nginx 8888:80Forwarding from 127.0.0.1:8888 -> 80
Forwarding from [::1]:8888 -> 80

Deployment (deploy)

website: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/

Take the previous pod and convert to a deployment

Code Block
languageyml
titlenginx deployment
collapsetrue
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    # copied from before
    spec:
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: kooper-pvc
      initContainers:
      - name: create-index-html
        image: busybox:1.28
        command:
          - sh
          - -c
          - echo "<html><body>Created $(date)</body></html>">/data/index.html
        volumeMounts:
        - name: data
          mountPath: /data
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
        livenessProbe:
          httpGet:
            port: 80
        readinessProbe:
          httpGet:
            port: 80
          initialDelaySeconds: 5
          periodSeconds: 10
        volumeMounts:
        - name: data
          mountPath: /usr/share/nginx/html

Scaling the deployment

Code Block
languagebash
titleport forward
collapsetrue
> kubectl scale deployment/nginx --replicas=0deployment.apps/nginx scaled

Service (svc)

website: https://kubernetes.io/docs/concepts/services-networking/service/

Exposing pod(s) with a service. This will act as a load balancer

Code Block
languageyml
titlenginx service
collapsetrue
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  selector:
    app: nginx
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

We can check if this work by port forwarding

Ingress (ing)

website: https://kubernetes.io/docs/concepts/services-networking/ingress/

Final step is to map the hostname to the ingress, to the service, to the pod.

Code Block
languageyml
titlenginx ingress
collapsetrue
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx
spec:
  rules:
  - host: kooper.k8s-class.ncsa.cloud
    http:
      paths:
      - path: /
        pathType: ImplementationSpecific
        backend:
          service:
            name: nginx
            port:
              number: 80

We can now visit all of our work at https://kooper.k8s-class.ncsa.cloud/

Helm

website: https://helm.sh

Create Chart

Create the helm chart

Code Block
languagebash
titlecreate helm chart
collapsetrue
> helm create nginx
Creating nginx 

What is the version of the helm chart?

Code Block
languageyml
titleversion in chart.yaml
collapsetrue
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.0

# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "1.16.0"

Install Chart

Customize the helm chart using some values

Code Block
languageyml
titlecustomization
collapsetrue
replicaCount: 3

ingress:
  enabled: true
  hosts:
    - host: kooper.k8s-class.ncsa.cloud
      paths:
        - path: /
          pathType: ImplementationSpecific

Check helm template

Code Block
languagebash
titlecreate helm chart
collapsetrue
> helm template . -f custom.yaml
---
# Source: nginx/templates/serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
...
spec:
  containers:
    - name: wget
      image: busybox
      command: ['wget']
      args: ['release-name-nginx:80']
  restartPolicy: Never

Installing helm chart

Code Block
languagebash
titleinstall helm chart
collapsetrue
> helm install nginx . -f custom.yaml
NAME: nginx
LAST DEPLOYED: Sun Jun 25 22:45:54 2023
NAMESPACE: kooper
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
  http://kooper.k8s-class.ncsa.cloud/

Listing helm chart

Code Block
languagebash
titlelist helm chart
collapsetrue
> helm ls
NAME 	NAMESPACE	REVISION	UPDATED                                	STATUS  	CHART      	APP VERSION
nginx	kooper   	1       	2023-06-25 22:45:54.045250697 -0500 CDT	deployed	nginx-0.1.0	1.16.0

Storage

add storage to the helm chart (need to modify 2 files)

Code Block
languageyml
titlepvc.yaml
collapsetrue
{{- if and .Values.persistence.enabled (not .Values.persistence.existingClaim) }}
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: {{ template "nginx.fullname" . }}-data
  labels:
    {{ include "nginx.labels" . | nindent 4 }}
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: {{ .Values.persistence.size | quote }}
  {{- if .Values.persistence.storageClass }}
  storageClassName: {{ .Values.persistence.storageClass }}
  {{- end -}}
{{- end }}

Extend Chart (database)

We will have to do a few steps

  • update the chart to add a depende
  • update values.yaml with some defaults
  • update deployment to wait for the database

Update chart.yaml

Code Block
languageyml
titleChart.yaml
collapsetrue
# Dependencies for chart. Some of the dependencies are only installed if they
# are enabled.
dependencies:
- name: postgresql
  version: ~8.9
  repository:  https://raw.githubusercontent.com/bitnami/charts/pre-2022/bitnami
  condition: postgres.enabled

Update values.yaml with defaults:

Code Block
languageyml
titlevalues.yaml
collapsetrue
# start with the name of the chart (or alias)
postgresql:
  ## enabled should the postgresql chart be installed as well. If not make
  ## sure that the other values are provied to get access to the database.
  enabled: true

  ## postgresqlUsername is the initial user added to the system. 
  # postgresqlUsername: postgres

  ## postgresqlPassword the password for postgresqlUsername
  postgresqlPassword: supersecret

  # need more space
  persistence:
    size: 20Gi

And make sure we have a simple check in the deployment to see if database is ready

(thanks Sara Lambert for this trick, see https://github.com/ncsa/checks)

Code Block
languageyml
titledeployment.yaml
collapsetrue
apiVersion: apps/v1
kind: Deployment
metadata:
...
spec:
  ...
  template:
    ...
    spec:
      ...
      initContainers:
        - name: check-postgresql
          image: "ncsa/checks:1.0.1"
          env:
            - name: PGHOST
              value: {{ .Release.Name }}-postgresql
            - name: PGPORT
              value: "5432"
            - name: PGUSER
              value: {{ .Values.postgresql.postgresqlUsername | default "postgres" | quote }}
            - name: PGPASSWORD
              valueFrom:
                secretKeyRef:
                  name: {{ .Release.Name }}-postgresql
                  key: postgresql-password

Page properties
hiddentrue
Related issues