Create a Pod

Pod creation was covered in the previous series. We will look into it from a different angle. We will use the pod definition file created in the previous module to create a pod.

If you do not have any previous experience on the pod, have a look at the previous series Kubernetes in 7 days? In a week !!! –> Creating Pod and come back here.

rajith@k8s-master:~$ vi my-first-pod.yaml 
rajith@k8s-master:~$ cat my-first-pod.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: my-first-pod
  labels:
      type : web
      env : prod  
spec:
    # The container specification section starts here
   containers:
   - name: my-first-pod
     image: nginx
    # The container specification section ends here
rajith@k8s-master:~$ kubectl create -f my-first-pod.yaml 
pod/my-first-pod created
rajith@k8s-master:~$ kubectl get pods 
NAME           READY   STATUS              RESTARTS   AGE
my-first-pod   0/1     ContainerCreating   0          5s
rajith@k8s-master:~$ kubectl get pods 
NAME           READY   STATUS              RESTARTS   AGE
my-first-pod   0/1     ContainerCreating   0          8s
rajith@k8s-master:~$

It is containerising. How do we check the details of it? I mean, to see what is happening in the background. 🤔

rajith@k8s-master:~$ kubectl describe pod my-first-pod 
Name:         my-first-pod
Namespace:    default
Priority:     0
Node:         node-3/192.168.50.13
Start Time:   Mon, 05 Jul 2021 16:09:50 +0000
Labels:       env=prod
              type=web
Annotations:  <none>
Status:       Running
IP:           10.32.0.2
IPs:
  IP:  10.32.0.2
Containers:
  my-first-pod:
    Container ID:   docker://3134b4d480414366cccdb9baa208d0eb618124bdb92c13e2f715b335510f61d9
    Image:          nginx
    Image ID:       docker-pullable://nginx@sha256:47ae43cdfc7064d28800bc42e79a429540c7c80168e8c8952778c0d5af1c09db
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Mon, 05 Jul 2021 16:10:00 +0000
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-d7gtz (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  kube-api-access-d7gtz:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  18s   default-scheduler  Successfully assigned default/my-first-pod to node-3
  Normal  Pulling    17s   kubelet            Pulling image "nginx"
  Normal  Pulled     8s    kubelet            Successfully pulled image "nginx" in 8.907628369s
  Normal  Created    8s    kubelet            Created container my-first-pod
  Normal  Started    8s    kubelet            Started container my-first-pod
rajith@k8s-master:~$ 

Yes, the command kubectl describe pod my-first-pod give the details.

We will break it down further.

The first section containes the metadata information of the pod.

Name:         my-first-pod
Namespace:    default
Priority:     0
Node:         node-3/192.168.50.13
Start Time:   Mon, 05 Jul 2021 16:09:50 +0000
Labels:       env=prod
              type=web
Annotations:  <none>
Status:       Running
IP:           10.32.0.2
IPs:
  IP:  10.32.0.2
Containers:
  my-first-pod:
    Container ID:   docker://3134b4d480414366cccdb9baa208d0eb618124bdb92c13e2f715b335510f61d9
    Image:          nginx
    Image ID:       docker-pullable://nginx@sha256:47ae43cdfc7064d28800bc42e79a429540c7c80168e8c8952778c0d5af1c09db
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Mon, 05 Jul 2021 16:10:00 +0000
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-d7gtz (ro)

Next, it talks about the container details inside the pod. You can see the container name, container ID, image used, Image ID, etc in it.

Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 

Now it talks about the condition of the pod, hopes it is self-explanatory.

Volumes:
  kube-api-access-d7gtz:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s

It is about the volume information. If we mount any persistent volume, it shows here. Then about Toleration, Node-Selectors those are a completely different topic we will talk about it later.

Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  18s   default-scheduler  Successfully assigned default/my-first-pod to node-3
  Normal  Pulling    17s   kubelet            Pulling image "nginx"
  Normal  Pulled     8s    kubelet            Successfully pulled image "nginx" in 8.907628369s
  Normal  Created    8s    kubelet            Created container my-first-pod
  Normal  Started    8s    kubelet            Started container my-first-pod

Now it is about the event, the actual part which we thought of discussing here.

In the event, the first line talks about pod scheduling.

  • It used the ‘default scheduler to schedule the pod. We discussed scheduler in our first series, control plane components.
  • It says it assigned to node-3. How do we verify? Yes, we knew that.
rajith@k8s-master:~$ kubectl get pod  -o wide
NAME           READY   STATUS    RESTARTS   AGE   IP          NODE     NOMINATED NODE   READINESS GATES
my-first-pod   1/1     Running   0          96m   10.32.0.2   node-3   <none>           <none>
rajith@k8s-master:~$ 

It assigned to node-3.

  Normal  Pulling    17s   kubelet            Pulling image "nginx"
  Normal  Pulled     8s    kubelet            Successfully pulled image "nginx" in 8.907628369s

The next two lines says, it is pulling the image “Nginx” from the docker repository. It took 8s to pull the image.

  Normal  Created    8s    kubelet            Created container my-first-pod
  Normal  Started    8s    kubelet            Started container my-first-pod

And the last two line says it created the pod named ‘my-first-pod’, it is started.

That’s it, initially by looking at the big output we thought it is rocket science. It is as simple as that of our first standard textbook.

let us watch the pod creation output step by step

Now let us see the pod creation output at each step. To facilitate that, I am using a simile for loop with a filtering option in Unix.

To see the subsequent ten lines after a string. Use the grep command with this combination. grep -A 10 Events

We will delete and recreate the same pod and see the events. To view only the events, we will form a Unix command combination with a simple for loop.

We will see in action.

rajith@k8s-master:~$ kubectl create -f my-first-pod.yaml 
pod/my-first-pod created
rajith@k8s-master:~$ for i in {1..5} ; do kubectl describe pod my-first-pod |grep -A 10 Events  ; sleep 2  ;done
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  2s    default-scheduler  Successfully assigned default/my-first-pod to node-3     ##<--- It is scheduled on node-3 
  Normal  Pulling    0s    kubelet            Pulling image "nginx"																	##<--- Started pulling images
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  4s    default-scheduler  Successfully assigned default/my-first-pod to node-3
  Normal  Pulling    2s    kubelet            Pulling image "nginx"
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  6s    default-scheduler  Successfully assigned default/my-first-pod to node-3
  Normal  Pulling    4s    kubelet            Pulling image "nginx"
  Normal  Pulled     1s    kubelet            Successfully pulled image "nginx" in 2.880713986s					##<--- Image pull completed.
  Normal  Created    1s    kubelet            Created container my-first-pod													##<---  It created the container
  Normal  Started    1s    kubelet            Started container my-first-pod												    ##<---  And started
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  8s    default-scheduler  Successfully assigned default/my-first-pod to node-3
  Normal  Pulling    6s    kubelet            Pulling image "nginx"
  Normal  Pulled     3s    kubelet            Successfully pulled image "nginx" in 2.880713986s
  Normal  Created    3s    kubelet            Created container my-first-pod
  Normal  Started    3s    kubelet            Started container my-first-pod
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  10s   default-scheduler  Successfully assigned default/my-first-pod to node-3
  Normal  Pulling    8s    kubelet            Pulling image "nginx"
  Normal  Pulled     5s    kubelet            Successfully pulled image "nginx" in 2.880713986s
  Normal  Created    5s    kubelet            Created container my-first-pod
  Normal  Started    5s    kubelet            Started container my-first-pod
rajith@k8s-master:~$ kubectl get pods
NAME           READY   STATUS    RESTARTS   AGE
my-first-pod   1/1     Running   0          19s
rajith@k8s-master:~$ 

Hope this gave you a better idea.

Now let us see some failed events.

 rajith@k8s-master:~$ kubectl delete pod my-first-pod 
pod "my-first-pod" deleted
rajith@k8s-master:~$ kubectl create -f my-first-pod.yaml 
pod/my-first-pod created
rajith@k8s-master:~$ for i in {1..3} ; do kubectl describe pod my-first-pod |grep -A 10 Events  ; sleep 2  ;done
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  2s    default-scheduler  Successfully assigned default/my-first-pod to node-3
  Normal  Pulling    1s    kubelet            Pulling image "nginxx"
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  4s    default-scheduler  Successfully assigned default/my-first-pod to node-3
  Normal  Pulling    3s    kubelet            Pulling image "nginxx"
Events:
  Type     Reason     Age   From               Message
  ----     ------     ----  ----               -------
  Normal   Scheduled  6s    default-scheduler  Successfully assigned default/my-first-pod to node-3
  Normal   Pulling    5s    kubelet            Pulling image "nginxx"
  Warning  Failed     0s    kubelet            Failed to pull image "nginxx": rpc error: code = Unknown desc = Error response from daemon: pull access denied for nginxx, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
  Warning  Failed     0s    kubelet            Error: ErrImagePull
  Normal   BackOff    0s    kubelet            Back-off pulling image "nginxx"
  Warning  Failed     0s    kubelet            Error: ImagePullBackOff
rajith@k8s-master:~$ 

We executed the same step but, in the third occurrence, the image pull failed this time.😫

What happened? 🤔

Have a close look at the image which it is trying to pull. 🧐

  Warning  Failed     4s    kubelet            Failed to pull image "nginxx": rpc error: code = Unknown desc = Error response from daemon: pull access denied for nginxx, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
  Warning  Failed     4s    kubelet            Error: ErrImagePull

It is “nginxx” docker repository does not contain such images. That is the reason for the failure. It tried pulling the image, but the image is not available in the repository.

I modified our pod definition file to create this issue. 😜

rajith@k8s-master:~$ cat my-first-pod.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: my-first-pod
  labels:
      type : web
      env : prod  
spec:
    # The container specification section starts here
   containers:
   - name: my-first-pod
     image: nginxx							##<-- See the image name 
    # The container specification section ends here
rajith@k8s-master:~$ 

Let us see another scenario

I corrected the pod definition file and recreated the pod.

Let us see what is happening?

rajith@k8s-master:~$ kubectl delete pod my-first-pod 
pod "my-first-pod" deleted
rajith@k8s-master:~$ vi my-first-pod.yaml 
rajith@k8s-master:~$ kubectl create -f my-first-pod.yaml 
pod/my-first-pod created
rajith@k8s-master:~$ for i in {1..5} ; do kubectl describe pod my-first-pod |grep -A 10 Events  ; sleep 2  ;done
Events:
  Type     Reason            Age              From               Message
  ----     ------            ----             ----               -------
  Warning  FailedScheduling  1s (x2 over 3s)  default-scheduler  0/4 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate, 3 node(s) were unschedulable.
Events:
  Type     Reason            Age              From               Message
  ----     ------            ----             ----               -------
  Warning  FailedScheduling  3s (x2 over 5s)  default-scheduler  0/4 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate, 3 node(s) were unschedulable.
Events:
  Type     Reason            Age              From               Message
  ----     ------            ----             ----               -------
  Warning  FailedScheduling  5s (x2 over 7s)  default-scheduler  0/4 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate, 3 node(s) were unschedulable.
Events:
  Type     Reason            Age              From               Message
  ----     ------            ----             ----               -------
  Warning  FailedScheduling  7s (x2 over 9s)  default-scheduler  0/4 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate, 3 node(s) were unschedulable.
Events:
  Type     Reason            Age               From               Message
  ----     ------            ----              ----               -------
  Warning  FailedScheduling  9s (x2 over 11s)  default-scheduler  0/4 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate, 3 node(s) were unschedulable.
rajith@k8s-master:~$ 

Now it is failing in the first step itself. It is unable to schedule the pod anywhere.

Why? Yes, we can not schedule anything on master node. But what about the other three nodes?

rajith@k8s-master:~$ kubectl get nodes
NAME         STATUS                        ROLES                  AGE   VERSION
k8s-master   Ready                         control-plane,master   34d   v1.21.1
node-1       Ready,SchedulingDisabled      <none>                 34d   v1.21.1
node-2       NotReady,SchedulingDisabled   <none>                 34d   v1.21.1
node-3       Ready,SchedulingDisabled      <none>                 34d   v1.21.1
rajith@k8s-master:~$ 

I manually made the other node non-schedulable.

Try scheduling it again

rajith@k8s-master:~$ kubectl get nodes
NAME         STATUS                        ROLES                  AGE   VERSION
k8s-master   Ready                         control-plane,master   34d   v1.21.1
node-1       Ready                         <none>                 34d   v1.21.1
node-2       NotReady,SchedulingDisabled   <none>                 34d   v1.21.1
node-3       Ready,SchedulingDisabled      <none>                 34d   v1.21.1
rajith@k8s-master:~$ for i in {1..5} ; do kubectl describe pod my-first-pod |grep -A 10 Events  ; sleep 2  ;done
Events:
  Type     Reason            Age                 From               Message
  ----     ------            ----                ----               -------
  Warning  FailedScheduling  61s (x13 over 12m)  default-scheduler  0/4 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate, 3 node(s) were unschedulable.
  Warning  FailedScheduling  49s                 default-scheduler  0/4 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate, 1 node(s) had taint {node.kubernetes.io/unschedulable: }, that the pod didn't tolerate, 2 node(s) were unschedulable.
  Normal   Scheduled         38s                 default-scheduler  Successfully assigned default/my-first-pod to node-1                 ##<-- It scheduled the pod on node-1.
  Normal   Pulling           37s                 kubelet            Pulling image "nginx"
  Normal   Pulled            34s                 kubelet            Successfully pulled image "nginx" in 2.946054283s
  Normal   Created           34s                 kubelet            Created container my-first-pod
  Normal   Started           34s                 kubelet            Started container my-first-pod

I made node-1 ready. Immediately after that, it scheduled the pod on node-1. See the events.

Our next module is on deployment, Click here