Menu

Dipesh Majumdar

Blog and Paintings

ps not available in docker container with debian os

Solution:

apt-get update && apt-get install procps

Cannot login to amazon ecr

Person providing solution says: That's okay but what's the error dude?

 

Person in trouble says: Well, the error is this:

unknown shorthand flag: 'e' in -e

See 'docker login --help'.

 

Person providing solution says:

When you do a aws ecr get-login, just remove this from the output  

-e none

And then use the resultant output to login. Simple

empower sed to to work on case insensitive strings

This is what you need to do in a macOS:

brew install gnu-sed

The end of the installation looks like this:


==> Pouring gnu-sed-4.7.mojave.bottle.3.tar.gz
==> Caveats
GNU "sed" has been installed as "gsed".
If you need to use it as "sed", you can add a "gnubin" directory
to your PATH from your bashrc like:
PATH="/usr/local/opt/gnu-sed/libexec/gnubin:$PATH"
==> Summary
/usr/local/Cellar/gnu-sed/4.7: 12 files, 562.0KB

 

Now not only author but also AUTHOR will get changed to publish

sed -e 's/author/publish/gI' deploy.yaml | tee edited.yaml

Installing Terraform in macOS

 

First download the software (a zip file) from here - https://www.terraform.io/downloads.html  (for the relevant OS - in this case MAC-OS_

 

dipeshmajumdar$ unzip terraform_0.12.10_darwin_amd64.zip 

Archive:  terraform_0.12.10_darwin_amd64.zip

  inflating: terraform               

MacBook-Pro-van-Dipesh:Software dipeshmajumdar$ ls -ltra

total 143376

-rwxr-xr-x@  1 dipeshmajumdar  staff  55634200 Oct  7 18:15 terraform

-rw-r--r--@  1 dipeshmajumdar  staff  17095466 Oct 17 10:59 terraform_0.12.10_darwin_amd64.zip

drwxr-xr-x+ 57 dipeshmajumdar  staff      1824 Oct 17 11:01 ..

drwxr-xr-x   4 dipeshmajumdar  staff       128 Oct 17 11:02 .

 

Now this file - terraform is there in `/Software so adding same path to .bash_profile

 

Now all set... terraform command should work now as shown below - 

 

 

dipeshmajumdar$ terraform

Usage: terraform [-version] [-help] <command> [args]

 

The available commands for execution are listed below.

The most common, useful commands are shown first, followed by

less common or more advanced commands. If you're just getting

started with Terraform, stick with the common commands. For the

other commands, please read the help and docs before usage.

 

Common commands:

    apply              Builds or changes infrastructure

    console            Interactive console for Terraform interpolations

    destroy            Destroy Terraform-managed infrastructure

    env                Workspace management

    fmt                Rewrites config files to canonical format

    get                Download and install modules for the configuration

    graph              Create a visual graph of Terraform resources

    import             Import existing infrastructure into Terraform

    init               Initialize a Terraform working directory

    output             Read an output from a state file

    plan               Generate and show an execution plan

    providers          Prints a tree of the providers used in the configuration

    refresh            Update local state file against real resources

    show               Inspect Terraform state or plan

    taint              Manually mark a resource for recreation

    untaint            Manually unmark a resource as tainted

    validate           Validates the Terraform files

    version            Prints the Terraform version

    workspace          Workspace management

 

All other commands:

    0.12upgrade        Rewrites pre-0.12 module source code for v0.12

    debug              Debug output management (experimental)

    force-unlock       Manually unlock the terraform state

    push               Obsolete command for Terraform Enterprise legacy (v1)

    state              Advanced state management

 

HTTP/1.1 413 Request Entity Too Large

Is that the error you see while uploading big packages?

Well the solution is simple - if you have an ingress for the domain name to which you are uploading the package - just configure the following: 

nginx.ingress.kubernetes.io/proxy-body-size: 5000m

note: change the 5000m that has been mentioned as an example as per your requirement... enjoy!

variable substitution in for loop with curly brackets

Requirement - 

I want to calculate the no. of AEM_INSTANCES which I am passing as an argument to a shell script. Suppose the name of the script is script.sh and I want to pass 5 AEM Instances... the usage will be: ./script.sh 5

#!/bin/bash
#set -x TOTAL_AEM_INSTANCE=${1}
echo ${TOTAL_AEM_INSTANCE}
for i in {1..${TOTAL_AEM_INSTANCE}}
do echo $i AEM_TYPE=publish${i}
echo ${AEM_TYPE} done

Unfortunately the variable substitution is not possible like above...

So the Solution is with sequence as shown below - 

#!/bin/bash
#set -x
TOTAL_AEM_INSTANCE=${1}
echo ${TOTAL_AEM_INSTANCE}
for i in `seq ${TOTAL_AEM_INSTANCE}`
do echo $i
AEM_TYPE=publish${i}
echo ${AEM_TYPE} done

Getting jq to work on Macbook

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" < /dev/null 2> /dev/null
brew install jq

Lessons learnt from "mysql golang rest api project"

Recently I was working on a mysql golang rest api project and these are some of the issues i faced....

1. couldn't get to use where clause in mysql statement inside a pod until i used the command this way - 

[dipesh.majumdar@demo ~]$ k -n api-test1-mysql exec mysql-5bbdb4b5d7-wmcxs -- bash -c 'mysql -u root -ppassword -e "select *  from opencommdb.people where uuid = \"a43d29fb-6b5f-11e9-b784-8adc4de39d81\""'
mysql: [Warning] Using a password on the command line interface can be insecure.
uuid    survived        passengerClass  name    sex     age     siblingsOrSpousesAboard parentsOrChildrenAboard fare
a43d29fb-6b5f-11e9-b784-8adc4de39d81    0       3       Mr. Raun Harris Braund  male    23      1       0       7.15
[dipesh.majumdar@demo ~]$

changing sql mode was a challenge inside container... lots of wrong stuff all over the place in internet..

the error that was bugging me was:

Error 1366: Incorrect integer value: '' for column 'survived' at row 1[GIN] 2019/04/30 - 17:00:01 | 200 |    1.068836ms |      10.132.0.6 | POST     /people

finally i could change sql mode like this

[dipesh.majumdar@demo src]$ k -n api-test1-mysql exec -it mysql-5bbdb4b5d7-wmcxs -- sh
# mysql -u root -ppassword -e "SET GLOBAL sql_mode = '';"
mysql: [Warning] Using a password on the command line interface can be insecure.

activeDeadlineSeconds and backoffLimit

Lets create a job - 

apiVersion: batch/v1
kind: Job
metadata:
  creationTimestamp: null
  labels:
    run: shuffle-job
  name: shuffle-job
spec:
  backoffLimit: 40
  activeDeadlineSeconds: 120
  template:
    metadata:
      creationTimestamp: null
      labels:
        run: shuffle-job
    spec:
      containers:
      - args:
        - /bin/sh
        - -c
        - while true; do i=$(shuf -i 1-100 -n 1); if [ $i -gt 90 ];then echo success;
          exit 0; else echo failure; exit 1; fi; done;
        image: busybox
        name: shuffle-job
        resources: {}
      restartPolicy: Never
status: {}

The above job exits with 0 and thus deemed successful only if the random number generated is greater than 90. We allow the job to fail only if no. of attempts exceeds backoffLimit or activeDeadlineSeconds. activeDeadlineSeconds always takes precedence over backoffLimit.

Here is sample output of a describe job - 

Active Deadline Seconds:  120s
Pods Statuses:            0 Running / 0 Succeeded / 5 Failed
Pod Template:
  Labels:  controller-uid=32760c17-6403-11e9-a71d-42010a8400e1
           job-name=shuffle-job
           run=shuffle-job
  Containers:
   shuffle-job:
    Image:      busybox
    Port:       <none>
    Host Port:  <none>
    Args:
      /bin/sh
      -c
      while true; do i=$(shuf -i 1-100 -n 1); if [ $i -gt 90 ];then echo success; exit 0; else echo failure; exit 1; fi; done;
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Events:
  Type     Reason            Age    From            Message
  ----     ------            ----   ----            -------
  Normal   SuccessfulCreate  14m    job-controller  Created pod: shuffle-job-r4mst
  Normal   SuccessfulCreate  14m    job-controller  Created pod: shuffle-job-wp6rz
  Normal   SuccessfulCreate  14m    job-controller  Created pod: shuffle-job-gswlc
  Normal   SuccessfulCreate  13m    job-controller  Created pod: shuffle-job-p8rts
  Normal   SuccessfulCreate  12m    job-controller  Created pod: shuffle-job-x6sxq
  Warning  DeadlineExceeded  7m19s  job-controller  Job was active longer than specified deadline

 

 

 

Difference between service port and container port

service port is marked in yellow highlight below - 

k run  nginx --image nginx:1.7.7 --port 8080

k expose deploy nginx --name nginx-svc --port 6060 --target-port 8080

 

 

To those who seek to be noticed...

Be so good in your craft that people are bound to notice you.  - Dipesh Majumdar

vi editor tips and tricks

  1. go to top of screen or the beginning of the file
  2. go to middle of file
  3. go to end of file (b) Go to the 10th line
  4. start inserting just above the cursor line (better than i)
  5. start inserting below the cursor line... though i rarely need this because i like to insert from above cursor line
  6. go to 9th line of job.yaml
  7. find out all occurences of the string job in job.yaml
  8. traverse word by word - forward and then backword
  9. traverse left right
  10. traverse up and down
  11. delete to end of word. but if word: the : wont be deleted so now delete complete word
  12. delete word up until next space
  13. go to a particular character say - t
  14. go back to a particular character say - T
  15. go to beginning of line and then to end of line
  16. yank till just before a particular character, say d
  17. yank till a particular character including that character, say d
  18. See lines in a particular file and then stop seeing lines
  19. Indent Line 13 to 18 in a file by two spaces
  20. Ok your curser points to a particular word - how will you go to the next occurence of the same word

answers: 

  1. H (or gg)
  2. M
  3. L  (or you can do a capital G - which means the end of file and not screen).  b) 10G
  4. O
  5. o
  6. vi job.yaml +9
  7. grep -in job job.yaml
  8. w for forward and b for backward
  9. h and l
  10. k and j
  11. dw  word and dW for complete word
  12. as you would all know the meaning of dd - which is to delete a line... and x - which is to delete a character... have you ever thought about as to how to delete a group of characters till a particular character. Well the answer is: dt<that_character>.... so it can be like dt<SPACE> if you want to delete till the next space character.
  13. ft
  14. fT
  15. 0 $
  16. ytd
  17. yfd
  18. set numbers && set nonumbers
  19. 13,18s/^/  /g
  20. *

 

links and references:

http://www.lagmonster.org/docs/vi.html

https://www.thomas-krenn.com/en/wiki/Vi_editor_tips_and_tricks

https://www.tecmint.com/how-to-use-vi-and-vim-editor-in-linux/

https://openvim.com/

skeleton ingress to feed nginx ingress controller

The way it works is: 

nginx ingress controller has to be deployed in your nginx namespace which is exclusively reserved for nginx-ingress related stuff.

in that nginx-ingress-controller will have a pod (or more replicas) and a matching service. 

Now individual ingresses for your workloads need to be created so these are then consumed by nginx-ingress-controller pod

To create individual ingress resources one must know to create a skeleton ingress (bare minimum stuff that must go into an ingress) first. So here is syntax of skeleton-ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
 name: dipz22-ingress
 annotations:
  nginx.ingress.kubernetes.io/rewrite-target: /
  nginx.ingress.kubernetes.io/ssl-redirect: "false"
 labels: 
   app: ingress
spec:
 rules: 
  - http:
      paths: 
       - path: /
           backend: 
           servicePort: 80
           serviceName: dipz22-service

Enabling K8s network policy during Cluster creation

While creating k8s cluster you need to enable this feature. 

Common Mistakes in declarative yaml manifests and imperative commands

Its a common mistake - you need to know yaml properly. so here i first did this

    readinessProbe:
     httpGet:
     - port: 80
       path: /

and got this error - 

 got "array", expected "map";

The correct way is this

    readinessProbe:
     httpGet:
       port: 80
       path: /

******************************************************

got map expected array - 

    volumeMounts:
      name: xyz
      mountPath: /etc/foo

should be - 

    volumeMounts:
      - name: xyz
        mountPath: /etc/foo

**********************************

 unknown field "name"

spec:
  volumes:
     name: xyz
     secret:
       name: mysecret3

should be - 

spec:
  volumes:
   - name: xyz
     secret:
       secretName: mysecret3

 

*************************************

Attaching service account with pod. one tends to do it this way - 

  serviceAccount:
    name: myuser

Correct way is this:

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  serviceAccountName: myuser
  containers:
  - image: nginx
    name: nginx
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

***************************************

While creating secret from a file create the password or username as the file and then value of password or username should be entered in the file. only one value - remember... otherwise its better to create secret from literal

[dipesh.majumdar@demo ~]$ echo pass >password
[dipesh.majumdar@demo ~]$ k create secret generic mysecret2 --from-file=password

***********************************************************************

This is not the way to create configmap

[dipesh.majumdar@demo ~]$ k -n temp create cm cm1 --from-literal=db=mysqldipz22,user=dipz22,password=password123
configmap/cm1 created
[dipesh.majumdar@demo ~]$ k -n temp get cm cm1 -o yaml
apiVersion: v1
data:
  db: mysqldipz22,user=dipz22,password=password123
kind: ConfigMap
metadata:
  creationTimestamp: "2019-04-15T18:54:54Z"
  name: cm1
  namespace: temp
  resourceVersion: "1948893"
  selfLink: /api/v1/namespaces/temp/configmaps/cm1
  uid: f4d36380-5faf-11e9-9e93-42010a8400d9
[dipesh.majumdar@demo ~]$ k -n temp

THIS IS THE CORRECT WAY:

[dipesh.majumdar@demo ~]$ k -n temp create cm cm1 --from-literal=db=mysqldipz22 --from-literal=user=dipz22 --from-literal=password=password123
configmap/cm1 created

 

Sitting on a Jumpbox and testing with network policy

You want to sit on a jumpbox so you can wget to some clusterip:port for testing only....

well if you create a busybox pod - its going to get completed as the entrypoint of the image of its container is only 'sh'

so you have to make it with say 'sleep 3600' 

but you dont what to do that so you can just do this - 

[dipesh.majumdar@demo ~]$ k run busybox --restart=Never --image=busybox -it
If you don't see a command prompt, try pressing enter.
/ # pwd
/
/ #

/ # wget -O- 10.112.11.76:7777
Connecting to 10.112.11.76:7777 (10.112.11.76:7777)
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
-                    100% |*********************************************************************************************************************************************|   612  0:00:00 ETA
/ #

open another terminal and check the busybox pod and see how it's running and not completed.

[dipesh.majumdar@demo ~]$ k get po
NAME                    READY   STATUS    RESTARTS   AGE
busybox                 1/1     Running   0          30s
nginx-966857787-2h9fb   1/1     Running   0          5m
nginx-966857787-vwtwt   1/1     Running   0          5m
[dipesh.majumdar@demo ~]$

the moment you come out of the busybox the pod status will be completed

NAME                    READY   STATUS      RESTARTS   AGE
busybox                 0/1     Completed   0          8m
nginx-966857787-2h9fb   1/1     Running     0          13m
nginx-966857787-vwtwt   1/1     Running     0          13m

But what if you want to delete it the moment you come out of the busybox pod - this can be done in 2 ways (either way is good):

[dipesh.majumdar@demo ~]$ k run busybox --restart=Never --image=busybox -it --rm
If you don't see a command prompt, try pressing enter.
/ # exit
pod "busybox" deleted
[dipesh.majumdar@demo ~]$ k run busybox --restart=Never --image=busybox -it --rm -- sh
If you don't see a command prompt, try pressing enter.
/ #

Use cases of jump box shown below - 

Create a dummy api server

kubectl run apiserver --restart=Never --image=nginx --labels app=web,role=api --expose --port 80

Create a network policy to bind the api server so access is only possible from pods with label: devops-team

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: devops-np
spec:
  podSelector:
    matchLabels:
      app: web
  ingress:
    - from:
        - podSelector:           # chooses pods with tag devops-team
           matchLabels:
             tag: devops-team

 

k run busybox$RANDOM --restart=Never --image=busybox -it --rm -- sh

[dipesh.majumdar@demo ~]$ k run busybox$RANDOM --restart=Never --image=busybox -it --rm -- sh
If you don't see a command prompt, try pressing enter.
/ # wget -qO- --timeout=2 http://apiserver
wget: download timed out

[dipesh.majumdar@demo ~]$ k run busybox$RANDOM --restart=Never --image=busybox --labels 'tag=devops-team' -it --rm -- sh
If you don't see a command prompt, try pressing enter.
/ # wget -qO- --timeout=2 http://apiserver
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
/ #

One more time this time very fast - 

quick creation of dummy apiserver was seen (this created pod and service)

lets do a dummy web... this time this creates pod, matching deployment and matching service. 

k -n temp run web --image nginx:1.7.9 --expose --labels 'app=web' --port 80

now a simple jumpbox...

k -n temp run jumpbox --restart Never --image busybox --labels 'app=jumpbox' --rm -it

###this is same as 

k -n temp run jumpbox --restart Never --image busybox --labels 'app=jumpbox' --rm -it sh

/ # wget --timeout 2 -O- web:80
Connecting to web:80 (10.112.12.179:80)
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>

want to run some command without getting into the pod.. instead of sh run the command in this case - whoami

[dipesh.majumdar@demo ~]$ k -n temp run jumpbox --restart Never --image busybox --labels 'app=jumpbox' --rm -it whoami
root
pod "jumpbox" deleted

 

 

 

Little more magic...

ready for it?

do this...

[dipesh.majumdar@demo ~]$ k -n temp run jumpbox --image alpine --restart Never --labels 'app=jumpbox' --rm -it
If you don't see a command prompt, try pressing enter.
/ # apk add --no-cache curl openssl
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
(1/6) Installing ca-certificates (20190108-r0)
(2/6) Installing nghttp2-libs (1.35.1-r0)
(3/6) Installing libssh2 (1.8.2-r0)
(4/6) Installing libcurl (7.64.0-r1)
(5/6) Installing curl (7.64.0-r1)
(6/6) Installing openssl (1.1.1b-r1)
Executing busybox-1.29.3-r10.trigger
Executing ca-certificates-20190108-r0.trigger
OK: 8 MiB in 20 packages
/ # curl -v -I -o /dev/null http://web:80
* Expire in 0 ms for 6 (transfer 0x560fa43a77a0)
* Expire in 1 ms for 1 (transfer 0x560fa43a77a0)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Expire in 0 ms for 1 (transfer 0x560fa43a77a0)
* Expire in 2 ms for 1 (transfer 0x560fa43a77a0)
* Expire in 0 ms for 1 (transfer 0x560fa43a77a0)
* Expire in 0 ms for 1 (transfer 0x560fa43a77a0)
* Expire in 2 ms for 1 (transfer 0x560fa43a77a0)
* Expire in 0 ms for 1 (transfer 0x560fa43a77a0)
* Expire in 0 ms for 1 (transfer 0x560fa43a77a0)
* Expire in 2 ms for 1 (transfer 0x560fa43a77a0)
* Expire in 1 ms for 1 (transfer 0x560fa43a77a0)
* Expire in 1 ms for 1 (transfer 0x560fa43a77a0)
* Expire in 2 ms for 1 (transfer 0x560fa43a77a0)
* Expire in 1 ms for 1 (transfer 0x560fa43a77a0)
* Expire in 1 ms for 1 (transfer 0x560fa43a77a0)
* Expire in 1 ms for 1 (transfer 0x560fa43a77a0)
*   Trying 10.112.14.49...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x560fa43a77a0)
* Connected to web (10.112.14.49) port 80 (#0)
> HEAD / HTTP/1.1
> Host: web
> User-Agent: curl/7.64.0
> Accept: */*

The docker equivalent of  k8s alpine jumpbox  is - 

[dipesh.majumdar@demo ~]$ docker run --rm -it alpine
/ # apk add --no-cahe curl openssl
apk: unrecognized option: no-cahe
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
(1/6) Installing ca-certificates (20190108-r0)
(2/6) Installing nghttp2-libs (1.35.1-r0)
(3/6) Installing libssh2 (1.8.2-r0)
(4/6) Installing libcurl (7.64.0-r1)
(5/6) Installing curl (7.64.0-r1)
(6/6) Installing openssl (1.1.1b-r1)
Executing busybox-1.29.3-r10.trigger
Executing ca-certificates-20190108-r0.trigger
OK: 8 MiB in 20 packages
/ # curl -v -I -o /dev/null http://web:80
* Expire in 0 ms for 6 (transfer 0x55e43644d7a0)
* Expire in 1 ms for 1 (transfer 0x55e43644d7a0)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Expire in 0 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 1 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 0 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 0 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 1 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 0 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 0 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 1 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 0 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 0 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 1 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 2 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 2 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 2 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 2 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 2 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 2 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 3 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 3 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 4 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 3 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 3 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 4 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 4 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 4 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 8 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 6 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 6 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 8 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 8 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 8 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 16 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 11 ms for 1 (transfer 0x55e43644d7a0)
* Expire in 11 ms for 1 (transfer 0x55e43644d7a0)
* Could not resolve host: web
* Expire in 14 ms for 1 (transfer 0x55e43644d7a0)
* Closing connection 0
curl: (6) Could not resolve host: web   

#####OBVIOUSLY IT CAN'T RESOLVE HOST BCOZ THIS IS DOCKER AND THE HOST IS IN K8S CLUSTER AS SERVICE

THE ALPINE CONTAINER DISAPPEARS AS SOON AS WE COME OUT OF THE EXECUTABLE SHELL !!!!

login as: dipesh.majumdar
Authenticating with public key "imported-openssh-key"
Passphrase for key "imported-openssh-key":
Last login: Sun Apr 14 11:36:43 2019 from dhcp-077-249-177-085.chello.nl
[dipesh.majumdar@demo ~]$ docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                  PORTS               NAMES
de27d1c58963        alpine              "/bin/sh"                21 seconds ago      Up 20 seconds                               nostalgic_lamport
8d7696365904        mysql               "docker-entrypoint..."   7 days ago          Exited (1) 7 days ago                       distracted_shockley
c667d456911e        busybox             "sh"                     7 days ago          Exited (0) 7 days ago                       nostalgic_shaw
[dipesh.majumdar@demo ~]$ docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                  PORTS               NAMES
8d7696365904        mysql               "docker-entrypoint..."   7 days ago          Exited (1) 7 days ago                       distracted_shockley
c667d456911e        busybox             "sh"                     7 days ago          Exited (0) 7 days ago                       nostalgic_shaw
[dipesh.majumdar@demo ~]$

However if you want that the docker container should not disappear - run it like this

[dipesh.majumdar@demo ~]$ docker run --rm -dit alpine
6a3c0f71d34fea50f2d6a66b7370986625d6983b6c991f7db7a25f87632c4223

and then enter the shell like this... 

[dipesh.majumdar@demo ~]$ docker exec -it 6a3c0f71d34f sh
/ # whoami
root
/ # exit

[dipesh.majumdar@demo ~]$

even after you exit the container will be up and running

want to execute a command without entering the docker - do it this way

[dipesh.majumdar@demo ~]$ docker exec -it 6a3c0f71d34f whoami
root
[dipesh.majumdar@demo ~]$ docker exec -it 6a3c0f71d34f pwd
/

 

 

 

Creating a pod and a matching service imperatively

[dipesh.majumdar@demo ~]$ k run nginx --restart=Never --image=nginx --dry-run --port=80  --expose -o yaml
apiVersion: v1
kind: Service
metadata:
  creationTimestamp: null
  name: nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
status:
  loadBalancer: {}
---
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  containers:
  - image: nginx
    name: nginx
    ports:
    - containerPort: 80
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

take it in ps.yaml with > ps.yaml and then....

[dipesh.majumdar@demo ~]$ vi ps.yaml
[dipesh.majumdar@demo ~]$ k create -f ps.yaml
service/nginx created
pod/nginx created
[dipesh.majumdar@demo ~]$

so service is created is this 

[dipesh.majumdar@demo ~]$ k get svc nginx
NAME    TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
nginx   ClusterIP   10.112.2.47   <none>        80/TCP    12m

 

this cluster ip seems useless to me as i can't connect from my local.

Hey Wait!

But the cluster ip can reached at from another pod

so create a busybox pod and run below from inside the pod....

wget -O- 10.112.2.47:80
Connecting to 10.112.2.47:80 (10.112.2.47:80)
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }

 

Now it's time to expose deployment. first create the deployment - 

[dipesh.majumdar@demo ~]$ cat d.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: simpleapp
  name: simpleapp
spec:
  replicas: 3
  selector:
    matchLabels:
      app: foo
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: foo
    spec:
      containers:
      - image: dgkanatsios/simpleapp
        name: simpleapp
        ports:
          - containerPort: 8080
        resources: {}
status: {}

 

k expose deploy simpleapp --port 6666 --target-port 8080

So the svc simpleapp can connect to 3 pod ips:ports (these are the endpoints) see them with a k describe svc simple command. 

[dipesh.majumdar@demo ~]$ k get svc simpleapp
NAME        TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
simpleapp   ClusterIP   10.112.14.183   <none>        6666/TCP   5m

[dipesh.majumdar@demo ~]$ k exec -it busybox -- wget -O- 10.112.14.183:6666
Connecting to 10.112.14.183:6666 (10.112.14.183:6666)
Hello world from simpleapp-5b6f9c5676-j8szn and version 2.0
-                    100% |*********************************************************************************************************************************************|    60  0:00:00 ETA

 

you can definitely connect to pod-ip from jumbox server busybox as shown below - 

 

[dipesh.majumdar@demo ~]$ k get po -o wide
NAME                         READY   STATUS    RESTARTS   AGE   IP           NODE                                                NOMINATED NODE
busybox                      1/1     Running   0          12m   10.48.0.32   gke-standard-cluster-1-default-pool-c7a16408-wk8x   <none>
simpleapp-5b6f9c5676-bxc5w   1/1     Running   0          20m   10.48.0.30   gke-standard-cluster-1-default-pool-c7a16408-wk8x   <none>
simpleapp-5b6f9c5676-j8szn   1/1     Running   0          20m   10.48.0.31   gke-standard-cluster-1-default-pool-c7a16408-wk8x   <none>
simpleapp-5b6f9c5676-tg9kv   1/1     Running   0          20m   10.48.1.63   gke-standard-cluster-1-default-pool-c7a16408-1bnp   <none>
[dipesh.majumdar@demo ~]$ k exec -it busybox -- wget -O- 10.48.0.30:8080
Connecting to 10.48.0.30:8080 (10.48.0.30:8080)
Hello world from simpleapp-5b6f9c5676-bxc5w and version 2.0
-                    100% |********************************|    60  0:00:00 ETA
[dipesh.majumdar@demo ~]$

 

Volumes, Storageclasses, pvc

First of all, if you are creating a hostpath volume within a pod of type Dir and it looks like this...

volumes:
  - name: test-volume
    hostPath:
      path: /etc/some/path  #MIGHT NOT WORK
      type: Directory

it might not work because that path /etc/some/path might not be present in your node file system. 

volumes:
  - name: test-volume
    hostPath:
      path: /sys  #WORKED FOR ME
      type: Directory

so i created one like this /sys which might be present in the node file sytem and then only it started to work.  Whether this is something you should do is another debate. This blog is not responsible for anything that breaks in production systems. So adopt caution!!!

Now coming to the second point ->

After a lot of hard work I managed to write a volume from yaml file. I was a bit surprised to see that there is no way you create a volume imperatively - k create pv ...blah blah... so here is the yaml manifest of the pv

[dipesh.majumdar@demo ~]$ cat pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: myvolume
spec:
 storageClassName: normal
 hostPath:
  path: /etc/foo
 accessModes:
  - ReadWriteOnce
  - ReadWriteMany
 capacity:
  storage: "10Gi"

But then i got intriged by the storageClassName, becausae i knew that I had my storageClass somewhere in the cluster but then i soon discovered - i do have and it is standard and it DOES NOT HAVE A NAMESPACE

HOLY COW

STORAGECLASS IS NAMESPACE FREE - Imprint that in memory my dear readers!!!

anyway we will visit that later....

so i create my pv.yaml

[dipesh.majumdar@demo ~]$ k create -f pv.yaml
persistentvolume/myvolume created

[dipesh.majumdar@demo ~]$ k get pv
NAME       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE
myvolume   10Gi       RWO,RWX        Retain           Available           normal                  1m

Looks good so far...

I dig deeper... and I find that this storageclassName is not the storageClass i was talking about in the beginning but it specifies a class which can be used for association with pvc for example... 

it is like a label

[dipesh.majumdar@demo ~]$ k get sc normal
Error from server (NotFound): storageclasses.storage.k8s.io "normal" not found
[dipesh.majumdar@demo ~]$ k get sc
NAME                 PROVISIONER            AGE
standard (default)   kubernetes.io/gce-pd   3d

Only PVs of the requested class, ones with the same storageClassName as the PVC, can be bound to the PVC.

[dipesh.majumdar@demo ~]$ cat pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
 name: mypvc
spec:
 storageClassName: normal
 accessModes:
  - ReadWriteOnce
 resources:
  requests:
    storage: 4Gi

 

[dipesh.majumdar@demo ~]$ k get pvc
NAME    STATUS   VOLUME     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
mypvc   Bound    myvolume   10Gi       RWO,RWX        normal         1m
[dipesh.majumdar@demo ~]$

ok so now its time to create a pod and associate the volume just created with it and make the volume belong to pvc and not configMap or secret which is also possible

$ k run busybox --restart=Never --image=busybox --dry-run -o yaml --  sleep 3600 >p.yaml

Edit the p.yaml with the yellow highlights ->

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: busybox
  name: busybox
spec:
  volumes:
   - name: myvolume #this volume can be named xyz and is not the pv we created earlier
     persistentVolumeClaim:
       claimName: mypvc
  containers:
  - args:
    - sleep
    - "3600"
    image: busybox
    name: busybox
    volumeMounts:
     - mountPath: /etc/foo #this path need not match with pv hostpath we created earlier
       name: myvolume
#this volume can be named xyz and is not the pv we created earlier
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

 

Makes a lot of sense now. 

Let me enter something into this directory now by getting inside the pod - >

[dipesh.majumdar@demo ~]$ k exec -it busybox -- sh
/ # pwd
/
/ # cd /etc/foo
/etc/foo # ls
/etc/foo # date > test.txt
/etc/foo # cat test.txt
Sun Apr  7 09:55:18 UTC 2019
/etc/foo # pwd
/etc/foo

 

I will copy the /etc/foo/test.txt to my local

[dipesh.majumdar@demo ~]$ k cp mns/busybox:etc/foo/test.txt /home/dipesh.majumdar
[dipesh.majumdar@demo ~]$ ls test.txt
test.txt
[dipesh.majumdar@demo ~]$ cat test.txt
Sun Apr  7 09:55:18 UTC 2019

Copying from local to pod

[dipesh.majumdar@demo ~]$ date >test2.txt
[dipesh.majumdar@demo ~]$ k cp test2.txt mns/busybox:etc/foo/
[dipesh.majumdar@demo ~]$ k exec -it busybox -- ls -ltra /etc/foo/
total 12
drwxr-xr-x    1 root     root          4096 Apr  7 09:53 ..
-rw-r--r--    1 root     root            29 Apr  7 09:55 test.txt
-rw-rw-r--    1 1000     1001            29 Apr  7 10:05 test2.txt
drwxr-xr-x    2 root     root            80 Apr  7 10:05 .
[dipesh.majumdar@demo ~]$

 

fire you memory - 

spec of pv has 4 items - storageClassName, accessModes, capacity.storage, hostPath.path

spec of pvc has 3 items -  storageClassName, accessModes, resources.requests.storage

notes - 

when the pv is just created status of pv is available. when it is attached to pvc it becomes bound. Deleting pvc makes status of pv as released

[dipesh.majumdar@demo ~]$ k delete pvc mypvc
persistentvolumeclaim "mypvc" deleted
[dipesh.majumdar@demo ~]$ k get pv
NAME       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS     CLAIM       STORAGECLASS   REASON   AGE
myvolume   10Gi       RWO,RWX        Retain           Released   mns/mypvc   normal                  1h
[dipesh.majumdar@demo ~]$

the volume name inside pod is not the persistent volume name but more pod specific.... very important point

persistentVolumeReclaimPolicy: Retain #means pv will be retained even after pvc attached to it is deleted until pv is manually deleted

                                                                         Delete #deleted automatically freeing up storage

                                                                         Recycle #data in the volume will be deleted before it is made available to other pvcs

Finding out Entrypoint, CMD, env of docker from the image

Million Dollar Question: You have an image - but you wan to find out the entry point from the image. How will you do that? Well you need the Dockerfile to know entrypoint right? But if you dont have it right now how will you find out these informations - env, entrypoint, cmd?

[dipesh.majumdar@demo ~]$ docker inspect busybox |grep -A100 -i \"Config\"
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "sh"
            ],
            "ArgsEscaped": true,
            "Image": "sha256:90b7037cc5e65fa9e3f33e8096febd6fad8af0ff94876d73dabe048d65bec645",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 1199417,
        "VirtualSize": 1199417,
        "GraphDriver": {
            "Name": "overlay2",
            "Data": {
                "MergedDir": "/var/lib/docker/overlay2/c6026745fff5b1608f900c1415df5ec03cb3f97cdb54e831e636b35fa7f18548/merged",
                "UpperDir": "/var/lib/docker/overlay2/c6026745fff5b1608f900c1415df5ec03cb3f97cdb54e831e636b35fa7f18548/diff",
                "WorkDir": "/var/lib/docker/overlay2/c6026745fff5b1608f900c1415df5ec03cb3f97cdb54e831e636b35fa7f18548/work"
            }
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:0b97b1c81a3200e9eeb87f17a5d25a50791a16fa08fc41eb94ad15f26516ccea"
            ]
        }
    }
]

 

Monitoring K8s

- metrics server (heapster has been deprecated) - metrics server receives metrics from nodes and stores them in memory and doesn't store data in disk and historical data is not available. each cluster has one metrics server.  how is metrics collected in node? well the kubelet which is the agent on each node and takes instructions from kubernetes api master server and also runs pods in nodes has a sub-component within it which is called cAdvisor or container advisor. it receives metrics from the pods in nodes and makes them available to metrics server.

- prometheus - this can be installed using helm. it gives a GUI which makes it easy to monitor the kubernetes cluster.

- datadog

- dynatrace

View older posts »