15Stepで習得 Dockerから入るKubernetes その1

Kubernetes入門

Kubernetesを始めるにあたり、書籍を購入。
docker desktopのKubernetesでEnable Kubernetesにチェックを入れると使えるようになると記載されているものが多いがやってみたところうまくいかない
こちらではマネージドサービスの利用を強く推奨している

試したこと

  • Enable Kubernetesにチェックを入れる
  • Show system containers (advanced) にチェックを入れる
  • Applyする
1
2
3
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.5", GitCommit:"20c265fef0741dd71a66480e35bd69f18351daea", GitTreeState:"clean", BuildDate:"2019-10-15T19:16:51Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"darwin/amd64"}
Unable to connect to the server: Service Unavailable

Serverが反応しない

Show system containers (advanced) にチェックを入れると、いくつかコンテナが起動しているようにみえる

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0c0adb0a59ec eb516548c180 "/coredns -conf /etc…" 8 seconds ago Up 8 seconds k8s_coredns_coredns-5c98db65d4-ccpgw_kube-system_bca3360b-c620-4dc1-adbb-4f4f72e256f7_0
09027b92a495 eb516548c180 "/coredns -conf /etc…" 8 seconds ago Up 8 seconds k8s_coredns_coredns-5c98db65d4-wnz9s_kube-system_012b9fa8-c63d-49cb-9a92-4c0116283030_0
af28dfeeeab0 cbd7f21fec99 "/usr/local/bin/kube…" 9 seconds ago Up 8 seconds k8s_kube-proxy_kube-proxy-js5n2_kube-system_0f046985-bdd9-4c87-bf04-b8bc86520fb6_0
6f021151faee k8s.gcr.io/pause:3.1 "/pause" 9 seconds ago Up 8 seconds k8s_POD_coredns-5c98db65d4-wnz9s_kube-system_012b9fa8-c63d-49cb-9a92-4c0116283030_0
4be2180c3476 k8s.gcr.io/pause:3.1 "/pause" 9 seconds ago Up 8 seconds k8s_POD_coredns-5c98db65d4-ccpgw_kube-system_bca3360b-c620-4dc1-adbb-4f4f72e256f7_0
d37faa0633fe k8s.gcr.io/pause:3.1 "/pause" 9 seconds ago Up 9 seconds k8s_POD_kube-proxy-js5n2_kube-system_0f046985-bdd9-4c87-bf04-b8bc86520fb6_0
9faf284de0a2 fab2dded59dd "kube-scheduler --bi…" 23 seconds ago Up 23 seconds k8s_kube-scheduler_kube-scheduler-docker-desktop_kube-system_131c3f63daec7c0750818f64a2f75d20_0
adbe6374475a e534b1952a0d "kube-apiserver --ad…" 23 seconds ago Up 23 seconds k8s_kube-apiserver_kube-apiserver-docker-desktop_kube-system_556b996466155d7ad37d896897208f67_0
d063399d4feb 1399a72fa1a9 "kube-controller-man…" 23 seconds ago Up 23 seconds k8s_kube-controller-manager_kube-controller-manager-docker-desktop_kube-system_b2850863d2a1ddaac8ed962e6881584c_0
a6501f7001f5 2c4adeb21b4f "etcd --advertise-cl…" 23 seconds ago Up 23 seconds k8s_etcd_etcd-docker-desktop_kube-system_f132e1c85abd4953a304d1c65d9d74f9_0
61b6078a415c k8s.gcr.io/pause:3.1 "/pause" 23 seconds ago Up 23 seconds k8s_POD_kube-scheduler-docker-desktop_kube-system_131c3f63daec7c0750818f64a2f75d20_0
036ba80a78f6 k8s.gcr.io/pause:3.1 "/pause" 23 seconds ago Up 23 seconds k8s_POD_kube-controller-manager-docker-desktop_kube-system_b2850863d2a1ddaac8ed962e6881584c_0
c1f2ca81a768 k8s.gcr.io/pause:3.1 "/pause" 23 seconds ago Up 23 seconds k8s_POD_kube-apiserver-docker-desktop_kube-system_556b996466155d7ad37d896897208f67_0
a832aab318ac k8s.gcr.io/pause:3.1 "/pause" 23 seconds ago Up 23 seconds k8s_POD_etcd-docker-desktop_kube-system_f132e1c85abd4953a304d1c65d9d74f9_0

が、結果は変わらなかった

minikubeのインストール

Docker DesktopのKubernetesが動作しないのであればminikubeをインストールするしかなさそう

https://kubernetes.io/ja/docs/tasks/tools/install-minikube/
こちらを参考にインストール
kubectlも最新のものを利用するようにbrewでインストール

1
2
3
4
5
6
7
$ brew install virtualbox
$ brew install vagrant
# docker desktopでインストールされていたkubectlのシンボリックリンクをrename
# brew install kbuectlでリンクがはれずにエラーになるため
$ mv /usr/local/bin/kubectl /usr/local/bin/kubectl.docker
$ brew install kubectl
$ brew install minikube

Proxyを利用する環境での設定

無事インストールが終わる

1
2
3
$ minikube version
minikube version: v1.7.3
commit: 436667c819c324e35d7e839f8116b968a2d0a3ff

minikube startを実行してみる

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
$ minikube start
😄 minikube v1.7.3 on Darwin 10.13.6
✨ Automatically selected the hyperkit driver. Other choices: virtualbox, docker (experimental)
💾 Downloading driver docker-machine-driver-hyperkit:
> docker-machine-driver-hyperkit.sha256: 65 B / 65 B [---] 100.00% ? p/s 0s
> docker-machine-driver-hyperkit: 10.88 MiB / 10.88 MiB 100.00% 5.52 MiB p
🔑 The 'hyperkit' driver requires elevated permissions. The following commands will be executed:

$ sudo chown root:wheel /Users/username/.minikube/bin/docker-machine-driver-hyperkit
$ sudo chmod u+s /Users/username/.minikube/bin/docker-machine-driver-hyperkit


Password:
💿 Downloading VM boot image ...
> minikube-v1.7.3.iso.sha256: 65 B / 65 B [--------------] 100.00% ? p/s 0s
> minikube-v1.7.3.iso: 167.39 MiB / 167.39 MiB [] 100.00% 14.63 MiB p/s 12s
🔥 Creating hyperkit VM (CPUs=2, Memory=2000MB, Disk=20000MB) ...
🌐 Found network options:
▪ HTTP_PROXY=http://username:password@proxy:8080
⚠️ You appear to be using a proxy, but your NO_PROXY environment does not include the minikube IP (192.168.64.2). Please see https://minikube.sigs.k8s.io/docs/reference/networking/proxy/ for more details
▪ HTTPS_PROXY=http://username:password@proxy:8080
▪ http_proxy=http://username:password@proxy:8080
▪ https_proxy=http://username:password@proxy:8080
⚠️ VM is unable to access k8s.gcr.io, you may need to configure a proxy or set --image-repository
🐳 Preparing Kubernetes v1.17.3 on Docker 19.03.6 ...
▪ env HTTP_PROXY=http://username:password@proxy:8080
▪ env HTTPS_PROXY=http://username:password@proxy:8080
▪ env HTTP_PROXY=http://username:password@proxy:8080
▪ env HTTPS_PROXY=http://username:password@proxy:8080
💾 Downloading kubeadm v1.17.3
💾 Downloading kubelet v1.17.3
💾 Downloading kubectl v1.17.3
🚀 Launching Kubernetes ...
🌟 Enabling addons: default-storageclass, storage-provisioner
⌛ Waiting for cluster to come online ...
🏄 Done! kubectl is now configured to use "minikube"

Proxyの設定を環境変数から自動的に認識してくれていて、必要な設定NO_PROXYが設定されてないとも教えてくれている
NO_PROXYが設定されてないからか、Server側の返答が来ない

1
2
3
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.3", GitCommit:"06ad960bfd03b39c8310aaf92d1e7c12ce618213", GitTreeState:"clean", BuildDate:"2020-02-13T18:07:54Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"darwin/amd64"}
Unable to connect to the server: context deadline exceeded (Client.Timeout exceeded while awaiting headers)

NO_PROXYを設定する

1
$ export NO_PROXY=192.168.64.2

NO_PROXYの指定はsubnet形式で行う方が良い(IP変わるので)

1
$ export NO_PROXY=192.168.64.0/24

念のためdelete

1
2
3
$ minikube delete
🔥 Deleting "minikube" in hyperkit ...
💀 Removed all traces of the "minikube" cluster.

起動

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
$ minikube start
😄 minikube v1.7.3 on Darwin 10.13.6
✨ Automatically selected the hyperkit driver. Other choices: virtualbox, docker (experimental)
🔥 Creating hyperkit VM (CPUs=2, Memory=2000MB, Disk=20000MB) ...
🌐 Found network options:
▪ HTTP_PROXY=http://username:password@proxy:8080
▪ HTTPS_PROXY=http://username:password@proxy:8080
▪ NO_PROXY=192.168.64.2,192.168.64.3,192.168.64.4
▪ http_proxy=http://username:password@proxy:8080
▪ https_proxy=http://username:password@proxy:8080
⚠️ VM is unable to access k8s.gcr.io, you may need to configure a proxy or set --image-repository
🐳 Preparing Kubernetes v1.17.3 on Docker 19.03.6 ...
▪ env HTTP_PROXY=http://username:password@proxy:8080
▪ env HTTPS_PROXY=http://username:password@proxy:8080
▪ env NO_PROXY=192.168.64.2,192.168.64.3,192.168.64.4
▪ env HTTP_PROXY=http://username:password@proxy:8080
▪ env HTTPS_PROXY=http://username:password@proxy:8080
🚀 Launching Kubernetes ...
🌟 Enabling addons: default-storageclass, storage-provisioner
⌛ Waiting for cluster to come online ...
🏄 Done! kubectl is now configured to use "minikube"

今度は大丈夫そう

1
2
3
4
5
6
7
8
9
10
11
12
13
$ minikube status
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured
$ kubectl cluster-info
Kubernetes master is running at https://192.168.64.4:8443
KubeDNS is running at https://192.168.64.4:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.3", GitCommit:"06ad960bfd03b39c8310aaf92d1e7c12ce618213", GitTreeState:"clean", BuildDate:"2020-02-13T18:07:54Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.3", GitCommit:"06ad960bfd03b39c8310aaf92d1e7c12ce618213", GitTreeState:"clean", BuildDate:"2020-02-11T18:07:13Z", GoVersion:"go1.13.6", Compiler:"gc", Platform:"linux/amd64"}

やっとServerが応答してくれたが、大丈夫じゃなかった
minikube start の時のメッセージ

1
⚠️  VM is unable to access k8s.gcr.io, you may need to configure a proxy or set --image-repository

proxyの設定は環境変数から自動的に取得してくれているのに、なぜかimageの取得が行えない
deploymentを作成してpodのステータスをみるとErrImagePullとなってしまう

1
2
3
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
hello-minikube-797f975945-kl26w 0/1 ErrImagePull 0 54s

proxyの設定を見直してみると、proxyのホストをproxyとしてしまっていて、その名前が引けない
なのでIP指定にする
ローカルのPCにて設定

1
2
3
4
5
6
7
$ export HTTP_PROXY_USER=username
$ export HTTP_PROXY_PASSWORD=password
$ export HTTPS_PROXY=http://${HTTP_PROXY_USER}:${HTTP_PROXY_PASSWORD}@proxy_ip:8080
$ export HTTP_PROXY=http://${HTTP_PROXY_USER}:${HTTP_PROXY_PASSWORD}@proxy_ip:8080
$ export NO_PROXY=192.168.64.0/24
$ export http_proxy=$HTTP_PROXY
$ export https_proxy=$HTTPS_PROXY

再度minikube start

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
$ minikube start
😄 minikube v1.7.3 on Darwin 10.13.6
✨ Automatically selected the hyperkit driver. Other choices: virtualbox, docker (experimental)
🔥 Creating hyperkit VM (CPUs=2, Memory=2000MB, Disk=20000MB) ...
🌐 Found network options:
▪ HTTP_PROXY=http://username:password@proxy_ip:8080
▪ HTTPS_PROXY=http://username:password@proxy_ip:8080
▪ NO_PROXY=192.168.64.0/24
▪ http_proxy=http://username:password@proxy_ip:8080
▪ https_proxy=http://username:password@proxy_ip:8080
🐳 Preparing Kubernetes v1.17.3 on Docker 19.03.6 ...
▪ env HTTP_PROXY=http://username:password@proxy_ip:8080
▪ env HTTPS_PROXY=http://username:password@proxy_ip:8080
▪ env NO_PROXY=192.168.64.0/24
▪ env HTTP_PROXY=http://username:password@proxy_ip:8080
▪ env HTTPS_PROXY=http://username:password@proxy_ip:8080
🚀 Launching Kubernetes ...
🌟 Enabling addons: default-storageclass, storage-provisioner
⌛ Waiting for cluster to come online ...
🏄 Done! kubectl is now configured to use "minikube"

大丈夫そう!

1
2
3
4
5
$ kubectl create deployment hello-minikube --image=k8s.gcr.io/echoserver:1.10
deployment.apps/hello-minikube created
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
hello-minikube-797f975945-jnfkt 1/1 Running 0 24s

statusがRunningになっている
これでやっと環境が整ったので、アプリケーションの設定を行うことができるようになった

おまけ

minikube stopしてからのminikube startが動かない

1
2
3
4
5
6
7
8
$ minikube stop
✋ Stopping "minikube" in hyperkit ...
🛑 "minikube" stopped.
$ minikube start
😄 minikube v1.7.3 on Darwin 10.13.6
✨ Using the hyperkit driver based on existing profile
⌛ Reconfiguring existing host ...
🔄 Starting existing hyperkit VM for "minikube" ...

ここから動かないので一旦ctrl-Cで止める
もう一度やってみる

1
$ minikube start

全く反応しなくなる…
ステータスを確認

1
2
3
4
5
$ minikube status
host: Stopped
kubelet: Stopped
apiserver: Stopped
kubeconfig: Stopped

もう一回だけやってみるとさっきのところまでは進んだ

1
2
3
4
5
$ minikube start
😄 minikube v1.7.3 on Darwin 10.13.6
✨ Using the hyperkit driver based on existing profile
⌛ Reconfiguring existing host ...
🔄 Starting existing hyperkit VM for "minikube" ...

が、結局ここから動かないので一旦削除する

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
$ minikube delete
🔥 Deleting "minikube" in hyperkit ...
💀 Removed all traces of the "minikube" cluster.
$ export NO_PROXY=192.168.64.2,192.168.64.3,192.168.64.4,192.168.64.5
$ minikube start
😄 minikube v1.7.3 on Darwin 10.13.6
✨ Automatically selected the hyperkit driver. Other choices: virtualbox, docker (experimental)
(省略)
🌟 Enabling addons: default-storageclass, storage-provisioner
⌛ Waiting for cluster to come online ...
🏄 Done! kubectl is now configured to use "minikube"
$ kubectl cluster-info
Kubernetes master is running at https://192.168.64.5:8443
KubeDNS is running at https://192.168.64.5:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.3", GitCommit:"06ad960bfd03b39c8310aaf92d1e7c12ce618213", GitTreeState:"clean", BuildDate:"2020-02-13T18:07:54Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.3", GitCommit:"06ad960bfd03b39c8310aaf92d1e7c12ce618213", GitTreeState:"clean", BuildDate:"2020-02-11T18:07:13Z", GoVersion:"go1.13.6", Compiler:"gc", Platform:"linux/amd64"}

次回はサンプルアプリを実行しながらKubernetesオブジェクトを理解していく