Kubernetes Hardway
Kubernetes
Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications.
Kubernetes The Hard Way is optimized for learning, which means taking the long route to ensure you understand each task required to bootstrap a Kubernetes cluster.
Prerequisites
Using Three servers
Master 192.168.1.1
Minion 192.168.1.2
Minion 192.168.1.3
Make sure that each of these servers has at least 1 core and 2 GB of memory
Disable Firewall
Make the status of the SELINUX disabled.
Cluster Configuration
Infrastructure Private subnet IP range: 192.168.1.1/16
Flannel subnet IP range: 172.30.0.0/16 (You can choose any IP range just make sure it does not overlap with any other IP range)
Service Cluster IP range for Kubernetes: 10.254.0.0/16 (You can choose any IP range just make sure it does not overlap with any other IP range)
Kubernetes Service IP: 10.254.0.1 (First IP from service cluster IP range is always allocated to Kubernetes Service)
DNS service IP: 10.254.3.100 (You can use any IP from the service cluster IP range just make sure that the IP is not allocated to any other service)
Step 1: Create a Repo on all Host i.e. Master, Minion1, and Minion2.
# vi /etc/yum.repos.d/virt7-docker-common-release.repo
[virt7-docker-common-release]
name=virt7-docker-common-release
baseurl=http://cbs.centos.org/repos/virt7-docker-common-release/x86_64/os/
gpgcheck=0
# yum update –y
Step 2: installing kubernetes, etcd and flannel
# yum -y install --enablerepo=virt7-docker-common-release kubernetes etcd flannel
Step 3: configure Kubernetes components
Kubernetes Common Configuration (On All Nodes) includes master and nodes.
# vi /etc/kubernetes/config
KUBE_ETCD_SERVERS="--etcd-servers=http://192.168.xxx.xxx:2379"
KUBE_LOGTOSTDERR="--logtostderr=true"
KUBE_LOG_LEVEL="--v=0"
KUBE_ALLOW_PRIV="--allow-privileged=false"
KUBE_MASTER="--master=http://192.168.xxx.xxx:8080"
etcd configuration (on master)
# vi /etc/etcd/etcd.conf
#[member]
ETCD_NAME=default
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
#[cluster]
ETCD_ADVERTISE_CLIENT_URLS="http://0.0.0.0:2379"
api server configuration (on master)
API Server handles the REST operations and acts as a front-end to the cluster’s shared state.
API Server Configuration is stored at /etc/kubernetes/apiserver.
Kubernetes uses certificates to authenticate API requests. Before configuring the API server, we need to generate certificates that can be used for authentication. Kubernetes provides ready-made scripts for generating these certificates.
# vi make-ca-cert.sh
#!/bin/bash
set -o errexit
set -o nounset
set -o pipefail
DEBUG="${DEBUG:-false}"
if [ "${DEBUG}" == "true" ]; then
set -x
fi
cert_ip=$1
extra_sans=${2:-}
cert_dir=${CERT_DIR:-/srv/kubernetes}
cert_group=${CERT_GROUP:-kube}
mkdir -p "$cert_dir"
use_cn=false
if [ "$cert_ip" == "_use_gce_external_ip_" ]; then
cert_ip=$(curl -s -H Metadata-Flavor:Google http://metadata.google.internal./computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip)
fi
if [ "$cert_ip" == "_use_aws_external_ip_" ]; then
if cert_ip=$(curl -f -s http://169.254.169.254/latest/meta-data/public-ipv4); then
:
else
cert_ip=$(curl -f -s http://169.254.169.254/latest/meta-data/local-ipv4)
fi
fi
sans="IP:${cert_ip}"
if [[ -n "${extra_sans}" ]]; then
sans="${sans},${extra_sans}"
fi
tmpdir=$(mktemp -d -t kubernetes_cacert.XXXXXX)
trap 'rm -rf "${tmpdir}"' EXIT
cd "${tmpdir}"
if [ -f ~/kube/easy-rsa.tar.gz ]; then
ln -s ~/kube/easy-rsa.tar.gz .
else
curl -L -O https://storage.googleapis.com/kubernetes-release/easy-rsa/easy
rsa.tar.gz > /dev/null 2>&1
fi
tar xzf easy-rsa.tar.gz > /dev/null 2>&1
cd easy-rsa-master/easyrsa3
./easyrsa init-pki > /dev/null 2>&1
./easyrsa --batch "--req-cn=$cert_ip@`date +%s`" build-ca nopass > /dev/null 2>&1
if [ $use_cn = "true" ]; then
./easyrsa build-server-full $cert_ip nopass > /dev/null 2>&1
cp -p pki/issued/$cert_ip.crt "${cert_dir}/server.cert" > /dev/null 2>&1
cp -p pki/private/$cert_ip.key "${cert_dir}/server.key" > /dev/null 2>&1
else
./easyrsa --subject-alt-name="${sans}" build-server-full kubernetes-master
nopass > /dev/null 2>&1
cp -p pki/issued/kubernetes-master.crt "${cert_dir}/server.cert" > /dev/null 2>&1
cp -p pki/private/kubernetes-master.key "${cert_dir}/server.key" > /dev/null 2>&1
fi
./easyrsa build-client-full kubecfg nopass > /dev/null 2>&1
cp -p pki/ca.crt "${cert_dir}/ca.crt"
cp -p pki/issued/kubecfg.crt "${cert_dir}/kubecfg.crt"
cp -p pki/private/kubecfg.key "${cert_dir}/kubecfg.key"
chgrp $cert_group "${cert_dir}/server.key" "${cert_dir}/server.cert"
"${cert_dir}/ca.crt"
chmod 660 "${cert_dir}/server.key" "${cert_dir}/server.cert" "${cert_dir}/ca.crt"
Now, run the script with following parameters to create certificates:
# bash make-ca-cert.sh "192.168.1.1" "IP:192.168.1.1,IP:10.254.0.1,DNS:kubernetes,DNS:kubernetes.default,DNS:kubernetes.default.svc,DNS:kubernetes.default.svc.cluster.local"
Where 192.168.1.1 is the IP of the master server, 10.254.0.1 is the IP of the Kubernetes service. Now, we can configure the API server:
# vi /etc/kubernetes/apiserver
# Bind kube API server to this IP
KUBE_API_ADDRESS="--address=0.0.0.0"
# Port that Kube API server listens to.
KUBE_API_PORT="--port=8080"
# Port kubelet listen on
KUBELET_PORT="--kubelet-port=10250"
# Address range to use for services(Work unit of Kubernetes)
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
# default admission control policies
KUBE_ADMISSION_CONTROL="--admission><span "font-family:="" -apple-system;="" color:="" rgb(33,="" 37,="" 41);="" font-size:="" 18px;"="">control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota"
# Add your own!
KUBE_API_ARGS="--client-ca-file=/srv/kubernetes/ca.crt --tls-cert-file=/srv/kubernetes/server.cert --tls-private-key-file=/srv/kubernetes/server.key"
Controller Manager Configuration (On Master)
# vi /etc/kubernetes/controller-manager
# Add your own!
KUBE_CONTROLLER_MANAGER_ARGS="--root-ca-file=/srv/kubernetes/ca.crt --service-account-private-key-file=/srv/kubernetes/server.key"
Kubelet Configuration (On Minions)
Kubelet is a node/minion agent that runs pods and makes sure that it is healthy. It also communicates pod details to Kubernetes Master. Kubelet configuration is stored
/etc/kubernetes/kubelet
[On Minion1]
vi /etc/kubernetes/kubelet
# kubelet bind ip address(Provide private ip of minion)
KUBELET_ADDRESS="--address=0.0.0.0"
# port on which kubelet listen
KUBELET_PORT="--port=10250"
# leave this blank to use the hostname of server
KUBELET_HOSTNAME="--hostname-override=192.168.1.2"
# Location of the api-server
KUBELET_API_SERVER="--api-servers=http://192.168.1.1:8080"
KUBELET_ARGS=""
[On Minion2]
vi /etc/kubernetes/kubelet
# kubelet bind ip address(Provide private ip of minion)
KUBELET_ADDRESS="--address=0.0.0.0"
# port on which kubelet listen
KUBELET_PORT="--port=10250"
# leave this blank to use the hostname of server
KUBELET_HOSTNAME="--hostname-override=192.168.1.3"
# Location of the api-server
KUBELET_API_SERVER="--api-servers=http://192.168.1.1:8080"
KUBELET_ARGS=""
Create network configuration for Flannel in etcd. (On Master)
1.Start the etcd node on the master using the following command:
# systemctl start etcd
2. Create a new key in etcd to store Flannel configuration using the following command:
# etcdctl mkdir /kube-centos/network
3. Define the network configuration for Flannel
# etcdctl mk /kube-centos/network/config "{ \"Network\": \"172.30.0.0/16\",
\"SubnetLen\": 24, \"Backend\": { \"Type\": \"vxlan\" } }"
The above command allocates the 172.30.0.0/16 subnet to the Flannel network. A flannel
subnet of CIDR 24 is allocated to each server in Kubernetes cluster.
Step 4: start services on master and minion
On Master
systemctl enable kube-apiserver
systemctl start kube-apiserver
systemctl enable kube-controller-manager
systemctl start kube-controller-manager
systemctl enable kube-scheduler
systemctl start kube-scheduler
systemctl enable flanneld
systemctl start flanneld
On Minions
systemctl enable kube-proxy
systemctl start kube-proxy
systemctl enable kubelet
systemctl start kubelet
systemctl enable flanneld
systemctl start flanneld
systemctl enable docker
systemctl start docker
Relevant Blogs:
Recent Comments
No comments
Leave a Comment
We will be happy to hear what you think about this post