https://github.com/kubernetes/kubernetes/blob/master/docs/user-guide/connecting-applications.md


사전 Docker Registry 를 만든 다음에

http://www.ahnseungkyu.com/206


1. tomcat RC 생성

$ cd Documents/registry/tomcat


$ vi tomcat8-rc.yaml


apiVersion: v1

kind: ReplicationController

metadata:

  name: tomcat8

  labels:

    name: tomcat8

spec:

  replicas: 1

  selector:

    name: tomcat8

  template:

    metadata:

      labels:

        name: tomcat8

    spec:

      containers:

      - name: tomcat8

        image: privateregistry.com:5000/tomcat-jre8:8.0.30

        ports:

        - containerPort: 8080



$ kubectl -s 192.168.230.211:8080 create -f tomcat8-rc.yaml


$ kubectl -s 192.168.230.211:8080 get rc tomcat8                    # 조회


2. tomcat service 생성

$ vi tomcat8-svc.yaml


apiVersion: v1

kind: Service

metadata:

  labels:

    name: tomcat8

  name: tomcat8

spec:

  ports:

    # the port that this service should serve on

    - port: 8088                     # Service 자신의 포트

      targetPort: 8080             # pod 내 컨테이너 포트

      nodePort: 30001

  # label keys and values that must match in order to receive traffic for this service

  selector:                            # 뒷단의 pod 와 연계

    name: tomcat8

  type: NodePort


$ kubectl create -f tomcat8-svc.yaml



[ 서비스 확인 ]

http://192.168.75.212:30001/

http://192.168.75.213:30001/



$ kubectl describe pod tomcat8-5pchl

$ kubectl get rc

$ kubectl describe rc tomcat8

$ kubectl get service

$ kubectl describe service tomcat8


$ kubectl get endpoints


$ $ kubectl get event



[ label 로 조회하기 ]

$ kubectl get service -a -l name=tomcat8

$ kubectl get pods -l name=tomcat8 -o json | grep podIP


[ 전체 조회하기 ]

$ kubectl get --all-namespaces -a service


[ container 안으로 들어가기 ]

$ kubectl exec [ pod 명 ] -c [ Container 명 ] -i -t -- bash -il

$ kubectl exec tomcat8-5pchl -c tomcat8 -i -t -- bash -il


[ Built-in 서비스 확인 ]

$ kubectl cluster-info



[ 어떻게 접근하는지 ]

$ kubectl describe svc tomcat8


Name: tomcat8

Namespace: default

Labels: name=tomcat8

Selector:         name=tomcat8

Type: NodePort

IP:         192.168.230.17                         # Service ip

Port:         <unnamed> 8088/TCP          # Service port

NodePort:         <unnamed> 30001/TCP

Endpoints:         172.16.84.4:8080                       #  Pod ip, port

Session Affinity: None

No events.


# node01 혹은 node02 에서 서비스 IP 포트로 접속 가능

curl -k http://192.168.230.17:8088


# node01 혹은 node02 에서 pod 에 직접 호출

$ kubectl get pods -o json | grep -i podip

$ curl -k http://172.16.84.4:8080




$ kubectl exec tomcat8-5pchl -- printenv | grep SERVICE

KUBERNETES_SERVICE_HOST=192.168.230.1

KUBERNETES_SERVICE_PORT=443

KUBERNETES_SERVICE_PORT_HTTPS=443


$ kubectl scale rc tomcat8 --replicas=0; kubectl scale rc tomcat8 --replicas=2


$ kubectl get pods -l name=tomcat8 -o wide

NAME            READY     STATUS    RESTARTS   AGE       NODE

tomcat8-dqvcu   1/1       Running   0          35s       192.168.75.212

tomcat8-sppk6   1/1       Running   0          35s       192.168.75.212


$ kubectl exec tomcat8-dqvcu -- printenv | grep SERVICE

KUBERNETES_SERVICE_PORT=443

TOMCAT8_SERVICE_PORT=8088

KUBERNETES_SERVICE_HOST=192.168.230.1

KUBERNETES_SERVICE_PORT_HTTPS=443

TOMCAT8_SERVICE_HOST=192.168.230.17


3. DNS 확인

$ vi curlpod.yaml


apiVersion: v1

kind: Pod

metadata:

  labels:

    name: curlpod

  name: curlpod

spec:

  containers:

  - image: radial/busyboxplus:curl

    command:

      - sleep

      - "3600"

    imagePullPolicy: IfNotPresent

    name: curlcontainer

  restartPolicy: Always



$ kubectl create -f curlpod.yaml

kubectl describe pod curlpod



[ DNS 확인 ]

$ kubectl exec curlpod -- nslookup tomcat8

kubectl exec curlpod -- curl http://tomcat8:8088



$ kubectl exec curlpod -c curlcontainer -it -- /bin/sh -il



4. 각 인스턴스 pod 에 접속

https://github.com/kubernetes/kubernetes/blob/master/docs/user-guide/accessing-the-cluster.md#accessing-services-running-on-the-cluster


$ kubectl get pods


http://192.168.75.211:8080/api/v1/proxy/namespaces/default/pods/tomcat8-dqvcu/


# docker id 조회

docker ps -l -q












Posted by Kubernetes Korea co-leader seungkyua@gmail.com

먼저 kubernetes cluster 를 설치 해야 함

http://www.ahnseungkyu.com/200


1. Create self-signed certificate

$ cd Documents

$ mkdir registry

$ cd registry


$ mkdir -p certs && openssl req \

-newkey rsa:4096 -nodes -sha256 -keyout certs/domain.key \

-x509 -days 36500 -out certs/domain.crt


Country Name (2 letter code) [AU]:  

State or Province Name (full name) [Some-State]:

Locality Name (eg, city) []:

Organization Name (eg, company) [Internet Widgits Pty Ltd]:

Organizational Unit Name (eg, section) []:

Common Name (e.g. server FQDN or YOUR name) []:privateregistry.com

Email Address []:



2. 패스워드 파일 생성 (이건 나중에)

$ mkdir -p auth

$ docker run --entrypoint htpasswd registry:2 -Bbn test test > auth/htpasswd



3. cert 파일 복사

$ vi deployCert.sh

#!/bin/bash


FQDN=privateregistry.com


echo $FQDN


sudo mkdir -p /etc/docker/certs.d/$FQDN

sudo cp certs/domain.crt /etc/docker/certs.d/$FQDN/ca.crt


sudo mkdir -p /opt/docker_volumes/registry/$FQDN

sudo mkdir -p /opt/docker_volumes/registry/$FQDN/data

sudo cp -r certs /opt/docker_volumes/registry/$FQDN


$ ./deployCert.sh



# Ubuntu 에 Cert 설치

$ sudo cp /home/stack/Documents/registry/certs/domain.crt /usr/local/share/ca-certificates/.

$ sudo update-ca-certificates


# docker restart

$ sudo service docker restart



# node01, node02 에도 cert 복사

$ sudo mkdir -p /etc/docker/privateregistry.com

$ sudo cp /home/stack/Documents/registry/certs/domain.crt /etc/docker/privateregistry.com/ca.crt


$ sudo cp /home/stack/Documents/registry/certs/domain.crt /usr/local/share/ca-certificates/.

$ sudo update-ca-certificates


# docker restart

$ sudo service docker restart


$ sudo vi /etc/hosts

192.168.75.211  privateregistry.com


4. Registry 생성

# docker-compose 로 실행

# root 권한으로 변환해서 docker-compose 설치

$ sudo su -

# curl -L https://github.com/docker/compose/releases/download/1.5.2/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose


$ vi kube-registry.yml


kube-registry:

  container_name: kube-registry

  restart: always

  image: registry:2

  ports:

    - 5000:5000

  environment:

    REGISTRY_HTTP_TLS_CERTIFICATE: /certs/domain.crt

    REGISTRY_HTTP_TLS_KEY: /certs/domain.key

    REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /var/lib/registry

  volumes:

    - /opt/docker_volumes/registry/privateregistry.com/data:/var/lib/registry

    - /opt/docker_volumes/registry/privateregistry.com/certs:/certs


$ docker-compose -f kube-registry.yml up -d



# docker run 으로 실행

docker run -d -p 5000:5000 --restart=always --name kube-registry \

  -v `pwd`/certs:/certs \

  -v /opt/docker_volumes/registry/privateregistry.com/data:/var/lib/registry \

  -e REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY=/var/lib/registry \

  -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \

  -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \

  registry:2


5. Registry 확인

https://192.168.230.211:5000/v2/_catalog


# node01 확인

$ sudo vi /etc/hosts

192.168.75.211  privateregistry.com


$ docker pull ubuntu

$ docker tag ubuntu privateregistry.com:5000/ubuntu

$ docker push privateregistry.com:5000/ubuntu


# master 에서 확인

$ docker pull privateregistry.com:5000/ubuntu


6. Registry 삭제

docker stop kube-registry && docker rm kube-registry


7. image 조회

docker images privateregistry.com:5000



8. Tomcat8 Docker file 만들기

$ mkdir -p tomcat

$ cd tomcat

$ vi Dockerfile


FROM java:8-jre


ENV CATALINA_HOME /usr/local/tomcat

ENV PATH $CATALINA_HOME/bin:$PATH

RUN mkdir -p "$CATALINA_HOME"

WORKDIR $CATALINA_HOME


# runtime dependency for Tomcat Native Libraries

RUN apt-get update && apt-get install -y libapr1 && rm -rf /var/lib/apt/lists/*


# see https://www.apache.org/dist/tomcat/tomcat-8/KEYS

RUN set -ex \

&& for key in \

05AB33110949707C93A279E3D3EFE6B686867BA6 \

07E48665A34DCAFAE522E5E6266191C37C037D42 \

47309207D818FFD8DCD3F83F1931D684307A10A5 \

541FBE7D8F78B25E055DDEE13C370389288584E7 \

61B832AC2F1C5A90F0F9B00A1C506407564C17A3 \

79F7026C690BAA50B92CD8B66A3AD3F4F22C4FED \

9BA44C2621385CB966EBA586F72C284D731FABEE \

A27677289986DB50844682F8ACB77FC2E86E29AC \

A9C5DF4D22E99998D9875A5110C01C5A2F6059E7 \

DCFD35E0BF8CA7344752DE8B6FB21E8933C60243 \

F3A04C595DB5B6A5F1ECA43E3B7BBB100D811BBE \

F7DA48BB64BCB84ECBA7EE6935CD23C10D498E23 \

; do \

gpg --keyserver ha.pool.sks-keyservers.net --recv-keys "$key"; \

done


ENV TOMCAT_MAJOR 8

ENV TOMCAT_VERSION 8.5.0

ENV TOMCAT_TGZ_URL https://www.apache.org/dist/tomcat/tomcat-$TOMCAT_MAJOR/v$TOMCAT_VERSION/bin/apache-tomcat-$TOMCAT_VERSION.tar.gz


# Tomcat Native 1.2+ requires a newer version of OpenSSL than debian:jessie has available (1.0.2g+)

# see http://tomcat.10.x6.nabble.com/VOTE-Release-Apache-Tomcat-8-0-32-tp5046007p5046024.html (and following discussion)


RUN set -x \

\

&& curl -fSL "$TOMCAT_TGZ_URL" -o tomcat.tar.gz \

&& curl -fSL "$TOMCAT_TGZ_URL.asc" -o tomcat.tar.gz.asc \

&& gpg --batch --verify tomcat.tar.gz.asc tomcat.tar.gz \

&& tar -xvf tomcat.tar.gz --strip-components=1 \

&& rm bin/*.bat \

&& rm tomcat.tar.gz* \

\

&& nativeBuildDir="$(mktemp -d)" \

&& tar -xvf bin/tomcat-native.tar.gz -C "$nativeBuildDir" --strip-components=1 \

&& nativeBuildDeps=" \

gcc \

libapr1-dev \

libssl-dev \

make \

openjdk-${JAVA_VERSION%%[-~bu]*}-jdk=$JAVA_DEBIAN_VERSION \

" \

&& apt-get update && apt-get install -y --no-install-recommends $nativeBuildDeps && rm -rf /var/lib/apt/lists/* \

&& ( \

export CATALINA_HOME="$PWD" \

&& cd "$nativeBuildDir/native" \

&& [ "$(openssl version | cut -d' ' -f2)" = '1.0.1k' ] \

# http://tomcat.10.x6.nabble.com/VOTE-Release-Apache-Tomcat-8-0-32-tp5046007p5048274.html (ie, HACK HACK HACK)

&& cp src/sslcontext.c src/sslcontext.c.orig \

&& awk ' \

/^    eckey = EC_KEY_new_by_curve_name/ { print "    EC_KEY *eckey = NULL;" } \

{ print } \

' src/sslcontext.c.orig > src/sslcontext.c \

&& ./configure \

--libdir=/usr/lib/jni \

--prefix="$CATALINA_HOME" \

--with-apr=/usr/bin/apr-1-config \

--with-java-home="$(docker-java-home)" \

--with-ssl=yes \

&& make -j$(nproc) \

&& make install \

) \

&& apt-get purge -y --auto-remove $nativeBuildDeps \

&& rm -rf "$nativeBuildDir" \

&& rm bin/tomcat-native.tar.gz


# verify Tomcat Native is working properly

RUN set -e \

&& nativeLines="$(catalina.sh configtest 2>&1)" \

&& nativeLines="$(echo "$nativeLines" | grep 'Apache Tomcat Native')" \

&& nativeLines="$(echo "$nativeLines" | sort -u)" \

&& if ! echo "$nativeLines" | grep 'INFO: Loaded APR based Apache Tomcat Native library' >&2; then \

echo >&2 "$nativeLines"; \

exit 1; \

fi


EXPOSE 8080

CMD ["catalina.sh", "run"]


$ docker build -tag tomcat-jre8:8 .                # 처음 이미지는 . 을 추가할 수 없음

docker tag tomcat-jre8:8 tomcat-jre8:8.5.0         # 태그에 . 을 추가

$ docker rmi tomcat-jre8:8                          # 처음 태그를 삭제



# 태그이름을 리모트로 변경

docker tag tomcat-jre8:8.5.0 privateregistry.com:5000/tomcat-jre8:8.5.0


# 태그 이름이 리모트이므로 리모트로 올리게 됨

$ docker push privateregistry.com:5000/tomcat-jre8:8.0.30



# node01 에서 확인

$ docker pull privateregistry.com:5000/tomcat-jre8:8.0.30



$ https://192.168.230.211:5000/v2/tomcat-jre8/tags/list

$ curl https://privateregistry.com:5000/v2/tomcat-jre8/tags/list











Posted by Kubernetes Korea co-leader seungkyua@gmail.com

haproxy 설치

Linux/Ubuntu 2016.01.09 15:15

1. hpproxy install

$ sudo apt-get install haproxy


$ sudo vi /etc/haproxy/haproxy.cfg

...

defaults

log        global

mode    http

retries   3                  # 추가

option   httplog

option   dontlognull

option   redispatch      # 추가 : 한 서버가 죽으면 다른 서버로 보내라

...

...

listen serv 0.0.0.0:80        # 추가 : serv 는 아무 이름이나 줘도 됨

mode http

option http-server-close

timeout http-keep-alive 3000             # 추가 : 이미지 같은 것은 하나의 컨넥션으로 연결하기 위해

server serv 127.0.0.1:9000 check       # server1, server2 이런 식으로 서버 이름을 준다.


$ sudo service haproxy reload


















Posted by Kubernetes Korea co-leader seungkyua@gmail.com

1. 다운로드

https://golang.org/doc/install?download=go1.5.2.darwin-amd64.tar.gz     # Mac

https://storage.googleapis.com/golang/go1.5.2.linux-amd64.tar.gz           # Linux


$ sudo tar -C /usr/local -xzf go1.5.2.darwin-amd64.tar.gz

$ cd /usr/local

$ sudo chown -R root go


2. 환경 변수

sudo vi /etc/profile

export GOROOT=/usr/local/go                                 # go 설치 위치

export PATH=$PATH:/usr/local/go/bin                      # go 실행파일 위치


$ cd Documents

mkdir -p go_workspace{,/bin,/pkg,/src}


vi .bash_profile 

export GOPATH=$HOME/Documents/go_workspace                     # go workspace 위치

export PATH=$HOME/Documents/go_workspace/bin:$PATH         # go 실행파일 위치



## go tool 다운로드

$ got get golang.org/x/tools/cmd/...



3. go 샘플 다운로드

go get github.com/GoesToEleven/GolangTraining


# kubernetes 소스 다운로드

$ go get k8s.io/kubernetes       # 이렇게 하면 git clone https://github.com/kubernetes/kubernetes


4. Go Workspace 디렉토리 위치

- bin

- pkg

- src - github.com - GoesToEleven - GolangTraining



5. editor WebStorm 다운로드 및 세팅

https://www.jetbrains.com/webstorm/download/

버전 : WebStorm-11.0.3-custom-jdk-bundled.dmg



6. golang plugin 설치

https://plugins.jetbrains.com/plugin/5047?pr=idea

버전 : Go-0.10.749.zip


# Project Open

/Users/ahnsk/Documents/go_workspace/src/github.com/GoesToEleven/GolangTraining


# Preferences 세팅

Go SDK : /usr/local/go

Go Libraries : go_worksapce/src



7. theme 다운로드 및 설정

http://color-themes.com/?view=index

Sublime Text 2.jar 다운로드


File >> import Settings 에서 Sublime Text 2.jar 선택


# Preferences 세팅

Editor -> Colors & Fonts : Scheme을 Sublime Text2로 설정



8. JavaScript Debug 를 위한 live edit plugin 설치

https://plugins.jetbrains.com/plugin/7007?pr=pycharm

LiveEdit.jar 다운로드


# Preferences 세팅

Build, Execution, Deployment -> Debugger -> Live Edit

체크 : Highlight current....

Update Auto in (ms):   16 


# 우측 상단 돋보기 클릭하여 Edit Configuration 조회

창에서 좌측 상단 + 클릭 후 JavaScript Debug 추가


# chrom 웹 스토어에서 확장 프로그램 설치

JetBrains IDE Support



# WebStorm 단축키

파일찾기 : Command + Shift + O

단어찾기 : Command + Shift + F

실행       : Crtl + Alt + R

디버그    : Ctrl + Alt + D

줄삭제    : Command + Backspace

줄복사    : Command + D

포맷       : Command + Alt + L



# go file 규칙 테스트

$ gofmt -s -w file.go


$ git rebase -i   혹은    git push -f   로 작업의 논리적인 유닛으로 커밋



# Docker contribute 시에 DCO (Developer Certificate of Origin) 설정

# commit 마다 설정

Docker-DCO-1.1-Signed-off-by: Seungkyu Ahn <seungkyua@gmail.com> (github: seungkyua)



# 혹은 hook 를 설정

$ cd docker

$ curl -o .git/hooks/prepare-commit-msg \

https://raw.githubusercontent.com/dotcloud/docker/master/contrib/prepare-commit-msg.hook

$ chmod -x .git/hooks/prepare-commit-msg



# github user 를 세팅

$ git config -set github.user seungkyua



# Channel

# deadlock 을 막을려면 채널로 값을 보내는 쪽에서 close 채널을 해야 한다.

# 채널을 받는 쪽에서는 defer sync.WaitGroup.Done() 을 한다.

# 혹은 새로운 go 루틴을 만들고 sync.WaitGroup.Wait() 으로 끝나길 기달려서 close 채널을 한다.




# 문서 보기

## 문법 에러 검사

$ go vet wordcount.go


## tar 패키지 사용법 보기

$ go doc tar


## 로컬 문서 서버 띄우기

$ godoc -http=:6060



# godep 설치

go get github.com/tools/godep

$ cd ~/Documents/go_workspace/src/github.com/tools/godep

$ go install


## godep 을 사용하는 프로젝트로 이동

$ cd ~/Documents/go_workspace/src/k8s.io/kubernetes/


## godep get 으로 Godeps/_workspace 에 패키지를 다운한다.

## _workspace 는 deprecated 예정

godep get 패키지명


 







Posted by Kubernetes Korea co-leader seungkyua@gmail.com

유용한 Site

프로그래밍 2015.12.22 13:12

1. Kubernates

    구글 논문 : https://research.google.com/pubs/pub43438.html

    구글 발표 : https://speakerdeck.com/jbeda/containers-at-scale











슬라이드 작성

http://prezi.com



















Posted by Kubernetes Korea co-leader seungkyua@gmail.com

 0. 서버 설정

Master   : 192.168.75.211  (etcd, kube-apiserver, kube-controller-manager, kube-scheduler)

Node01  : 192.168.75.212  (kube-proxy, kubelet)

Node02  : 192.168.75.213  (kube-proxy, kubelet)


etcd-2.2.1, flannel-0.5.5, k8s-1.1.2



[ Master Node 서버에 모두 설치 ]

1. apt-get 으로 필요 s/w 설치

# docker 설치

$ sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D

$ sudo vi /etc/apt/sources.list.d/docker.list


# Debian Jessie

#deb https://apt.dockerproject.org/repo debian-jessie main


# Debian Stretch/Sid

#deb https://apt.dockerproject.org/repo debian-stretch main


# Ubuntu Precise

#deb https://apt.dockerproject.org/repo ubuntu-precise main


# Ubuntu Trusty (14.04 LTS)

deb https://apt.dockerproject.org/repo ubuntu-trusty main


# Ubuntu Utopic (14.10)

#deb https://apt.dockerproject.org/repo ubuntu-utopic main


# Ubuntu Vivid (15.04)

#deb https://apt.dockerproject.org/repo ubuntu-vivid main


# Ubuntu Wily (15.10)

#deb https://apt.dockerproject.org/repo ubuntu-wily main


$ sudo apt-get update

$ sudo apt-get purge lxc-docker*

$ sudo apt-get purge docker.io

$ sudo apt-get autoremove

$ sudo apt-get install docker-engine


$ sudo apt-get install bridge-utils

sudo usermod -a -G docker stack      # stack user에 docker 그룹을 추가

$ sudo service docker restart



2. sudo 세팅

# gpasswd -a stack sudo   (이건 안되는데??)

stack   ALL=(ALL:ALL) NOPASSWD: ALL



3. ntp 설치 & ssh 키 설치

# ssh 로 master <-> Node 사이에 stack 계정으로 바로 접속할 수 있어야 함

# ssh 로 master, Node 각각 자기 서버 내에서 stack 계정에서 root 계정으로 바로 접속할 수 있어야 함



4. host 세팅

192.168.75.211    master

192.168.75.212    node01

192.168.75.213    node02



5. Go 설치

1. 다운로드

$ cd /home/stack/downloads

wget https://storage.googleapis.com/golang/go1.5.2.linux-amd64.tar.gz

sudo tar -C /usr/local -xzf go1.5.2.linux-amd64.tar.gz


2. 환경변수 세팅

sudo vi /etc/profile

export GOROOT=/usr/local/go

export PATH=$PATH:/usr/local/go/bin


sudo visudo             # sudo 에서도 go path가 적용될려면 여기에 세팅

Defaults    env_reset

Defaults    env_keep += "GOPATH"

Defaults        secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/go/bin"


$ cd

vi .bash_profile

export GOPATH=$HOME/Documents/go_workspace:$HOME/Documents/go_workspace/src/k8s.io/kubernetes/Godeps/_workspace

export PATH=$HOME/Documents/go_workspace/bin:$PATH



6. kubernetes 설치

# go 로 다운로드하기

$ go get k8s.io/kubernetes   # git clone https://github.com/kubernetes/kubernetes.git


$ cd ~/Documents/go_workspace/src/k8s.io/kubernetes

$ git checkout -b v1.1.2 tags/v1.1.2

$ make all                                      # _output 디렉토리에 결과 파일이 생성


# 소스 수정 후 make 로 재빌드 (참고)   _output 디렉토리에 결과 파일이 생성

$ make all WHAT=plugin/cmd/kube-scheduler GOFLAGS=-v      # scheduler

$ make all WHAT=cmd/kubelet GOFLAGS=-v                           # kubelet

$ make all WHAT=cmd/kube-apiserver GOFLAGS=-v                # apiserver


# 소스 수정 후 재빌드 (참고)

$ hack/build-go.sh                  # make를 돌리면 build-go.sh 가 수행됨
$ hack/local-up-cluster.sh        # 로컬 클러스터를 생성할 때


$ sudo su -

# cd ~/Documents/go_workspace/src/k8s.io/kubernetes/cluster/ubuntu


# export KUBE_VERSION=1.1.2

# export FLANNEL_VERSION=0.5.5

# export ETCD_VERSION=2.2.1


# ./build.sh                 # binaries 디렉토리에 다운 받음

# exit



$ cd ~/Documents/go_workspace/src/k8s.io/kubernetes/cluster/ubuntu

$ vi config-default.sh


export nodes="stack@192.168.75.211 stack@192.168.75.212"

export role="a i"

export NUM_MINIONS=${NUM_MINIONS:-1}

export SERVICE_CLUSTER_IP_RANGE=192.168.230.0/24

export FLANNEL_NET=172.16.0.0/16



ENABLE_CLUSTER_DNS="${KUBE_ENABLE_CLUSTER_DNS:-true}"

DNS_SERVER_IP=${DNS_SERVER_IP:-"192.168.230.10"}

DNS_DOMAIN="cluster.local"

DNS_REPLICAS=1


ENABLE_CLUSTER_UI="${KUBE_ENABLE_CLUSTER_UI:-true}"


$ cd ~/Documents/go_workspace/src/k8s.io/kubernetes/cluster

$ KUBERNETES_PROVIDER=ubuntu ./kube-up.sh


# 복사한 파일

make-ca-cert.sh    

reconfDocker.sh    

config-default.sh    

util.sh    

kube-scheduler.conf    

kube-apiserver.conf    

etcd.conf    

kube-controller-manager.conf    

flanneld.conf    

kube-controller-manager    

kube-scheduler    

etcd    

kube-apiserver    

flanneld    

kube-controller-manager    

etcdctl    

kube-scheduler    

etcd    

kube-apiserver    

flanneld



# kubectl 복사

$ sudo cp ubuntu/binaries/kubectl /opt/bin/.


# 경로 추가

$ vi ~/.bash_profile

export PATH=/opt/bin:$PATH

export KUBECTL_PATH=/opt/bin/kubectl



# Add-on 설치

$ cd ~/Documents/go_workspace/src/k8s.io/kubernetes/cluster/ubuntu

$ KUBERNETES_PROVIDER=ubuntu ./deployAddons.sh


# 에러 발생하면 아래 실행 (Docker image 를 다운로드 함)

$ cd ~/Documents/go_workspace/src/k8s.io/kubernetes

./build/run.sh hack/build-cross.sh


# Add-on 설치 다시

cd ~/Documents/go_workspace/src/k8s.io/kubernetes/cluster/ubuntu

$ KUBERNETES_PROVIDER=ubuntu ./deployAddons.sh



[ Kubernetes 설치 지우기 ]

$ cd ..

$ KUBERNETES_PROVIDER=ubuntu ./kube-down.sh


# node01 에 떠 있는 docker 삭제하기

docker ps -a | awk '{print $1}' | xargs docker stop

docker ps -a | awk '{print $1}' | xargs docker rm

$ sudo cp ubuntu/binaries/kubectl /opt/bin/.                # kubectl 을 /opt/bin 에 복사해야 함


$ KUBERNETES_PROVIDER=ubuntu ./kube-up.sh



[ Master의 Docker를 flannel 로 연결 ]

sudo service docker stop

$ sudo ip link set dev docker0 down

$ sudo brctl delbr docker0

$ cat /run/flannel/subnet.env      # flannel의 subnet 과 mtu 값을 확인한다.

$ sudo vi /etc/default/docker

DOCKER_OPTS=" -H tcp://127.0.0.1:4243 -H unix:///var/run/docker.sock --bip=172.16.25.1/24 --mtu=1472"


$ sudo service docker start

$ sudo ip link set dev docker0 up



# node01 에서 docker ps -a 로 가비지가 많이 쌓임. 지워주면 됨

# ssh node01 로 접속하여 가비지 조회

docker ps -a | grep Exited | awk '{print $1}'

docker ps -a | grep Exited | awk '{print $1}' | xargs docker rm


# kubernetes volume 생성되는 곳 : /var/lib/kubelet/pods

# kubernetes garbage-collection https://github.com/kubernetes/kubernetes/blob/master/docs/admin/garbage-collection.md


$ kubectl get nodes

$ kubectl get pods --namespace=kube-system         # add-on pods 확인

$ kubectl cluster-info


# Skydns Pod 정보 보기

kubectl describe pod kube-dns-v9-549av --namespace=kube-system


# DNS 확인

$ kubectl create -f busybox.yaml


$ vi busybox.yaml

apiVersion: v1

kind: Pod

metadata:

  name: busybox

  namespace: default

spec:

  containers:

  - image: busybox

    command:

      - sleep

      - "3600"

    imagePullPolicy: IfNotPresent

    name: busybox

  restartPolicy: Always


$ kubectl get pods busybox


kubectl exec Pod명 [-c Container명] -i -t -- COMMAND [args..] [flags]

$ kubectl exec busybox -- nslookup kubernetes.default


# busybox 삭제하기

$ kubectl delete -f busybox.yaml



# 웹화면 확인

http://192.168.75.211:8080/


# UI 확인

http://192.168.75.211:8080/ui    >> 아래 화면으로 리다이렉션 됨

http://192.168.75.211:8080/api/v1/proxy/namespaces/kube-system/services/kube-ui



# Mac 에서 소스 개발하고 Master 에 소스 커밋하기  (참고)

# 원격에 tag 에도 v1.1.2 가 있고 branch 에도 v1.1.2 가 있으면 remote branch 를 지정

# git push [저장소] (local branch명:)remote branch명

git push origin refs/heads/v1.1.2


git config --global user.name "Seungkyu Ahn" 

git config --global user.email "seungkyua@gmail.com"


# 로컬 파일을 Master 서버로 복사

$ vi ~/bin/cmaster.sh

#!/bin/bash


function change_directory {

  cd /Users/ahnsk/Documents/go_workspace/src/k8s.io/kubernetes

}


change_directory

files=$(git status | grep -E 'modified|new file' | awk -F':' '{print$2}')


for file in $files; do

    scp $file stack@192.168.230.211:/home/stack/Documents/go_workspace/src/k8s.io/kubernetes/$file

done



# kube-apiserver 소스로 띄우기

$ cd ~/Documents/go_workspace/src/k8s.io/kubernetes/cmd/kube-apiserver


sudo -E go run apiserver.go --insecure-bind-address=0.0.0.0 --insecure-port=8080 --etcd-servers=http://127.0.0.1:4001 --logtostderr=true --service-cluster-ip-range=192.168.230.0/24 --admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,DenyEscalatingExec,SecurityContextDeny --service-node-port-range=30000-32767 --client-ca-file=/srv/kubernetes/ca.crt --tls-cert-file=/srv/kubernetes/server.cert --tls-private-key-file=/srv/kubernetes/server.key 



# Document 만들기

$ cd ~/Documents/go_workspace/src/k8s.io/kubernetes/cmd/genkubedocs

$ mkdir -p temp

$  go run gen_kube_docs.go temp kube-apiserver



7. Sample App 올려보기

https://github.com/kubernetes/kubernetes/tree/master/examples/guestbook


# 디렉토리 위치는 kubernetes 설치한 위치

$ sudo kubectl create -f examples/guestbook/redis-master-controller.yaml

$ sudo kubectl get rc

$ sudo kubectl get pods

$ sudo kubectl describe pods/redis-master-xssrd

$ sudo kubectl logs <pod_name>          # container log 확인


$ sudo kubectl create -f examples/guestbook/redis-master-service.yaml

$ sudo kubectl get services


$ sudo kubectl create -f examples/guestbook/redis-slave-controller.yaml

$ sudo kubectl get rc

$ sudo kubectl get pods


$ sudo kubectl create -f examples/guestbook/redis-slave-service.yaml

$ sudo kubectl get services



$ sudo kubectl create -f examples/guestbook/frontend-controller.yaml

$ sudo kubectl get rc

$ sudo kubectl get pods



$ sudo kubectl create -f examples/guestbook/frontend-service.yaml

$ sudo kubectl get services





sudo kubectl describe services frontend

$ sudo kubectl get ep


# dns 보기

$ sudo kubectl get services kube-dns --namespace=kube-system


# 환경변수 보기

$ sudo kubectl get pods -o json

$ sudo kubectl get pods -o wide

$ sudo kubectl exec frontend-cyite -- printenv | grep SERVICE


8. Sample App 삭제

$ sudo kubectl stop rc -l "name in (redis-master, redis-slave, frontend)"

$ sudo kubectl delete service -l "name in (redis-master, redis-slave, frontend)"



# Network

TAP : vm과 eth0 (physical port) 와 연결할 때 사용. tap <-> bridge <-> eth0 로 됨

VETH : docker <-> bridge,  docker <-> OVS, bridge <-> OVS 를 연결할 때 사용


# interconnecting namespaces

http://www.opencloudblog.com/?p=66



# Docker <-> veth 알아내기

$ vi veth.sh


#!/bin/bash


set -o errexit

set -o nounset

#set -o pipefail


VETHS=`ifconfig -a | grep "Link encap" | sed 's/ .*//g' | grep veth`

DOCKERS=$(docker ps -a | grep Up | awk '{print $1}')


for VETH in $VETHS

do

  PEER_IFINDEX=`ethtool -S $VETH 2>/dev/null | grep peer_ifindex | sed 's/ *peer_ifindex: *//g'`

  for DOCKER in $DOCKERS

  do

    PEER_IF=`docker exec $DOCKER ip link list 2>/dev/null | grep "^$PEER_IFINDEX:" | awk '{print $2}' | sed 's/:.*//g'`

    if [ -z "$PEER_IF" ]; then

      continue

    else

      printf "%-10s is paired with %-10s on %-20s\n" $VETH $PEER_IF $DOCKER

      break

    fi

  done

done






Posted by Kubernetes Korea co-leader seungkyua@gmail.com

$ cd /opt

$ sudo wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u20-b26/jdk-8u20-linux-x64.tar.gz"


$ sudo tar -zxvf jdk-8u20-linux-x64.tar.gz


$ sudo update-alternatives --install /usr/bin/java java /opt/jdk1.8.0_20/bin/java 2


$ sudo update-alternatives --config java


There are 2 choices for the alternative java (providing /usr/bin/java).


  Selection    Path                                            Priority   Status

------------------------------------------------------------

* 0            /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java   1071      auto mode

  1            /opt/jdk1.8.0_20/bin/java                                 2         manual mode

  2            /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java   1071      manual mode


Press enter to keep the current choice[*], or type selection number: 1



$ sudo update-alternatives --install /usr/bin/javac javac /opt/jdk1.8.0_20/bin/javac 2

$ sudo update-alternatives --config javac



$ sudo update-alternatives --install /usr/bin/jar jar /opt/jdk1.8.0_20/bin/jar 2

$ sudo update-alternatives --config jar


$ sudo vi .bashrc


export JAVA_HOME=/opt/jdk1.8.0_20

export JRE_HOME=/opt/jdk1.8.0_20/jre

export PATH=$PATH:/opt/jdk1.8.0_20/bin:/opt/jdk1.8.0_20/jre/bin


$ echo $JAVA_HOME

$ echo $JRE_HOME






Posted by Kubernetes Korea co-leader seungkyua@gmail.com

1화 Sand Hill Shuffle

2화 Runaway Devaluation

3화 Bad Money

4화 The Lady

5화 Server Space

6화 Homicide

7화 Adult Content

8화 White Hat / Black Hat

9화 Binding Arbitration

10화 Two Days of the Condor











Posted by Kubernetes Korea co-leader seungkyua@gmail.com

docker ssh + git

Container 2015.08.13 15:20

1. docker 설치하기

# docker 설치

$ sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D

$ sudo vi /etc/apt/sources.list.d/docker.list


# Debian Jessie

#deb https://apt.dockerproject.org/repo debian-jessie main


# Debian Stretch/Sid

#deb https://apt.dockerproject.org/repo debian-stretch main


# Ubuntu Precise

#deb https://apt.dockerproject.org/repo ubuntu-precise main


# Ubuntu Trusty (14.04 LTS)

deb https://apt.dockerproject.org/repo ubuntu-trusty main


# Ubuntu Utopic (14.10)

#deb https://apt.dockerproject.org/repo ubuntu-utopic main


# Ubuntu Vivid (15.04)

#deb https://apt.dockerproject.org/repo ubuntu-vivid main


# Ubuntu Wily (15.10)

#deb https://apt.dockerproject.org/repo ubuntu-wily main


$ sudo apt-get update

$ sudo apt-get purge lxc-docker*

$ sudo apt-get purge docker.io

$ sudo apt-get autoremove

$ sudo apt-get install docker-engine


$ sudo apt-get install bridge-utils

$ sudo usermod -a -G docker stack      # stack user에 docker 그룹을 추가

$ sudo service docker restart


# Mac 에서 Docker 설치하기

$ ruby -e \

"$(curl -fsSL \ https://raw.githubusercontent.com/Homebrew/install/master/install)"


$ brew update

$ brew install caskroom/cask/brew-cask


$ brew cask install virtualbox

$ brew install docker

$ brew install boot2docker


$ boot2docker init

$ boot2docker up


To connect the Docker client to the Docker daemon, please set:

    export DOCKER_HOST=tcp://192.168.59.103:2376

    export DOCKER_CERT_PATH=/Users/ahnsk/.boot2docker/certs/boot2docker-vm

    export DOCKER_TLS_VERIFY=1


$ $(boot2docker shellinit)       # 환경변수 세팅


$ docker info

$ boot2docker ssh                 # vm 접속

$ boot2docker ip                   # vm ip


$ docker run --rm -ti ubuntu:latest /bin/bash        # ubuntu 이미지 테스트

$ docker run --rm -ti fedora:latest /bin/bash         # fedora 이미지 테스트

$ docker run --rm -ti centos:latest /bin/bash         # centos 이미지 테스트


# Upgrade the Boot2docker VM image

$ boot2docker stop

$ boot2docker download

$ boot2docker up


$ boot2docker delete


# Docker Hub 로그인

$ docker login


Username: seungkyua

Password: 

Email: seungkyua@gmail.com


$  cat ~/.docker/config.json


$ docker logout


# Docker Registry 를 insecure 로 변경


# boot2docker

sudo touch /var/lib/boot2docker/profile

$ sudo vi /var/lib/boot2docker/profile

EXTRA_ARGS="--insecure-registry 192.168.59.103:5000"

sudo /etc/init.d/docker restart


# Ubuntu

$ sudo vi /etc/default/docker

DOCKER_OPTS="--insecure-registry 192.168.59.103:5000"

$ sudo service docker restart


# Fedora

$ sudo vi /etc/sysconfig/docker

OPTIONS="--insecure-registry 192.168.59.103:5000"

$ sudo systemctl daemon-reload

$ sudo systemctl restart docker


# CoreOS

$ sudo cp /usr/lib/systemd/system/docker.service /etc/systemd/system/

$ sudo vi  /etc/systemd/system/docker.service

ExecStart=/usr/lib/coreos/dockerd --daemon --host=fd:// \

$DOCKER_OPTS $DOCKER_OPT_BIP $DOCKER_OPT_MTU $DOCKER_OPT_IPMASQ \

--insecure-registry 192.168.59.103:5000

$ sudo systemctl daemon-reload

$ sudo systemctl restart docker


# Local Registry 띄우기

$ sudo mkdir -p /var/lib/registry

$ docker run -d -p 5000:5000 \

-v /var/lib/registry:/var/lib/registry \

--restart=always --name registry registry:2



# 테스트

$ docker pull ubuntu

$ docker tag ubuntu 192.168.59.103:5000/ubuntu


$ docker push 192.168.59.103:5000/ubuntu

$ docker pull 192.168.59.103:5000/ubuntu


$ docker stop registry

$ docker rm -v registry




2. docker file 만들기

# mkdir docker

# cd docker

# mkdir git-ssh

# cd git-ssh

# vi Dockerfile

FROM ubuntu:14.04


RUN apt-get -y update

RUN apt-get -y install openssh-server

RUN apt-get -y install git


# Setting openssh

RUN mkdir /var/run/sshd

RUN sed -i "s/#PasswordAuthentication yes/PasswordAuthentication no/" /etc/ssh/sshd_config


# Adding git user

RUN adduser --system git

RUN mkdir -p /home/git/.ssh


# Clearing and setting authorized ssh keys

RUN echo '' > /home/git/.ssh/authorized_keys

RUN echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDTFEBrNfpSIvgz7mZ+I96/UqKFCxcouoiDDS9/XPNB1Tn7LykgvHHaR5mrPOQIJ/xTFhSVWpwsmEvTLdv3QJYLB5P+UfrjY5fUmiYgGpKKr5ym2Yua2wykHgQYdT4+lLhyq3BKbnG9vgc/FQlaCWntLckJfAYnHIGYWl1yooMAOka0/pOeJ+hPF0TxLQtrjoVJWiaHLVnB8qgPiCgvSyKROvW6cs1AhY9abasUWrQ5eNsLLMY1rDWccantMjVlcUdDZuPzI4g+/MtfE3IAs7JxtmwMvCMFRMuzWTtZkZSVyqpEGDeLnPGgMNTYUwaxQhlJLtcYnNTqdyZr8ZCcz3zP stephen@Stephenui-MacBook-Pro.local' >> /home/git/.ssh/authorized_keys


# Updating shell to bash

RUN sed -i s#/home/git:/bin/false#/home/git:/bin/bash# /etc/passwd


EXPOSE 22

CMD ["/usr/sbin/sshd", "-D"]

docker build -t git-ssh-img .

docker run --name git-ssh -d -p 1234:22 git-ssh-img


3. docker container bash로 접속

docker run -i -t --rm --net='host' ubuntu:14.04 bash


3. docker container 접속

docker exec -it <containerIdOrName> bash


4. docker 모든 컨테이너 보기

# docker ps -a


5. 모든 컨테이너 삭제

docker ps -a | awk '{print $1}' | grep -v CONTAINER | xargs sudo docker rm


6. docker 모든 <none> 이미지 삭제

docker images | grep "<none>" | awk '{print $3}' | xargs sudo docker rmi


7. 이미지 조회 및 실행

$ sudo docker search ubuntu

sudo docker run --name myssh -d -p 4444:22 rastasheep/ubuntu-sshd


8. stack 사용자 docker 그룹 권한 추가

$ sudo usermod -aG docker stack

$ sudo service docker restart

$ 재로그인


9. docker 이미지 가져오기

$ docker pull ubuntu:lates


10. docker bash쉘로 실행 및 빠져나오기기

docker run -i -t --name hello ubuntu /bin/bash

root@bb97e5f57596:/#


Ctrl + p, Ctrl + q        => 멈추지 않고 빠져나오기


$ docker attach hello            => 다시 접속하기 (enter를 한번 쳐야 함)


11. nginx 설치하기

# mkdir data


# vi Dockerfile

FROM ubuntu:14.04.3


RUN apt-get update

RUN apt-get install -y nginx

RUN echo "\ndaemon off;" >> /etc/nginx/nginx.conf

RUN chown -R www-data:www-data /var/lib/nginx


VOLUME ["/data", "/etc/nginx/site-enabled", "/var/log/nginx"]


WORKDIR /etc/nginx


CMD ["nginx"]


EXPOSE 80

EXPOSE 443


# docker build -t nginx:0.1 .

docker run --name hello-nginx -d -p 2080:80 -v /root/data:/data nginx:0.1



11. 파일 꺼내서 보기

# docker cp hello-nginx:/etc/nginx/nginx.conf ./


12. 컨테이러를 이미지로 생성

# docker commit -a "aaa <aaa@aaa.com>" -m "Initial commit" hello-nginx nginx:0.2


13. 이미지와 컨테이너 변경사항 보기

# docker diff 컨테이너ID

# docker history 이미지ID


14. 컨테이너 내부 보기

# docker inspect hello-nginx


15. docker 컨테이너의 pid 알아내기

docker inspect -f '{{.State.Pid}}' containerID


16. Docker 끼리 point to point 통신하기 (도커마다 네임스페이스를 만들어서 VETH 로 연결)

https://docs.docker.com/v1.7/articles/networking/#building-your-own-bridge


$ docker run -i -t --rm --net=none base /bin/bash

root@1f1f4c1f931a:/#


$ docker run -i -t --rm --net=none base /bin/bash

root@12e343489d2f:/#


# Learn the container process IDs

# and create their namespace entries


$ docker inspect -f '{{.State.Pid}}' 1f1f4c1f931a

2989

$ docker inspect -f '{{.State.Pid}}' 12e343489d2f

3004

$ sudo mkdir -p /var/run/netns

$ sudo ln -s /proc/2989/ns/net /var/run/netns/2989

$ sudo ln -s /proc/3004/ns/net /var/run/netns/3004


# Create the "peer" interfaces and hand them out


$ sudo ip link add A type veth peer name B


$ sudo ip link set A netns 2989

$ sudo ip netns exec 2989 ip addr add 10.1.1.1/32 dev A

$ sudo ip netns exec 2989 ip link set A up

$ sudo ip netns exec 2989 ip route add 10.1.1.2/32 dev A


$ sudo ip link set B netns 3004

$ sudo ip netns exec 3004 ip addr add 10.1.1.2/32 dev B

$ sudo ip netns exec 3004 ip link set B up

$ sudo ip netns exec 3004 ip route add 10.1.1.1/32 dev B



# ssh 다른 샘플

FROM ubuntu:14.04

RUN echo "deb http://archive.ubuntu.com/ubuntu/ trusty main universe" > /etc/apt/sources.list

RUN apt-get update


RUN apt-get install -y openssh-server

RUN mkdir /var/run/sshd

RUN echo 'root:screencast' | chpasswd


EXPOSE 22

CMD /usr/sbin/sshd -D



# NodeJS 샘플

git clone https://github.com/spkane/docker-node-hello.git

cd docker-node-hello


$ brew install tree

tree -a -I .git             # Directory 를 tree 구조로 봄


docker build --no-cache -t example/docker-node-hello:latest .

$ docker run -d -p 8081:8080 example/docker-node-hello:latest    # host 8081, docker 8080


$ echo $DOCKER_HOST


$ docker stop DOCKER_ID


# -e 옵션으로 env 넘기기

$ docker run -d -p 8081:8080 -e WHO="Seungkyu Ahn" example/docker-node-hello:latest


$ docker inspect DOCKER_ID









Posted by Kubernetes Korea co-leader seungkyua@gmail.com

1. Mac 에서 brew 를 통한 nodejs, npm 설치

http://brew.sh

$ brew install npm                   # npm 을 설치하면 이펜던시에 의해서 nodejs 도 설치됨


2. Ubuntu 에서 nodejs 설치

http:nodejs.org

$ sudo apt-get install g++


$ ./configure

$ make

$ sudo make install


3. npm 업그레이드

$ sudo npm install -g npm           # /usr/local/lib/node_modules/npm


4. bower 설치  (패키지 매니저 for Web) 및 활용

http://bower.io

$ sudo npm install -g bower


$ bower install jquery                                          # registered package

$ bower install desandro/masonry                         # GitHub shorthand
$ bower install git://github.com/user/package.git   # Git endpoint

$ bower install http://example.com/script.js           # URL


$ bower install angular         # 하위 디렉토리 bower_components/angular 만들고 다운로드


$ vi .bowerrc

{

  "directory": "WebContent/bower"

}

$ bower install angular         # 하위 디렉토리 WebContent/bower/angular 만들고 다운로드


5. eclipse 로 프로젝트 생성 (Project Type : Dynamic Web Project)





















Posted by Kubernetes Korea co-leader seungkyua@gmail.com