kubernetes 에서 ceph rbd storageclass 활용법

Kubernetes 2017. 10. 20. 17:43
반응형


[ dynamic volume 사용법 ]
1. pvc 에 storageclass 를 지정하여 pvc 만 생성하면 pv 가 다이너믹하게 생성되고 pvc 도 생성된다.
2. 이 때 rbd 이미지도 자동으로 생성된다.

$ vi jenkins-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins
  namespace: ci-infra
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 100Gi
  storageClassName: ceph




[ static volume 사용법 ]
1. rbd 이미지를 수동으로 생성해야 한다.
2. pv 에 storageclass 와 rbd 값을 모두 넣어야 한다.
    pv 에 pvc 에서 pv 를 selector 로 찾을 수 있게 label 값을 넣어야 한다.
    (keyring 값은 안넣어도 됨, storageclass의 secret 이용)
3. pvc 에 storageclass 와 selector 나 volumeName 둘 중에 하나를 사용하여 pv 와 연결한다.
    (storageclass 는 값은 없어도 됨.  없으면 default 인 storageclass 값을 활용함)


$ vi jenkins-pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: jenkins
  labels:
    app: jenkins
spec:
  capacity:
    storage: 100Gi
  accessModes:
  - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  storageClassName: ceph
  rbd:
    image: jenkins
    monitors:
    - 192.168.30.23:6789
    - 192.168.30.24:6789
    - 192.168.30.25:6789
    pool: kubes
    secretRef:
      name: ceph-secret-user
    user: kube


$ vi jenkins-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins
  namespace: ci-infra
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 100Gi
  storageClassName: ceph
  selector:
    matchLabels:
      app: jenkins
# volumeName: jenkins


$ vi jenkins-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: jenkins
  namespace: ci-infra
  labels:
    app: jenkins
spec:
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: cicd-services
                operator: In
                values:
                - enabled
      securityContext:
        runAsUser: 1000
        fsGroup: 1000
      containers:
      - name: master
        env:
        - name: JENKINS_OPTS
          value: "--httpsPort=0 --http2Port=0"
        - name: JAVA_OPTS
          value: "-Xms8G -Xmx8G -XX:+UseG1GC -XX:+ExplicitGCInvokesConcurrent -XX:+ParallelRefProcEnabled -XX:+UseStringDeduplication -XX:+UnlockExperimentalVMOptions -XX:G1NewSizePercent=20 -XX:+UnlockDiagnosticVMOptions -XX:G1SummarizeRSetStatsPeriod=1 -Dorg.apache.commons.jelly.tags.fmt.timeZone=Asia/Seoul"
        image: jenkins/jenkins:latest
        imagePullPolicy: Always
        ports:
        - containerPort: 8080
          name: http
          protocol: TCP
        - containerPort: 50000
          name: jnlp
          protocol: TCP
        readinessProbe:
          httpGet:
            path: /login
            port: 8080
          periodSeconds: 10
          timeoutSeconds: 5
          successThreshold: 2
          failureThreshold: 5
        volumeMounts:
        - mountPath: /var/jenkins_home
          name: jenkins
#        resources:
#          limits:
#            cpu: 4000m
#            memory: 8000Mi
#          requests:
#            cpu: 1000m
#            memory: 8000Mi
      volumes:
      - name: jenkins
        persistentVolumeClaim:
          claimName: jenkins




## 생성
$ rbd create kubes/jenkins -s 100G
$ kubectl create -f jenkins-pv.yaml
$ kubectl create -f jenkins-pvc.yaml

$ kubectl create -f jenkins-deployment.yaml 


반응형
저작자표시 변경금지
Posted by seungkyua@gmail.com
,

OpenVPN 설치 및 vpn을 통한 사설 DNS 설정

Linux/Ubuntu 2017. 10. 19. 10:04
반응형

## Install OpenVPN

# apt-get update
# apt-get install openvpn easy-rsa

## Set Up the CA Directory (using easy-rsa)
# make-cadir /etc/openvpn/ease-rsa
# cd /etc/openvpn/ease-rsa

## Configure the CA Variables
# vi vars
export KEY_COUNTRY="KR"
export KEY_PROVINCE="Seoul"
export KEY_CITY="Jongno"
export KEY_ORG="OpenStackKR"
export KEY_EMAIL="root@localhost"
export KEY_OU="OpenStack KR"
export KEY_NAME="server"

## Build the Certificate Authority
# source vars
# ./clean-all
# ./build-ca
Generating a 2048 bit RSA private key
.....................................................+++
...................+++
writing new private key to 'ca.key'
-----
Country Name (2 letter code) [KR]:
State or Province Name (full name) [Seoul]:
Locality Name (eg, city) [Jongno]:
Organization Name (eg, company) [OpenStackKR]:
Organizational Unit Name (eg, section) [OpenStackKR]:
Common Name (eg, your name or your server's hostname) [OpenStackKR CA]:
Name [server]:
Email Address [root@localhost]:


## Create the Server Certificate, Key, and Encryption Files
# ./build-key-server server

A challenge password []:               --> 그냥 엔터
Sign the certificate? [y/n]:y

1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated


# ./build-dh
# openvpn --genkey --secret keys/ta.key


## Generate a Client Certificate and Key Pair
# cd /etc/openvpn/ease-rsa
# source vars
# ./build-key seungkyua
Generating a 2048 bit RSA private key
.....+++
.................+++
writing new private key to 'seungkyua.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [KR]:
State or Province Name (full name) [Seoul]:
Locality Name (eg, city) [Jongno]:
Organization Name (eg, company) [OpenStackKR]:
Organizational Unit Name (eg, section) [OpenStackKR]:
Common Name (eg, your name or your server's hostname) [seungkyua]:
Name [server]:
Email Address [root@localhost]:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:password
An optional company name []: OpenStackKR
Using configuration from /etc/openvpn/easy-rsa/openssl-1.0.0.cnf
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
countryName           :PRINTABLE:'KR'
stateOrProvinceName   :PRINTABLE:'Seoul'
localityName          :PRINTABLE:'Jongno'
organizationName      :PRINTABLE:'OpenStackKR'
organizationalUnitName:PRINTABLE:'OpenStackKR'
commonName            :PRINTABLE:'seungkyua'
name                  :PRINTABLE:'server'
emailAddress          :IA5STRING:'root@localhost'
Certificate is to be certified until Aug 29 01:42:22 2027 GMT (3650 days)
Sign the certificate? [y/n]:y


1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated



## Configure the OpenVPN Service
# cd /etc/openvpn/ease-rsa/keys
# cp ca.crt ca.key server.crt server.key ta.key dh2048.pem /etc/openvpn
# gunzip -c /usr/share/doc/openvpn/examples/sample-config-files/server.conf.gz | sudo tee /etc/openvpn/server.conf

# vi /etc/openvpn/server.conf
32 port 1194
35 proto tcp
78 ca ca.crt
79 cert server.crt
80 key server.key

141 push "route 192.168.30.0 255.255.255.0"
142 push "route 192.168.230.0 255.255.255.0"
143 push "route 192.168.130.0 255.255.255.0"
144 push "route 192.168.49.0 255.255.255.0"
145 push "route 192.168.51.0 255.255.255.0"
146 push "route 192.168.54.0 255.255.255.0"

205 push "dhcp-option DNS 192.168.30.26"    # 사설 DNS 서버가 설치될 서버 IP
206 push "dhcp-option DNS 8.8.8.8"
208 push "dhcp-option DOMAIN cicd.seungkyua"   # 사설 DNS 도메인

250 tls-auth ta.key 0
256 cipher AES-128-CBC   # AES
259 auth SHA256
275 user nobody
276 group nogroup


## Adjust the Server Networking Configuration
# vi /etc/sysctl.conf
28 net.ipv4.ip_forward=1

# sysctl -p

# ip route | grep default
# iptables -t nat -A POSTROUTING -s 10.8.0.0/8 -o br-ex -j MASQUERADE



## Start and Enable the OpenVPN Service
# systemctl start openvpn@server
# systemctl status openvpn@server
# systemctl enable openvpn@server


## Create Client Configuration Infrastructure
# cd /etc/openvpn
# mkdir -p /etc/openvpn/client-configs/files
# chmod 700 /etc/openvpn/client-configs/files
# cp /usr/share/doc/openvpn/examples/sample-config-files/client.conf /etc/openvpn/client-configs/base.conf

# vi client-configs/base.conf
36 proto tcp
42 remote server_ip 1194
61 user nobody
62 group nogroup
88 #ca ca.crt
89 #cert client.crt
90 #key client.key
113 cipher AES-128-CBC
114 auth SHA256
115 key-direction 1


# vi client-configs/make_config.sh
#!/bin/bash

# First argument: Client identifier

KEY_DIR=/etc/openvpn/ease-rsa/keys
OUTPUT_DIR=/etc/openvpn/client-configs/files
BASE_CONFIG=/etc/openvpn/client-configs/base.conf

cat ${BASE_CONFIG} \
    <(echo -e '<ca>') \
    ${KEY_DIR}/ca.crt \
    <(echo -e '</ca>\n<cert>') \
    ${KEY_DIR}/${1}.crt \
    <(echo -e '</cert>\n<key>') \
    ${KEY_DIR}/${1}.key \
    <(echo -e '</key>\n<tls-auth>') \
    ${KEY_DIR}/ta.key \
    <(echo -e '</tls-auth>') \
    > ${OUTPUT_DIR}/${1}.ovpn


# chmod 700 client-configs/make_config.sh

# cd /etc/openvpn/client-configs

## seungkyua.ovpn 파일이 /etc/openvpn/client-configs/files 디렉토리 밑에 생성됨
# ./make_config.sh seungkyua



## 사용자 추가 시 클라이언트 파일 만드는 법

# cd /etc/openvpn/ease-rsa
# source vars
# ./build-key seungkyua

# cd /etc/openvpn/client-configs
# ./make_config.sh seungkyua


## server reboot 시 체크
# ip route | grep default

# iptables -t nat -A POSTROUTING -s 10.8.0.0/8 -o br-ex -j MASQUERADE



출처 : https://www.digitalocean.com/community/tutorials/how-to-set-up-an-openvpn-server-on-ubuntu-16-04






## Install Bind on the DNS Server
# sudo apt-get update
# apt-get install unbound


## config 설정
# cd /etc/unbound
# vi unbound.conf.d/root-auto-trust-anchor-file.conf
server:
    verbosity: 1
    statistics-interval: 0
    statistics-cumulative: no
    extended-statistics: yes
    num-threads: 2

        interface: 192.168.30.26
        interface: 127.0.0.1

    outgoing-range: 4096
    outgoing-port-permit: 40000-44096
    cache-max-ttl: 3600
    do-ip4: yes
    do-ip6: no

    access-control: 10.8.0.0/24 allow
    access-control: 192.168.30.0/24 allow
    access-control: 192.168.54.0/24 allow

    chroot: ""
    username: "unbound"
    directory: "/etc/unbound"
    log-time-ascii: yes
    pidfile: "/var/run/unbound/unbound.pid"
    hide-identity: yes
    hide-version: yes
    harden-short-bufsize: yes
    harden-large-queries: yes
    harden-glue: yes
    harden-dnssec-stripped: yes
    harden-below-nxdomain: yes
    harden-referral-path: yes
    use-caps-for-id: yes
    unwanted-reply-threshold: 10000000
    prefetch: yes
    prefetch-key: yes
    rrset-roundrobin: yes
    minimal-responses: yes
#    trusted-keys-file: /etc/unbound/keys.d/*.key
    auto-trust-anchor-file: "/var/lib/unbound/root.key"
    val-clean-additional: yes
    val-permissive-mode: no
    val-log-level: 1
    key-cache-size: 512m

    include: /etc/unbound/local.d/*.conf

# Remote control config section.
remote-control:
    control-enable: yes
    server-key-file: "/etc/unbound/unbound_server.key"
    server-cert-file: "/etc/unbound/unbound_server.pem"
    control-key-file: "/etc/unbound/unbound_control.key"
    control-cert-file: "/etc/unbound/unbound_control.pem"

# Stub and Forward zones
# include: /etc/unbound/conf.d/*.conf



## dns 설정
# mkdir -p local.d
# vi local.d/cicd.seungkyua.conf
local-zone: "cicd.stg.taco." static
local-data: "master01.cicd.seungkyua. IN A 192.168.30.13"
local-data: "node01.cicd.seungkyua. IN A 192.168.30.12"
local-data: "node02.cicd.seungkyua. IN A 192.168.30.17"
local-data: "node03.cicd.seungkyua. IN A 192.168.30.18"
local-data: "node04.cicd.seungkyua. IN A 192.168.30.21"

local-data: "centos-repo.cicd.seungkyua. IN A 192.168.30.12"
local-data: "dashboard.cicd.seungkyua. IN A 192.168.30.12"
local-data: "grafana.cicd.seungkyua. IN A 192.168.30.12"
local-data: "horizon.cicd.seungkyua. IN A 192.168.30.12"
local-data: "jenkins.cicd.seungkyua. IN A 192.168.30.12"
local-data: "keystone.cicd.seungkyua. IN A 192.168.30.12"
local-data: "kibana.cicd.seungkyua. IN A 192.168.30.12"
local-data: "minio.seungkyua. IN A 192.168.30.12"
local-data: "pypi-repo.cicd.seungkyua. IN A 192.168.30.12"
local-data: "registry.cicd.seungkyua. IN A 192.168.30.12"
local-data: "scope.cicd.seungkyua. IN A 192.168.30.12"
local-data: "prometheus.cicd.seungkyua. IN A 192.168.30.12"
local-data: "ubuntu-repo.cicd.seungkyua. IN A 192.168.30.12"



## start unbound
# systemctl restart unbound.service

# systemctl enable unbound.service 



출처 : https://calomel.org/unbound_dns.html



## reload unbound server with new configuration

# unbound-control reload






반응형
저작자표시 변경금지
Posted by seungkyua@gmail.com
,

티스토리툴바