[ devstack 사전 조건 ]
devstack 설치 시 충분한 cpu, memory, disk 가 있어야 함.


[ localrc ]

VOLUME_BACKING_FILE_SIZE=70000M


[ rabbitmq memory leak 해결 ]
$ sudo vi /etc/rabbitmq/rabbitmq-env.conf
#celery_ignore_result = true

$ sudo service rabbitmq-server restart


[ cpu, memory overcommit ]
$ vi /etc/nova/nova.conf

scheduler_default_filters = RetryFilter,AvailabilityZoneFilter,CoreFilter,RamFilter,ComputeFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter
cpu_allocation_ratio = 16.0
ram_allocation_ratio = 100.0
disk_allocation_ratio = 100.0



[ vm 에 할당되는 DNS 변경 ]
$ neutron subnet-list
$ neutron subnet-update <subnet> --dns_nameservers list=true 8.8.8.8 8.8.4.4



[ Ubuntu Server 14.04 Image Upload ]
이름 : Ubuntu Server 14.04 64-bit
경로 : http://uec-images.ubuntu.com/releases/14.04.2/14.04.2/ubuntu-14.04-server-cloudimg-amd64-disk1.img
포맷 : QCOW2 - QEMU Emulator
최소 디스크 : 5
최소 RAM : 1024

아래 사이트 이미지 참고
https://help.ubuntu.com/community/UEC/Images
http://uec-images.ubuntu.com/releases/



[ OpenStack API 서버 접근 가능한지 테스트 ]

[ ruby 설치 ]
$ sudo apt-get install build-essential ruby ruby-dev libxml2-dev libsqlite3-dev libxslt1-dev libpq-dev libmysqlclient-dev
$ sudo apt-get install liblzma-dev zlib1g-dev
$ ruby -v
$ nokogiri --version

$ sudo gem install fog
$ vi .fog

:openstack:
  :openstack_auth_url:  http://192.168.230.141:5000/v2.0/tokens
  :openstack_api_key:   패스워드
  :openstack_username:  admin
  :openstack_tenant:    demo
  :openstack_region:    RegionOne # Optional

$ fog openstack
>>Compute[:openstack].servers



[ OpenStack Metadata 서버 접속 확인 ]
curl http://169.254.169.254

[ fog 로 user_data 넣기 ]
$ fog openstack
>> s = Compute[:openstack].servers.create(name: 'test', flavor_ref: , image_ref: , personality: [{'path' => 'user_data.json', 'contents' => 'test' }])



[ OpenStack API call 제한 여부 확인 ]
$ fog openstack
>> 100.times { p Compute[:openstack].servers }


[ 큰 Volume 생성 테스트 ] 
1. 30G Volume 생성
2. instance 에 Volume attach

[ Volume Attach가 안되면 tgtd 이 떠 있는지 확인 ]
sudo netstat -tulpn | grep 3260
sudo service tgt start

3. 추가 볼륨 포맷
$ sudo fdisk -l
$ sudo fdisk /dev/vdb

Command (m for help): n
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-62914559, default 2048): ENTER
Last sector, +sectors or +size{K,M,G} (2048-62914559, default 62914559): ENTER
Command (m for help): t
Partition number (1-4, default 1): 1
Hex code (type L to list codes): 83
Command (m for help): w

sudo mkfs.ext3 /dev/vdb1
$ sudo mkdir /disk
$ sudo mount -t ext3 /dev/vdb1 /disk
$ cd /disk
$ sudo touch pla





MicroBOSH 설치를 위한 OpenStack 설정 ] 
$ mkdir ~/my-micro-deployment
$ cd my-micro-deployment

[ Nova Client 준비 ]
$ sudo apt-get install python-novaclient
$ unset OS_SERVICE_TOKEN
$ unset OS_SERVICE_ENDPOINT
$ vi adminrc
export OS_USERNAME=admin
export OS_PASSWORD=imsi00
export OS_TENANT_NAME=demo
export OS_AUTH_URL=http://192.168.230.141:35357/v2.0

1. keypair 생성 : microbosh
nova keypair-add microbosh >> microbosh.pem
$ chmod 600 microbosh.pem

2. Security Group 생성 : bosh
name : bosh
description : BOSH Security Group

3. Security Rule 입력
Direction Ether Type IP Protocol Port Range Remote
Ingress  IPv4          TCP                1-65535     bosh
Ingress  IPv4          TCP                 25777     0.0.0.0/0 (CIDR)
Ingress  IPv4          TCP         25555     0.0.0.0/0 (CIDR)
Ingress  IPv4          TCP         25250     0.0.0.0/0 (CIDR)
Ingress  IPv4          TCP                 6868     0.0.0.0/0 (CIDR)
Ingress  IPv4          TCP              4222     0.0.0.0/0 (CIDR)
Ingress  IPv4          UDP      68                0.0.0.0/0 (CIDR)
Ingress  IPv4          TCP               53         0.0.0.0/0 (CIDR)
Ingress  IPv4          UDP         53         0.0.0.0/0 (CIDR)
Egress  IPv4          Any             -             0.0.0.0/0 (CIDR)
Egress  IPv6          Any         -             ::/0 (CIDR)


4. Allocate Floating IP



MicroBOSH 설치 ] 
1. yml 작성

$ vi manifest.yml

name: microbosh

network:
  type: manual
  vip: 192.168.75.206       # Replace with a floating IP address
  ip: 10.0.0.15    # subnet IP address allocation pool of OpenStack internal network
  cloud_properties:
    net_id: a34928c6-9715-4a91-911e-a6822afd600b # internal network UUID

resources:
  persistent_disk: 20000
  cloud_properties:
    instance_type: m1.medium

cloud:
  plugin: openstack
  properties:
    openstack:
      auth_url: http://192.168.230.141:35357/v2.0   # Identity API endpoint
      tenant: demo          # Replace with OpenStack tenant name
      username: admin    # Replace with OpenStack username
      api_key: 패스워드      # Replace with your OpenStack password
      default_key_name: microbosh   # OpenStack Keypair name
      private_key: microbosh.pem     # Path to OpenStack Keypair private key
      default_security_groups: [bosh]

apply_spec:
  properties:
    director: {max_threads: 3}
    hm: {resurrector_enabled: true}
    ntp: [time.bora.net, 0.north-america.pool.ntp.org, 1.north-america.pool.ntp.org]


2. Bosh cli 설치
$ sudo gem install bosh_cli --no-ri --no-rdoc
$ sudo gem install bosh_cli_plugin_micro --no-ri --no-rdoc


3. Download stemcell
https://bosh.io/stemcells

[ Ubuntu Server 14.04 stemcell 다운로드 ]
https://bosh.io/d/stemcells/bosh-openstack-kvm-ubuntu-trusty-go_agent?v=2986

$ curl -k -L -J -O https://bosh.io/d/stemcells/bosh-openstack-kvm-ubuntu-trusty-go_agent?v=2986

혹은,

$ wget --no-check-certificate --content-disposition https://bosh.io/d/stemcells/bosh-openstack-kvm-ubuntu-trusty-go_agent?v=2986

4. MicroBosh Deploy
$ bosh micro deployment manifest.yml
bosh micro deploy bosh-stemcell-2986-openstack-kvm-ubuntu-trusty-go_agent.tgz


5. MicroBosh Undeploy
bosh micro delete


6. MicroBosh Redeploy
bosh micro deploy --update bosh-stemcell-2986-openstack-kvm-ubuntu-trusty-go_agent.tgz





[ MicroBosh VM에 Cloud Foundry 설치 (Waden기반) ]
1. vm 접속
    - id : vcap / c1oudc0w
    - sudo su -


2. Ruby 설치 & Bosh cli 설치
$ apt-get update
$ apt-get install build-essential ruby ruby-dev libxml2-dev libsqlite3-dev libxslt1-dev libpq-dev libmysqlclient-dev
$ apt-get install liblzma-dev zlib1g-dev

gem install bosh_cli --no-ri --no-rdoc -r
gem install bosh_cli_plugin_micro --no-ri --no-rdoc -r


* gem upgrade 방법
wget http://production.cf.rubygems.org/rubygems/rubygems-2.4.8.tgz
tar xvfz rubygems-2.4.8.tgz
$ cd rubygems-2.4.8
ruby setup.rb

* gem 리모트 소스 추가
gem sources --add http://rubygems.org/


3. go 설치
$ mkdir -p CloudFoundry
$ cd CloudFoundry
wget --no-check-certificate https://storage.googleapis.com/golang/go1.4.2.linux-amd64.tar.gz
$ tar -C /usr/local -xzf go1.4.2.linux-amd64.tar.gz
$ mkdir -p /usr/local/gopath

$ vi ~/.profile
export GOPATH=/usr/local/gopath
export PATH=$PATH:/usr/local/go/bin:$GOPATH/bin

$ . ~/.profile
$ apt-get install git
go get github.com/cloudfoundry-incubator/spiff


4. Cloud Foundry 소스 다운로드
git clone https://github.com/cloudfoundry/cf-release.git
cd cf-release
$ ./update


5. Cloud Foundry Manual 설치
bosh target 192.168.75.206
   admin / admin


* /tmp 디렉토리 용량 확장
$ 추가 디스크 증설
mkfs.ext3 /dev/vdc
$ mkdir -p /tmp2
mount -t ext3 /dev/vdc /tmp2
$ mount --bind /tmp2 /tmp
$ chown root.root /tmp
$ chmod 1777 /tmp

* mount 취소
$ umount /tmp


bosh upload release releases/cf-212.yml


cp spec/fixtures/openstack/cf-stub.yml .





[ Bosh lite 설치 (Waden기반) on Mac ]
1. install vagrant
http://www.vagrantup.com/downloads.html


2. bosh lite 다운로드
$ git clone https://github.com/cloudfoundry/bosh-lite
$ cd bosh-lite


3. install VirtualBox
https://www.virtualbox.org/wiki/Downloads


4. start vagrant
vagrant up --provider=virtualbox


5. Target the Bosh Director and login admin/admin
bosh target 192.168.50.4 lite
$ bosh login


6. HomeBrew 및 Spiff 설치
xcode-select --install
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
brew tap xoebus/homebrew-cloudfoundry
brew install spiff


7. cloud foundry 설치
git clone https://github.com/cloudfoundry/cf-release
$ cd cf-release
$ ./update


8. Single Command Deploy
$ cd ~/CloudFoundry/bosh-list
$ ./bin/provision_cf


9. route 추가
$ ./bin/add-route


10. VM Restart 후에 container restart
$ bosh cck
Choose "2" to create missing VM:
"yes"




[ simple go webapp Deploy ]

$ ssh vcap@192.168.50.4     # password : c1oudc0w

$ bosh vms

$ cf api --skip--ssl-validation https://api.192.168.0.34.xip.io           # ha-proxy ip


$ cf login

Email> admin

Password> admin


$ cf create-org test-org

$ cf target -o test-org

$ cf create-space development

$ cf target -o test-org -s development


$ sudo apt-get update

$ sudo apt-get install git

$ sudo apt-get install golang

cf update-buildpack go_buildpack

git clone https://github.com/cloudfoundry-community/simple-go-web-app.git

$ cd simple-go-web-app

### 다른 buildpack https://github.com/cloudfoundry/go-buildpack.git

$ cf push simple-go-web -b https://github.com/kr/heroku-buildpack-go.git

$ cf apps             # app list

$ cf logs simple-go-web --recent     # app deploy log


# cf login

$ cf login -a http://api.10.244.0.34.xip.io -u admin -o test-org -s development















Posted by Kubernetes Korea co-leader seungkyua@gmail.com