반응형
Kubebuilder 의 아키텍처에 대해서 살펴보고 Kubebuilder 로 프로젝트를 생성하는 방법을 알아본다.
Kubebuilder Architeture
[출처: https://book.kubebuilder.io/architecture.html]
위의 다이어그램에서 Kubebuilder 는 controller-runtime 모듈을 사용하는 것을 알 수 있다. 또한 사용자의 비즈니스 로직은 Reconciler 에 위치 시킨다는 것을 알 수 있다.
Kubebuilder 로 프로젝트 생성
Kubebuilder 를 사용하기 위해서 사전 준비 작업이 필요하다.
사전 준비 작업
- go version v1.19.0+
- docker version 17.03+.
- kubectl version v1.11.3+.
- Access to a Kubernetes v1.11.3+ cluster.
Kubebuilder 설치
kubebuilder 는 간단히 다운 받아서 설치할 수 있다. ~/bin/ 디렉토리가 path 로 잡혀있기 때문에 다운 받은 바이너리 파일을 이 곳으로 이동시켰다.
$ cd ~/Documents/tmp
$ curl -L -o kubebuilder https://go.kubebuilder.io/dl/latest/$(go env GOOS)/$(go env GOARCH)
$ chmod +x kubebuilder
$ mv ~/Documents/tmp/kubebuilder ~/bin/kubebuilder
$ kubebuilder version
--- output ---
Version: main.version{KubeBuilderVersion:"3.10.0", KubernetesVendor:"1.26.1", GitCommit:"0fa57405d4a892efceec3c5a902f634277e30732", BuildDate:"2023-04-15T08:10:35Z", GoOs:"darwin", GoArch:"amd64"}kustomize 설치
$ cd ~/Documents/tmp
$ curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash
$ mv ~/Documents/tmp/kustomize ~/bin/kustomize
$ kustomize version
--- output ---
v5.0.3kind 설치 및 cluster 생성
$ cd ~/Documents/tmp
$ [ $(uname -m) = x86_64 ]&& curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.19.0/kind-darwin-amd64
$ chmod +x kind
$ mv ~/Documents/tmp/kind ~/bin/kind
$ kind version
--- output ---
kind v0.19.0 go1.20.4 darwin/amd64
$ kind create clusterkubectl 설치
$ cd ~/Documents/tmp
$ curl -LO "https://dl.k8s.io/release/v1.27.1/bin/darwin/amd64/kubectl"
$ chmod +x kubectl
$ mv kubectl ~/bin/kubectl
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.1", GitCommit:"4c9411232e10168d7b050c49a1b59f6df9d7ea4b", GitTreeState:"clean", BuildDate:"2023-04-14T13:21:19Z", GoVersion:"go1.20.3", Compiler:"gc", Platform:"darwin/amd64"}
Kustomize Version: v5.0.1
Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.1", GitCommit:"4c9411232e10168d7b050c49a1b59f6df9d7ea4b", GitTreeState:"clean", BuildDate:"2023-05-12T19:03:40Z", GoVersion:"go1.20.3", Compiler:"gc", Platform:"linux/amd64"}
프로젝트 생성
kubebuilder 명령어로 간단히 프로젝트와 API 를 생성할 수 있다. 즉, 필요한 코드들이 자동으로 생성된다.
먼저 프로젝트를 생성한다.
$ mkdir -p guestbook-kubebuilder
$ cd guestbook-kubebuilder
$ kubebuilder init --domain my.domain --repo my.domain/guestbook
--- output ---
Writing kustomize manifests for you to edit...
Writing scaffold for you to edit...
Get controller runtime:
$ go get sigs.k8s.io/controller-runtime@v0.14.4
go: downloading sigs.k8s.io/controller-runtime v0.14.4
go: downloading k8s.io/apimachinery v0.26.1
go: downloading github.com/prometheus/client_golang v1.14.0
go: downloading k8s.io/client-go v0.26.1
go: downloading k8s.io/utils v0.0.0-20221128185143-99ec85e7a448
go: downloading github.com/prometheus/client_model v0.3.0
go: downloading k8s.io/api v0.26.1
go: downloading k8s.io/component-base v0.26.1
go: downloading golang.org/x/time v0.3.0
go: downloading k8s.io/apiextensions-apiserver v0.26.1
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.2
go: downloading golang.org/x/net v0.3.1-0.20221206200815-1e63c2f08a10
go: downloading github.com/imdario/mergo v0.3.6
go: downloading k8s.io/kube-openapi v0.0.0-20221012153701-172d655c2280
go: downloading golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b
Update dependencies:
$ go mod tidy
go: downloading go.uber.org/goleak v1.2.0
Next: define a resource with:
$ kubebuilder create api
다음으로 api 를 생성한다.
$ kubebuilder create api --group webapp --version v1 --kind Guestbook
--- output ---
Create Resource [y/n]
y
Create Controller [y/n]
y
Writing kustomize manifests for you to edit...
Writing scaffold for you to edit...
api/v1/guestbook_types.go
api/v1/groupversion_info.go
internal/controller/suite_test.go
internal/controller/guestbook_controller.go
Update dependencies:
$ go mod tidy
Running make:
$ make generate
mkdir -p /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin
test -s /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen && /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen --version | grep -q v0.11.3 || \
GOBIN=/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.3
go: downloading sigs.k8s.io/controller-tools v0.11.3
go: downloading golang.org/x/tools v0.4.0
go: downloading k8s.io/utils v0.0.0-20221107191617-1a15be271d1d
go: downloading github.com/mattn/go-colorable v0.1.9
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
Next: implement your new API and generate the manifests (e.g. CRDs,CRs) with:
$ make manifestsCR 이나 CRD 를 수정하면 마지막의 make manifests 를 수행하여 다신 generation 해야 한다고 친절히 알려주고 있다.
CR 과 CRD 는 아래 guestbook_types.go 파일에 struct 로 생성되어 있다. 이곳을 원하는 대로 변경하면 된다.
테스트로 아래과 같이 변경하자.
// GuestbookSpec defines the desired state of Guestbook
type GuestbookSpec struct {
// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
// Important: Run "make" to regenerate code after modifying this file
// Quantity of instances
// +kubebuilder:validation:Minimum=1
// +kubebuilder:validation:Maximum=10
Size int32 `json:"size"`
// Name of the ConfigMap for GuestbookSpec's configuration
// +kubebuilder:validation:MaxLength=15
// +kubebuilder:validation:MinLength=1
ConfigMapName string `json:"configMapName"`
// +kubebuilder:validation:Enum=Phone;Address;Name
Type string `json:"alias,omitempty"`
}
// GuestbookStatus defines the observed state of Guestbook
type GuestbookStatus struct {
// INSERT ADDITIONAL STATUS FIELD - define observed state of cluster
// Important: Run "make" to regenerate code after modifying this file
// PodName of the active Guestbook node.
Active string `json:"active"`
// PodNames of the standby Guestbook nodes.
Standby []string `json:"standby"`
}
//+kubebuilder:object:root=true
//+kubebuilder:subresource:status
// Guestbook is the Schema for the guestbooks API
type Guestbook struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
Spec GuestbookSpec `json:"spec,omitempty"`
Status GuestbookStatus `json:"status,omitempty"`
}
//+kubebuilder:object:root=true
// GuestbookList contains a list of Guestbook
type GuestbookList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`
Items []Guestbook `json:"items"`
}Guestbook struct 에 있는 metav1.TypeMeta 와 metav1.ObjectMeta 를 설명하면, 전자는 우리가 흔히 보는 yaml 파일에서 apiVersion 과 Kind 이고 후자는 metadata 의 name, namespace 등을 나타낸다. 다음에 우리가 정의한 Spec 과 Status 가 있음을 알 수 있다.
테스트 방법 1 - Cluster 밖에서 테스트 하기
CRD 를 cluster 에 설치한다.
$ make install
--- output ---
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
test -s /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/kustomize || { curl -Ss "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" --output install_kustomize.sh && bash install_kustomize.sh 5.0.0 /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin; rm install_kustomize.sh; }
v5.0.0
kustomize installed to /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/kustomize
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/kustomize build config/crd | kubectl apply -f -
customresourcedefinition.apiextensions.k8s.io/guestbooks.webapp.my.domain created
controller 를 실행시킨다. (터미널에서 포그라운드로 실행한다)
$ make run
--- output ---
test -s /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen && /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen --version | grep -q v0.11.3 || \
GOBIN=/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.3
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
go run ./cmd/main.go
2023-05-24T17:18:18+09:00 INFO controller-runtime.metrics Metrics server is starting to listen {"addr": ":8080"}
2023-05-24T17:18:18+09:00 INFO setup starting manager
2023-05-24T17:18:18+09:00 INFO Starting server {"kind": "health probe", "addr": "[::]:8081"}
2023-05-24T17:18:18+09:00 INFO Starting server {"path": "/metrics", "kind": "metrics", "addr": "[::]:8080"}
2023-05-24T17:18:18+09:00 INFO Starting EventSource {"controller": "guestbook", "controllerGroup": "webapp.my.domain", "controllerKind": "Guestbook", "source": "kind source: *v1.Guestbook"}
2023-05-24T17:18:18+09:00 INFO Starting Controller {"controller": "guestbook", "controllerGroup": "webapp.my.domain", "controllerKind": "Guestbook"}
2023-05-24T17:18:18+09:00 INFO Starting workers {"controller": "guestbook", "controllerGroup": "webapp.my.domain", "controllerKind": "Guestbook", "worker count": 1}참고로 앞서 api 를 생성할 때 Create Resource [y/n] y 로 했다면 CR 이 config/samples 디렉토리 아래에 생성되어 있다.
여기에 Spec 부분만 추가한다.
$ tree config/samples
config/samples
├── kustomization.yaml
└── webapp_v1_guestbook.yaml
$ vi config/samples/webapp_v1_guestbook.yaml
--- output ---
apiVersion: webapp.my.domain/v1
kind: Guestbook
metadata:
labels:
app.kubernetes.io/name: guestbook
app.kubernetes.io/instance: guestbook-sample
app.kubernetes.io/part-of: guestbook-kubebuilder
app.kubernetes.io/managed-by: kustomize
app.kubernetes.io/created-by: guestbook-kubebuilder
name: guestbook-sample
spec:
# TODO(user): Add fields here
size: 1
configMapName: "myconfig"
alias: "Address"
터미널을 새로 열어서 이를 설치한다.
$ kubectl apply -k config/samples/
--- output ---
guestbook.webapp.my.domain/guestbook-sample created
$ kubectl get guestbook
--- output ---
NAME AGE
guestbook-sample 29s
테스트 방법 2 - Cluster 안에서 돌리기
controller 를 cluster 안에서 돌리기 위해서는 먼저 이미지를 만들어야 한다.
$ docker login -u seungkyua
--- output ---
Password:
Login Succeeded
$ make docker-build docker-push IMG=docker.io/seungkyua/guestbook-kubebuilder:1.0
다음은 image 를 가지고 deploy 한다.
$ make deploy IMG=docker.io/seungkyua/guestbook-kubebuilder:1.0
--- output ---
test -s /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen && /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen --version | grep -q v0.11.3 || \
GOBIN=/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.3
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
test -s /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/kustomize || { curl -Ss "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" --output install_kustomize.sh && bash install_kustomize.sh 5.0.0 /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin; rm install_kustomize.sh; }
cd config/manager && /Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/kustomize edit set image controller=docker.io/seungkyua/guestbook-kubebuilder:1.0
/Users/ahnsk/Documents/github.com/seungkyua/guestbook-kubebuilder/bin/kustomize build config/default | kubectl apply -f -
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
namespace/guestbook-kubebuilder-system created
customresourcedefinition.apiextensions.k8s.io/guestbooks.webapp.my.domain configured
serviceaccount/guestbook-kubebuilder-controller-manager created
role.rbac.authorization.k8s.io/guestbook-kubebuilder-leader-election-role created
clusterrole.rbac.authorization.k8s.io/guestbook-kubebuilder-manager-role created
clusterrole.rbac.authorization.k8s.io/guestbook-kubebuilder-metrics-reader created
clusterrole.rbac.authorization.k8s.io/guestbook-kubebuilder-proxy-role created
rolebinding.rbac.authorization.k8s.io/guestbook-kubebuilder-leader-election-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/guestbook-kubebuilder-manager-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/guestbook-kubebuilder-proxy-rolebinding created
service/guestbook-kubebuilder-controller-manager-metrics-service created
deployment.apps/guestbook-kubebuilder-controller-manager created
확인하면 다음과 같이 pod 가 설치된 것을 알 수 있다.
$ kubectl get pods -n guestbook-kubebuilder-system
--- output ---
NAME READY STATUS RESTARTS AGE
guestbook-kubebuilder-controller-manager-5f74f9d765-r68gn 2/2 Running 0 2m55s
삭제하기
crd 삭제
$ make uninstallCluster 에 설치된 controller 삭제
$ make undeploy반응형
