記録。

めも。

今更RBAC

タイトルの通り、いまさらkubernetesのRBACをいじってみた。

Using RBAC Authorization - Kubernetes

ロールベースのアクセス制御  |  Kubernetes Engine ドキュメント  |  Google Cloud

KubernetesのRBACについて - Qiita

Kubernetes道場 20日目 - Role / RoleBinding / ClusterRole / ClusterRoleBindingについて - Toku's Blog

「RBACとは」的な説明は省略しているので、そのようなことを知りたい方は上記のリンクなどに記載している。

いじる前の準備（ベースの環境作成）

namespace

apiVersion: v1
kind: Namespace
metadata:
  name: wakashiyo

service account

apiVersion: v1
kind: ServiceAccount
metadata:
  name: sample
  namespace: wakashiyo

検証用Pod

pod

apiVersion: v1
kind: Pod
metadata:
  name: sample-kubectl
  namespace: wakashiyo
spec:
  serviceAccountName: sample
  containers:
    - name: kubectl-container
      image: lachlanevenson/k8s-kubectl:latest
      command: ["sleep", "86400"]

1. 検証用のPodからkubectlコマンドを使用してPodを作成してみる

この段階では何もRoleは設定されていない

作成してみる

kubectl exec -it sample-kubectl -n wakashiyo -- kubectl run nginx --image=nginx:latest

Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:wakashiyo:sample" cannot create resource "pods" in API group "" in the namespace "wakashiyo"
command terminated with exit code 1

作ることができなかった

Podのリソースを作成することが禁じられていると言われてしまったので、Roleを付与してみる

role

kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: sample-role
  namespace: wakashiyo
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create"]

role binding

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: sample-role-binding
  namespace: wakashiyo
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: sample-role
subjects:
  - kind: ServiceAccount
    name: sample
    namespace: wakashiyo

apply する

kubectl apply -f role.yml
kubectl apply -f rolebinding.yml

再度作成し直してみる

kubectl exec -it sample-kubectl -n wakashiyo -- kubectl run nginx --image=nginx:latest

pod/nginx created


kubectl get pods -n wakashiyo -o wide

NAME             READY   STATUS    RESTARTS   AGE     IP           NODE                                       NOMINATED NODE   READINESS GATES
nginx            1/1     Running   0          10s     10.48.1.12   gke-wakashiyo-default-pool-29ce2b10-b5gc   <none>           <none>
sample-kubectl   1/1     Running   0          9m55s   10.48.0.10   gke-wakashiyo-default-pool-29ce2b10-gx26   <none>           <none>

作成できた

2. 検証用のPodから get pods したりしてみる

kubectl exec -it sample-kubectl -n wakashiyo -- kubectl get pods -n wakashiyo

Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:wakashiyo:sample" cannot list resource "pods" in API group "" in the namespace "wakashiyo"
command terminated with exit code 1



kubectl exec -it sample-kubectl -n wakashiyo -- kubectl get pods -n wakashiyo

Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:wakashiyo:sample" cannot list resource "pods" in API group "" in the namespace "wakashiyo"
command terminated with exit code 1



kubectl exec -it sample-kubectl -n wakashiyo -- kubectl get pods nginx -n wakashiyo

Error from server (Forbidden): pods "nginx" is forbidden: User "system:serviceaccount:wakashiyo:sample" cannot get resource "pods" in API group "" in the namespace "wakashiyo"
command terminated with exit code 1



kubectl exec -it sample-kubectl -n wakashiyo -- kubectl describe pods nginx -n wakashiyo

Error from server (Forbidden): pods "nginx" is forbidden: User "system:serviceaccount:wakashiyo:sample" cannot get resource "pods" in API group "" in the namespace "wakashiyo"
command terminated with exit code 1

できない。

特定のPodを指定してdescribeしたり、getしたりするためにはgetの操作権限、

pod一覧を出したりするためにはlistの操作権限が必要そう。

roleを変更してみる

role

kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: sample-role
  namespace: wakashiyo
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create", "get", "list"] # get, listを追加

再度getしたりしてみる

% kubectl exec -it sample-kubectl -n wakashiyo -- kubectl get pods -n wakashiyo

NAME             READY   STATUS    RESTARTS   AGE
nginx            1/1     Running   0          8m5s
sample-kubectl   1/1     Running   0          17m



% kubectl exec -it sample-kubectl -n wakashiyo -- kubectl get pods nginx -n wakashiyo

NAME    READY   STATUS    RESTARTS   AGE
nginx   1/1     Running   0          8m13s



% kubectl exec -it sample-kubectl -n wakashiyo -- kubectl describe pods nginx -n wakashiyo
Name:         nginx
Namespace:    wakashiyo
Priority:     0
... 以下省略

getできた。

ちなみに今まで設定していたのはClusterRoleではなく、Roleだったので、 --all-namespaces でのget操作はできない。

% kubectl exec -it sample-kubectl -n wakashiyo -- kubectl get pods --all-namespaces

Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:wakashiyo:sample" cannot list resource "pods" in API group "" at the cluster scope
command terminated with exit code 1

cluster roleを追加する

cluster role

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: sample-cluster-role
  namespace: wakashiyo
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list"]

cluster role binging

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: sample-cluster-role-binding
  namespace: wakashiyo
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: sample-cluster-role
subjects:
  - kind: ServiceAccount
    name: sample
    namespace: wakashiyo

applyして再度 --all-namespace でgetしてみる

% kubectl exec -it sample-kubectl -n wakashiyo -- kubectl get pods --all-namespaces

NAMESPACE     NAME                                                        READY   STATUS    RESTARTS   AGE
kube-system   event-exporter-v0.2.5-599d65f456-jn2t4                      2/2     Running   0          4h
kube-system   fluentd-gcp-scaler-bfd6cf8dd-cj6m5                          1/1     Running   0          4h
kube-system   fluentd-gcp-v3.1.1-jml9r                                    2/2     Running   0          3h59m
kube-system   fluentd-gcp-v3.1.1-mvhhf                                    2/2     Running   0          3h59m
kube-system   heapster-gke-ddccfc6d-xtgkv                                 3/3     Running   0          3h59m
kube-system   kube-dns-5995c95f64-dclwz                                   4/4     Running   0          4h
kube-system   kube-dns-5995c95f64-qh26s                                   4/4     Running   0          4h
kube-system   kube-dns-autoscaler-8687c64fc-xsh7x                         1/1     Running   0          4h
kube-system   kube-proxy-gke-wakashiyo-default-pool-29ce2b10-b5gc         1/1     Running   0          4h
kube-system   kube-proxy-gke-wakashiyo-default-pool-29ce2b10-gx26         1/1     Running   0          4h
kube-system   l7-default-backend-8f479dd9-r2xgs                           1/1     Running   0          4h
kube-system   metrics-server-v0.3.1-5c6fbf777-26kst                       2/2     Running   0          3h59m
kube-system   prometheus-to-sd-4bkqh                                      2/2     Running   0          4h
kube-system   prometheus-to-sd-lhb4t                                      2/2     Running   0          4h
kube-system   stackdriver-metadata-agent-cluster-level-6f6775c594-qmcx7   2/2     Running   0          3h59m
wakashiyo     nginx                                                       1/1     Running   0          14m
wakashiyo     sample-kubectl                                              1/1     Running   0          24m

できた。

3. 検証用のPodから作成したnginxのPodを削除してみる

% kubectl exec -it sample-kubectl -n wakashiyo -- kubectl delete pods nginx -n wakashiyo

Error from server (Forbidden): pods "nginx" is forbidden: User "system:serviceaccount:wakashiyo:sample" cannot delete resource "pods" in API group "" in the namespace "wakashiyo"
command terminated with exit code 1

当然deleteの権限がないのでできない。

roleを変更する

role

kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: sample-role
  namespace: wakashiyo
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create", "get", "list", "delete"] # deleteを追加

applyして再度delete podを実行してみる

% kubectl exec -it sample-kubectl -n wakashiyo -- kubectl delete pods nginx -n wakashiyo

pod "nginx" deleted




% kubectl get pods -n wakashiyo

NAME             READY   STATUS    RESTARTS   AGE
sample-kubectl   1/1     Running   0          27m

削除できた

4. podをスケールさせてみる

% kubectl get pods -n wakashiyo

NAME                    READY   STATUS    RESTARTS   AGE
nginx-cd9dcdd6c-jwnvn   1/1     Running   0          78s
sample-kubectl          1/1     Running   0          55m




% kubectl exec -it sample-kubectl -n wakashiyo -- kubectl scale deploy nginx --replicas 2

Error from server (Forbidden): deployments.extensions "nginx" is forbidden: User "system:serviceaccount:wakashiyo:sample" cannot get resource "deployments" in API group "extensions" in the namespace "wakashiyo"
command terminated with exit code 1

できない。

roleに権限を追加する

role

kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: sample-role
  namespace: wakashiyo
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create", "get", "list", "delete"]
  - apiGroups: ["apps", "extensions"]
    resources: ["deployments"]
    verbs: ["get"] # getを追加

再度スケールさせてみる

% kubectl exec -it sample-kubectl -n wakashiyo -- kubectl scale rs nginx --replicas 2

Error from server (Forbidden): deployments.extensions "nginx" is forbidden: User "system:serviceaccount:wakashiyo:sample" cannot patch resource "deployments/scale" in API group "extensions" in the namespace "wakashiyo"
command terminated with exit code 1

まだできない

再度roleを変更する

role

kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: sample-role
  namespace: wakashiyo
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create", "get", "list", "delete"]
  - apiGroups: ["apps", "extensions"]
    resources: ["deployments"]
    verbs: ["get"]
  # 以下を追加
  - apiGroups: ["apps", "extensions"]
    resources: ["deployments/scale"]
    verbs: ["patch"]

再度スケールさせてみる

% kubectl exec -it sample-kubectl -n wakashiyo -- kubectl scale deploy nginx --replicas 2

deployment.extensions/nginx scaled




% kubectl get pods -n wakashiyo
NAME                    READY   STATUS    RESTARTS   AGE
nginx-cd9dcdd6c-jwnvn   1/1     Running   0          4m27s
nginx-cd9dcdd6c-zs7f9   1/1     Running   0          7s
sample-kubectl          1/1     Running   0          58m

やっとできた。

スケールさせるには deploymentsではなく、　deployments/scale というリソースに対する権限が必要だった。

その他

RoleBindingはRoleを1つしか設定することができない。

role binding

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: sample-role-binding
  namespace: wakashiyo
roleRef:
  apiGroup: rbac.authorization.k8s.io
  - kind: Role
    name: sample-role
  - kind: Role # 2つ目のrole
    name: sample-role2
subjects:
  - kind: ServiceAccount
    name: sample
    namespace: wakashiyo

% kubectl apply -f rolebinding.yml
error: error parsing rolebinding.yml: error converting YAML to JSON: yaml: line 7: did not find expected key

※ エディタで書いている段階でエラー出る

RoleBindingで紐付けるservice accountは複数設定できる

role binding

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: sample-role-binding
  namespace: wakashiyo
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: sample-role
subjects: # 複数設定している
  - kind: ServiceAccount
    name: sample
    namespace: wakashiyo
  - kind: ServiceAccount
    name: sample2
    namespace: wakashiyo

1つのRoleで同じリソースに対してのルールを複数設定することができる

role

kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: sample-role
  namespace: wakashiyo
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create", "delete"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list"]

上記で設定した操作ができる。

% kubectl exec -it sample-kubectl -n wakashiyo -- kubectl run nginx --image=nginx
pod/nginx created



% kubectl exec -it sample-kubectl -n wakashiyo -- kubectl get pods nginx -n wakashiyo
NAME    READY   STATUS    RESTARTS   AGE
nginx   1/1     Running   0          19s



% kubectl exec -it sample-kubectl -n wakashiyo -- kubectl delete pods nginx -n wakashiyo
pod "nginx" deleted

cluster roleに限ってはaggregationできる。

cluster role 1

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: sample-cluster-role
  labels:
    app: sample-role
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list"]

cluster role 2

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: sample-cluster-role2
  labels:
    app: sample-role
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["delete"]

cluster role 3

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: sample-cluster-role3
  labels:
    app: sample-role
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create"]

aggregated cluster role

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: sample-aggregated-cluster-role
aggregationRule:
  clusterRoleSelectors:
    - matchLabels:
        app: sample-role

aggregateされている

% kubectl get clusterrole sample-aggregated-cluster-role -o yaml
aggregationRule:
  clusterRoleSelectors:
  - matchLabels:
      app: sample-role
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"aggregationRule":{"clusterRoleSelectors":[{"matchLabels":{"app":"sample-role"}}]},"apiVersion":"rbac.authorization.k8s.io/v1beta1","kind":"ClusterRole","metadata":{"annotations":{},"name":"sample-aggregated-cluster-role"}}
  creationTimestamp: "2020-05-05T12:24:26Z"
  name: sample-aggregated-cluster-role
  resourceVersion: "76217"
  selfLink: /apis/rbac.authorization.k8s.io/v1/clusterroles/sample-aggregated-cluster-role
  uid: 5c5491eb-8ecb-11ea-a4d9-42010a9200fb
rules:
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - get
  - list
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - delete
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - create

実際にcreate, get, deleteできる

% kubectl exec -it sample-kubectl -n wakashiyo -- kubectl run nginx --image=nginx
pod/nginx created



% kubectl exec -it sample-kubectl -n wakashiyo -- kubectl get pods -n wakashiyo
NAME             READY   STATUS    RESTARTS   AGE
nginx            1/1     Running   0          17s
sample-kubectl   1/1     Running   0          102m




% kubectl exec -it sample-kubectl -n wakashiyo -- kubectl get pods nginx -n wakashiyo
NAME    READY   STATUS    RESTARTS   AGE
nginx   1/1     Running   0          26s




% kubectl exec -it sample-kubectl -n wakashiyo -- kubectl delete pods nginx -n wakashiyo
pod "nginx" deleted

おしまい。