Secret
시크릿이란
민감한 정보들을 저장하는 용도
- 쿠버네티스 내부에서 보안이 필요한 설정들을 구성할 때 사용
- configMap과 같은 key-value 저장소
- 패스워드, 암호화 키/인증서, 토큰 등 소량의 민감한 데이터 저장
- BASE64로 인코딩하여 데이터 저장
- 컨피그 맵은 단순 데이터나 민감하지 않은 설정 데이터를 저장하고 참조하는 용도
- 개별 오브젝트당 1MiB로 제한
시크릿 저장 데이터 종류
| 시크릿에 직접 저장 가능 |
generic |
key-value 형식의 임의의 데이터 |
| docker-registry |
도커 저장소 인증 정보 |
| tls |
TLS 키 및 인증서 |
| 시크릿에 직접 저장 불가능 |
service-account-token |
서비스 계정 토큰 |
시크릿 생성 방법
kubectl create secret <TYPE> <NAME> [OPTIONS]
kubectl create secret generic <NAME> [--from-file=[key=]source] [--from-literal=key1=value1]
kubectl create secret docker-registry <NAME> \
--docker-username=user --docker-password=password \
--docker-email=email [--docker-server=string]
kubectl create secrets tls <NAME> --cert=cert-file --key=key-file
실습 - 1
명령으로 Secret 생성
vagrant@kube-master1:~
$ mkdir secret
vagrant@kube-master1:~
$ cd secret/
- secret에 저장할 파일 생성
- id.txt 파일과 pwd.txt 파일을 secret 디렉토리에 생성
vagrant@kube-master1:~/secret
$ echo -n "admin" > id.txt
vagrant@kube-master1:~/secret
$ echo -n "P@ssw0rd" > pwd.txt
vagrant@kube-master1:~/secret
$ cat id.txt
adminvagrant@kube-master1:~/secret
$ cat pwd.txt
P@ssw0rdvagrant@kube-master1:~/secret
- id.txt 파일과 pwd.txt 파일을 이용하여 secret 생성
$ kubectl create secret generic my-secret --from-file=id.txt --from-file=pwd.txt
secret/my-secret created
vagrant@kube-master1:~/secret
$ kubectl get secrets
NAME TYPE DATA AGE
default-token-wl2vl kubernetes.io/service-account-token 3 9d
my-secret Opaque 2 5s
- 생성한 secret 정보 확인
- kubectl get
- kubectl describe
- 시크릿 생성 시 자동으로 필드 값을 base64로 인코딩 처리함 (.data 필드에서 확인 가능)
$ kubectl get secret my-secret -o yaml
apiVersion: v1
data:
id.txt: YWRtaW4=
pwd.txt: UEBzc3cwcmQ=
kind: Secret
metadata:
creationTimestamp: "2022-06-30T03:36:13Z"
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:data:
.: {}
f:id.txt: {}
f:pwd.txt: {}
f:type: {}
manager: kubectl
operation: Update
time: "2022-06-30T03:36:13Z"
name: my-secret
namespace: default
resourceVersion: "770149"
selfLink: /api/v1/namespaces/default/secrets/my-secret
uid: beb61678-e579-4d14-ba06-365f8f263159
type: Opaque
$ kubectl describe secret my-secret
Name: my-secret
Namespace: default
Labels: <none>
Annotations: <none>
Type: Opaque
Data
====
id.txt: 5 bytes
pwd.txt: 8 bytes
vagrant@kube-master1:~/secret
$ echo "YWRtaW4=" | base64 --decode
admin
vagrant@kube-master1:~/secret
$ echo "UEBzc3cwcmQ=" | base64 --decode
P@ssw0rd
$ kubectl get secrets
NAME TYPE DATA AGE
default-token-wl2vl kubernetes.io/service-account-token 3 9d
my-secret Opaque 2 7m15s
$ kubectl describe secrets default-token-wl2vl
Name: default-token-wl2vl
Namespace: default
Labels: <none>
Annotations: kubernetes.io/service-account.name: default
kubernetes.io/service-account.uid: c768f156-ad53-4038-bdb8-d7465bc4724d
Type: kubernetes.io/service-account-token
Data
====
ca.crt: 1025 bytes
namespace: 7 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IjZIWG9vU05yZF95VXo4bWVfR3Z2eV95THFRa0JFT2ZkRFZrODlqLWdLWXcifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImRlZmF1bHQtdG9rZW4td2wydmwiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGVmYXVsdCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImM3NjhmMTU2LWFkNTMtNDAzOC1iZGI4LWQ3NDY1YmM0NzI0ZCIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpkZWZhdWx0OmRlZmF1bHQifQ.rc6nzgKEUMID5BCoLlu_vhwPSPOSj0Dp30-0dVW1hiM6GKtrqhUVdXNHeFVFTcNzURNHtYZfuLEogsNAJJ9-iePOCdnNRmjnG4bdIQlhNpJJj7A8fF2jYPD_W5S7ir09eVHEZgcrNcRQLpuLnzsvalr_LrW_xNvBY--MB7W2jnV0iyiAslaOL8-KLSqLTh9yuJsg6CXEDLslHnLAvH_72f0ESM-CHKIxXbJ7_fLchvM8J0iSQAYvq5wc3yzaC1l2ZkpNQjphCSED7aoHvcpHbWWCM6MS6R04gTSYrctSEW8DBxHyVyhH_sNUYs9KXge0UQqo9OdAEHncptlcMOHZqQ
$ kubectl describe pod nginx-pod-compress
~
Volumes:
nginx-compress-config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: nginx-gzip-config
Optional: false
default-token-wl2vl:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-wl2vl
Optional: false
~
실습 - 2
Secret을 이용한 Nginx https 웹 서비스
vagrant@kube-master1:~/secret/nginx-tls
$ openssl req -x509 -nodes -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=myapp.exampl
e.com"
Can't load /home/vagrant/.rnd into RNG
140097390891456:error:2406F079:random number generator:RAND_load_file:Cannot open file:../crypto/rand/randfile.c:88:Filename=/home/vagrant/.rnd
Generating a RSA private key
.................+++++
.................................+++++
writing new private key to 'tls.key'
-----
vagrant@kube-master1:~/secret/nginx-tls
$ ls
tls.crt tls.key
- HTTPS 인증서를 저장하는 용도로 secret 생성
vagrant@kube-master1:~/secret/nginx-tls
$ kubectl create secret tls nginx-tls-secret --cert=tls.crt --key=tls.key
secret/nginx-tls-secret created
vagrant@kube-master1:~/secret/nginx-tls
$ kubectl get secrets
NAME TYPE DATA AGE
default-token-wl2vl kubernetes.io/service-account-token 3 9d
my-secret Opaque 2 151m
nginx-tls-secret kubernetes.io/tls 2 7s
- 생성한 secret 정보 확인 (yaml 또는 json 형식으로 확인)
vagrant@kube-master1:~/secret/nginx-tls
$ kubectl get secrets nginx-tls-secret -o yaml
$ kubectl get secrets nginx-tls-secret -o json
vagrant@kube-master1:~/secret/nginx-tls
$ kubectl describe secrets nginx-tls-secret
Name: nginx-tls-secret
Namespace: default
Labels: <none>
Annotations: <none>
Type: kubernetes.io/tls
Data
====
tls.crt: 1135 bytes
tls.key: 1704 bytes
vagrant@kube-master1:~/secret/nginx-tls
$ vi ../nginx-tls.conf
vagrant@kube-master1:~/secret/nginx-tls
$ cat ../nginx-tls.conf
server {
listen 80;
listen 443 ssl;
server_name myapp.example.com;
ssl_certificate /etc/nginx/ssl/tls.crt;
ssl_certificate_key /etc/nginx/ssl/tls.key;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
location / {
root /usr/share/nginx/html;
index index.html;
}
}
- nginx conf 구성 파일을 이용하여 configmap 생성
vagrant@kube-master1:~/secret/nginx-tls
$ kubectl create configmap nginx-tls-config --from-file=../nginx-tls.conf
configmap/nginx-tls-config created
vagrant@kube-master1:~/secret/nginx-tls
$ kubectl get cm
NAME DATA AGE
nginx-tls-config 1 4s
vagrant@kube-master1:~/secret/nginx-tls
$ kubectl describe configmap nginx-tls-config
Name: nginx-tls-config
Namespace: default
Labels: <none>
Annotations: <none>
Data
====
nginx-tls.conf:
----
server {
listen 80;
listen 443 ssl;
server_name myapp.example.com;
ssl_certificate /etc/nginx/ssl/tls.crt;
ssl_certificate_key /etc/nginx/ssl/tls.key;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
location / {
root /usr/share/nginx/html;
index index.html;
}
}
Events: <none>
vagrant@kube-master1:~/secret
$ cat nginx-pod-https.yml
apiVersion: v1
kind: Pod
metadata:
name: nginx-pod-https
spec:
containers:
- name: nginx-https
image: nginx
volumeMounts:
- name: nginx-tls-config
mountPath: /etc/nginx/conf.d
- name: https-cert
mountPath: /etc/nginx/ssl
readOnly: true
ports:
- containerPort: 80
protocol: TCP
- containerPort: 443
protocol: TCP
volumes:
- name: nginx-tls-config
configMap:
name: nginx-tls-config
- name: https-cert
secret:
secretName: nginx-tls-secret
vagrant@kube-master1:~/secret
$ kubectl create -f nginx-pod-https.yml
pod/nginx-pod-https created
vagrant@kube-master1:~/secret
$ kubectl get po
NAME READY STATUS RESTARTS AGE
nginx-pod-https 1/1 Running 0 7s
서비스 확인
- 8443 포트 사용 여부 확인 후 포트 포워딩 설정
vagrant@kube-master1:~/secret
$ netstat -anpt | grep :8443
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp 0 0 10.0.2.15:56940 10.233.89.41:8443 TIME_WAIT -
vagrant@kube-master1:~/secret
$ kubectl port-forward nginx-pod-https 8443:443
Forwarding from 127.0.0.1:8443 -> 443
Forwarding from [::1]:8443 -> 443
- 새 터미널에서 실행
- 인증서 적용이 정상적으로 됨
vagrant@kube-master1:~
$ curl https://localhost:8443 -k -v
* Rebuilt URL to: https://localhost:8443/
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Unknown (8):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Client hello (1):
* TLSv1.3 (OUT), TLS Unknown, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: CN=myapp.example.com
* start date: Jun 30 06:04:55 2022 GMT
* expire date: Jul 30 06:04:55 2022 GMT
* issuer: CN=myapp.example.com
* SSL certificate verify result: self signed certificate (18), continuing anyway.
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
> GET / HTTP/1.1
> Host: localhost:8443
> User-Agent: curl/7.58.0
> Accept: */*
>
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
< HTTP/1.1 200 OK
< Server: nginx/1.23.0
< Date: Thu, 30 Jun 2022 06:19:48 GMT
< Content-Type: text/html
< Content-Length: 615
< Last-Modified: Tue, 21 Jun 2022 14:25:37 GMT
< Connection: keep-alive
< ETag: "62b1d4e1-267"
< Accept-Ranges: bytes
<
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
* Connection #0 to host localhost left intact
참고
# 이벤트 확인
kubectl describe pod nginx-pod-https
# 어플리케이션 error 확인
kubectl log ngnix-pod-https
- 인증서 생성 시 에러 발생
- Can't load /home/vagrant/.rnd into RNG
$ openssl req -x509 -nodes -newkey rsa:2048 -keyout tls.key -out tls.crt -days 365
-subj "/CN=myapp.example.com"
Can't load /home/vagrant/.rnd into RNG
139993021399488:error:2406F079:random number generator:RAND_load_file:Cannot open file:../crypto/rand/randfile.c:88:Filename=/home/vagrant/.rnd
$ cd ~/; openssl rand -writerand .rnd
실습 - 3
TLS 종료 프록시 (Termination Proxy)
vagrant@kube-master1:~/secret
$ cat myapp-svc-np.yml
apiVersion: v1
kind: Service
metadata:
name: myapp-svc-np
spec:
type: NodePort
ports:
- port: 80
targetPort: 8080
nodePort: 31111
selector:
app: myapp-rs
$ cat myapp-rs.yml
apiVersion: apps/v1
kind: ReplicaSet
metadata:
name: myapp-rs
spec:
replicas: 3
selector:
matchLabels:
app: myapp-rs
template:
metadata:
labels:
app: myapp-rs
spec:
containers:
- name: myapp
image: ghcr.io/c1t1d0s7/go-myweb:alpine
ports:
- containerPort: 8080
vagrant@kube-master1:~/secret
$ kubectl create -f myapp-rs.yml -f myapp-svc-np.yml
replicaset.apps/myapp-rs created
service/myapp-svc-np created
vagrant@kube-master1:~/secret/ingress-tls
$ openssl req -x509 -nodes -newkey rsa:2048 -keyout tls.key -out tls.crt -days 365 -subj "/CN=myapp.example.com"
Generating a RSA private key
....................................................+++++
........................+++++
writing new private key to 'tls.key'
-----
vagrant@kube-master1:~/secret/ingress-tls
$ kubectl create secret tls ingress-tls-secret --cert=tls.crt --key=tls.key
secret/ingress-tls-secret created
vagrant@kube-master1:~/secret/ingress-tls
$ kubectl get secrets
NAME TYPE DATA AGE
default-token-wl2vl kubernetes.io/service-account-token 3 10d
ingress-tls-secret kubernetes.io/tls 2 9s
vagrant@kube-master1:~/secret
$ cat myapp-ing-tls-term.yml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: myapp-ing-tls-term
spec:
tls:
- hosts:
- myapp.example.com
secretName: ingress-tls-secret
rules:
- host: myapp.example.com
http:
paths:
- path: /
backend:
serviceName: myapp-svc-np
servicePort: 80
vagrant@kube-master1:~/secret
$ kubectl create -f myapp-ing-tls-term.yml
ingress.networking.k8s.io/myapp-ing-tls-term created
vagrant@kube-master1:~/secret
$ kubectl describe ingress myapp-ing-tls-term
Name: myapp-ing-tls-term
Namespace: default
Address: 192.168.56.21,192.168.56.22,192.168.56.23
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
TLS:
ingress-tls-secret terminates myapp.example.com
Rules:
Host Path Backends
---- ---- --------
myapp.example.com
/ myapp-svc-np:80 (10.233.101.113:8080,10.233.103.79:8080,10.233.76.161:8080)
Annotations: <none>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 5m39s nginx-ingress-controller Ingress default/myapp-ing-tls-term
Normal CREATE 5m39s nginx-ingress-controller Ingress default/myapp-ing-tls-term
Normal CREATE 5m39s nginx-ingress-controller Ingress default/myapp-ing-tls-term
Normal UPDATE 5m27s nginx-ingress-controller Ingress default/myapp-ing-tls-term
Normal UPDATE 5m27s nginx-ingress-controller Ingress default/myapp-ing-tls-term
Normal UPDATE 5m27s nginx-ingress-controller Ingress default/myapp-ing-tls-term
vagrant@kube-master1:~/secret
$ kubectl get po,svc,secret,ingress
NAME READY STATUS RESTARTS AGE
pod/myapp-rs-4vcvd 1/1 Running 0 106m
pod/myapp-rs-9spff 1/1 Running 0 106m
pod/myapp-rs-hxzdf 1/1 Running 0 106m
pod/nginx-pod-https 1/1 Running 0 162m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.233.0.1 <none> 443/TCP 5d22h
service/myapp-svc-np NodePort 10.233.3.139 <none> 80:31111/TCP 106m
NAME TYPE DATA AGE
secret/default-token-wl2vl kubernetes.io/service-account-token 3 10d
secret/ingress-tls-secret kubernetes.io/tls 2 8m12s
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.extensions/myapp-ing <none> www.192-168-56-21.nip.io 192.168.56.21,192.168.56.22,192.168.56.23 80 3d2h
ingress.extensions/myapp-ing-mpath <none> web1.example.com,web2.example.com 192.168.56.21,192.168.56.22,192.168.56.23 80 3d1h
ingress.extensions/myapp-ing-tls-term <none> myapp.example.com 192.168.56.21,192.168.56.22,192.168.56.23 80, 443 6m44s
서비스 확인
vagrant@kube-master1:~/secret
$ curl --resolve myapp.example.com:443:192.168.56.21 -k -v https://myapp.example.com
* Added myapp.example.com:443:192.168.56.21 to DNS cache
* Rebuilt URL to: https://myapp.example.com/
* Hostname myapp.example.com was found in DNS cache
* Trying 192.168.56.21...
* TCP_NODELAY set
* Connected to myapp.example.com (192.168.56.21) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Unknown (8):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Client hello (1):
* TLSv1.3 (OUT), TLS Unknown, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=myapp.example.com
* start date: Jun 30 08:45:52 2022 GMT
* expire date: Jun 30 08:45:52 2023 GMT
* issuer: CN=myapp.example.com
* SSL certificate verify result: self signed certificate (18), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* Using Stream ID: 1 (easy handle 0x5641f3eb8620)
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
> GET / HTTP/2
> Host: myapp.example.com
> User-Agent: curl/7.58.0
> Accept: */*
>
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
< HTTP/2 200
< server: nginx/1.19.2
< date: Thu, 30 Jun 2022 08:52:18 GMT
< content-type: text/plain; charset=utf-8
< content-length: 28
< strict-transport-security: max-age=15724800; includeSubDomains
<
Hello World!
myapp-rs-4vcvd
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
* Connection #0 to host myapp.example.com left intact
