Istio 4주차 정리
☸️ k8s(1.23.17) 배포 : NodePort(30000 HTTP, 30005 HTTPS)
1. 소스 코드 다운로드
1
2
3
4
5
6
git clone https://github.com/AcornPublishing/istio-in-action
cd istio-in-action/book-source-code-master
pwd # 각자 자신의 pwd 경로
# 결과
/home/devshin/workspace/istio/istio-in-action/book-source-code-master
2. Kind 클러스터 생성
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
kind create cluster --name myk8s --image kindest/node:v1.23.17 --config - <<EOF
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
extraPortMappings:
- containerPort: 30000 # Sample Application (istio-ingrssgateway) HTTP
hostPort: 30000
- containerPort: 30001 # Prometheus
hostPort: 30001
- containerPort: 30002 # Grafana
hostPort: 30002
- containerPort: 30003 # Kiali
hostPort: 30003
- containerPort: 30004 # Tracing
hostPort: 30004
- containerPort: 30005 # Sample Application (istio-ingrssgateway) HTTPS
hostPort: 30005
- containerPort: 30006 # TCP Route
hostPort: 30006
- containerPort: 30007 # kube-ops-view
hostPort: 30007
kubeadmConfigPatches:
- |
kind: ClusterConfiguration
controllerManager:
extraArgs:
bind-address: 0.0.0.0
extraMounts: # 해당 부분 생략 가능
- hostPath: /home/devshin/workspace/istio/istio-in-action/book-source-code-master # 각자 자신의 pwd 경로로 설정
containerPath: /istiobook
networking:
podSubnet: 10.10.0.0/16
serviceSubnet: 10.200.1.0/24
EOF
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
Creating cluster "myk8s" ...
✓ Ensuring node image (kindest/node:v1.23.17) 🖼
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
Set kubectl context to "kind-myk8s"
You can now use your cluster with:
kubectl cluster-info --context kind-myk8s
Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂
3. 클러스터 생성 확인
1
docker ps
✅ 출력
1
2
3
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7d34262e85f3 kindest/node:v1.23.17 "/usr/local/bin/entr…" 2 minutes ago Up 2 minutes 0.0.0.0:30000-30007->30000-30007/tcp, 127.0.0.1:34329->6443/tcp myk8s-control-plane
4. 노드에 기본 툴 설치
1
docker exec -it myk8s-control-plane sh -c 'apt update && apt install tree psmisc lsof wget bridge-utils net-tools dnsutils tcpdump ngrep iputils-ping git vim -y'
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
...
Setting up bind9-libs:amd64 (1:9.18.33-1~deb12u2) ...
Setting up openssh-client (1:9.2p1-2+deb12u5) ...
Setting up libxext6:amd64 (2:1.3.4-1+b1) ...
Setting up dbus-daemon (1.14.10-1~deb12u1) ...
Setting up libnet1:amd64 (1.1.6+dfsg-3.2) ...
Setting up libpcap0.8:amd64 (1.10.3-1) ...
Setting up dbus (1.14.10-1~deb12u1) ...
invoke-rc.d: policy-rc.d denied execution of start.
/usr/sbin/policy-rc.d returned 101, not running 'start dbus.service'
Setting up libgdbm-compat4:amd64 (1.23-3) ...
Setting up xauth (1:1.1.2-1) ...
Setting up bind9-host (1:9.18.33-1~deb12u2) ...
Setting up libperl5.36:amd64 (5.36.0-7+deb12u2) ...
Setting up tcpdump (4.99.3-1) ...
Setting up ngrep (1.47+ds1-5+b1) ...
Setting up perl (5.36.0-7+deb12u2) ...
Setting up bind9-dnsutils (1:9.18.33-1~deb12u2) ...
Setting up dnsutils (1:9.18.33-1~deb12u2) ...
Setting up liberror-perl (0.17029-2) ...
Setting up git (1:2.39.5-0+deb12u2) ...
Processing triggers for libc-bin (2.36-9+deb12u4) ...
🛡️ Istio 1.17.8 설치
1. myk8s-control-plane 진입
1
2
docker exec -it myk8s-control-plane bash
root@myk8s-control-plane:/#
2. (옵션) 코드 파일 마운트 확인
1
root@myk8s-control-plane:/# tree /istiobook/ -L 1
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
/istiobook/
|-- README.md
|-- appendices
|-- bin
|-- ch10
|-- ch11
|-- ch12
|-- ch13
|-- ch14
|-- ch2
|-- ch3
|-- ch4
|-- ch5
|-- ch6
|-- ch7
|-- ch8
|-- ch9
`-- services
17 directories, 1 file
3. istioctl 설치
1
2
3
4
5
6
root@myk8s-control-plane:/# export ISTIOV=1.17.8
echo 'export ISTIOV=1.17.8' >> /root/.bashrc
curl -s -L https://istio.io/downloadIstio | ISTIO_VERSION=$ISTIOV sh -
cp istio-$ISTIOV/bin/istioctl /usr/local/bin/istioctl
istioctl version --remote=false
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Downloading istio-1.17.8 from https://github.com/istio/istio/releases/download/1.17.8/istio-1.17.8-linux-amd64.tar.gz ...
Istio 1.17.8 download complete!
The Istio release archive has been downloaded to the istio-1.17.8 directory.
To configure the istioctl client tool for your workstation,
add the /istio-1.17.8/bin directory to your environment path variable with:
export PATH="$PATH:/istio-1.17.8/bin"
Begin the Istio pre-installation check by running:
istioctl x precheck
Try Istio in ambient mode
https://istio.io/latest/docs/ambient/getting-started/
Try Istio in sidecar mode
https://istio.io/latest/docs/setup/getting-started/
Install guides for ambient mode
https://istio.io/latest/docs/ambient/install/
Install guides for sidecar mode
https://istio.io/latest/docs/setup/install/
Need more information? Visit https://istio.io/latest/docs/
1.17.8
4. default 프로파일로 Istio 컨트롤 플레인 배포
1
istioctl install --set profile=default -y
✅ 출력
1
2
3
4
5
6
✔ Istio core installed
✔ Istiod installed
✔ Ingress gateways installed
✔ Installation complete Making this installation the default for injection and validation.
Thank you for installing Istio 1.17. Please take a few minutes to tell us about your install/upgrade experience! https://forms.gle/hMHGiwZHPU7UQRWe9
5. 실습을 위한 네임스페이스 설정
1
2
3
kubectl create ns istioinaction
kubectl label namespace istioinaction istio-injection=enabled
kubectl get ns --show-labels
✅ 출력
1
2
3
4
5
6
7
8
9
10
namespace/istioinaction created
namespace/istioinaction labeled
NAME STATUS AGE LABELS
default Active 13m kubernetes.io/metadata.name=default
istio-system Active 2m35s kubernetes.io/metadata.name=istio-system
istioinaction Active 0s istio-injection=enabled,kubernetes.io/metadata.name=istioinaction
kube-node-lease Active 13m kubernetes.io/metadata.name=kube-node-lease
kube-public Active 13m kubernetes.io/metadata.name=kube-public
kube-system Active 13m kubernetes.io/metadata.name=kube-system
local-path-storage Active 13m kubernetes.io/metadata.name=local-path-storage
6. istio-ingressgateway NodePort 및 트래픽 정책 수정
1
2
3
4
kubectl patch svc -n istio-system istio-ingressgateway -p '{"spec": {"type": "NodePort", "ports": [{"port": 80, "targetPort": 8080, "nodePort": 30000}]}}'
kubectl patch svc -n istio-system istio-ingressgateway -p '{"spec": {"type": "NodePort", "ports": [{"port": 443, "targetPort": 8443, "nodePort": 30005}]}}'
kubectl patch svc -n istio-system istio-ingressgateway -p '{"spec":{"externalTrafficPolicy": "Local"}}'
kubectl describe svc -n istio-system istio-ingressgateway
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
service/istio-ingressgateway patched
service/istio-ingressgateway patched
service/istio-ingressgateway patched
Name: istio-ingressgateway
Namespace: istio-system
Labels: app=istio-ingressgateway
install.operator.istio.io/owning-resource=unknown
install.operator.istio.io/owning-resource-namespace=istio-system
istio=ingressgateway
istio.io/rev=default
operator.istio.io/component=IngressGateways
operator.istio.io/managed=Reconcile
operator.istio.io/version=1.17.8
release=istio
Annotations: <none>
Selector: app=istio-ingressgateway,istio=ingressgateway
Type: NodePort
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.200.1.153
IPs: 10.200.1.153
Port: status-port 15021/TCP
TargetPort: 15021/TCP
NodePort: status-port 31958/TCP
Endpoints: 10.10.0.7:15021
Port: http2 80/TCP
TargetPort: 8080/TCP
NodePort: http2 30000/TCP
Endpoints: 10.10.0.7:8080
Port: https 443/TCP
TargetPort: 8443/TCP
NodePort: https 30005/TCP
Endpoints: 10.10.0.7:8443
Session Affinity: None
External Traffic Policy: Local
Internal Traffic Policy: Cluster
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Type 0s service-controller LoadBalancer -> NodePort
7. 내부 접속 테스트용 netshoot 파드 생성
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: netshoot
spec:
containers:
- name: netshoot
image: nicolaka/netshoot
command: ["tail"]
args: ["-f", "/dev/null"]
terminationGracePeriodSeconds: 0
EOF
# 결과
pod/netshoot created
📈 데이터 플레인의 메트릭 - 엔보이 표준/상세 메트릭 설정 및 확인
1. 초기화 및 실습 환경 구성
1
2
3
4
5
6
7
8
# catalog 앱 기동
kubectl apply -f services/catalog/kubernetes/catalog.yaml -n istioinaction
# webapp 앱 기동
kubectl apply -f services/webapp/kubernetes/webapp.yaml -n istioinaction
# gateway, virtualservice 설정
kubectl apply -f services/webapp/istio/webapp-catalog-gw-vs.yaml -n istioinaction
✅ 출력
1
2
3
4
5
6
7
8
serviceaccount/catalog created
service/catalog created
deployment.apps/catalog created
serviceaccount/webapp created
service/webapp created
deployment.apps/webapp created
gateway.networking.istio.io/coolstore-gateway created
virtualservice.networking.istio.io/webapp-virtualservice created
2. istioinaction 리소스 상태 확인
1
kubectl get deploy,pod,svc,ep,gw,vs -n istioinaction
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/catalog 1/1 1 1 41s
deployment.apps/webapp 1/1 1 1 41s
NAME READY STATUS RESTARTS AGE
pod/catalog-6cf4b97d-5kkz8 2/2 Running 0 41s
pod/webapp-7685bcb84-vlwjh 2/2 Running 0 41s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/catalog ClusterIP 10.200.1.143 <none> 80/TCP 41s
service/webapp ClusterIP 10.200.1.199 <none> 80/TCP 41s
NAME ENDPOINTS AGE
endpoints/catalog 10.10.0.9:3000 41s
endpoints/webapp 10.10.0.10:8080 41s
NAME AGE
gateway.networking.istio.io/coolstore-gateway 41s
NAME GATEWAYS HOSTS AGE
virtualservice.networking.istio.io/webapp-virtualservice ["coolstore-gateway"] ["webapp.istioinaction.io"] 41s
3. 웹앱 호출 테스트
1
curl -s http://webapp.istioinaction.io:30000/api/catalog | jq
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
[
{
"id": 1,
"color": "amber",
"department": "Eyewear",
"name": "Elinor Glasses",
"price": "282.00"
},
{
"id": 2,
"color": "cyan",
"department": "Clothing",
"name": "Atlas Shirt",
"price": "127.00"
},
{
"id": 3,
"color": "teal",
"department": "Clothing",
"name": "Small Metal Shoes",
"price": "232.00"
},
{
"id": 4,
"color": "red",
"department": "Watches",
"name": "Red Dragon Watch",
"price": "232.00"
}
]
4. 서비스의 사이드카 프록시가 유지하는 메트릭 확인
(1) istioinaction 네임스페이스 내 파드 상태 확인
1
kubectl get pod -n istioinaction
✅ 출력
1
2
3
NAME READY STATUS RESTARTS AGE
catalog-6cf4b97d-5kkz8 2/2 Running 0 3m11s
webapp-7685bcb84-vlwjh 2/2 Running 0 3m11s
(2) 프록시 연결 상태 확인
1
docker exec -it myk8s-control-plane istioctl proxy-status
✅ 출력
1
2
3
4
NAME CLUSTER CDS LDS EDS RDS ECDS ISTIOD VERSION
catalog-6cf4b97d-5kkz8.istioinaction Kubernetes SYNCED SYNCED SYNCED SYNCED NOT SENT istiod-7df6ffc78d-t5kbh 1.17.8
istio-ingressgateway-996bc6bb6-crh8b.istio-system Kubernetes SYNCED SYNCED SYNCED SYNCED NOT SENT istiod-7df6ffc78d-t5kbh 1.17.8
webapp-7685bcb84-vlwjh.istioinaction Kubernetes SYNCED SYNCED SYNCED SYNCED NOT SENT istiod-7df6ffc78d-t5kbh 1.17.8
(3) catalog 서비스 엔보이 메트릭 전체 조회
1
kubectl exec -it deploy/catalog -c istio-proxy -n istioinaction -- curl localhost:15000/stats
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
cluster_manager.cds.version_text: "2025-05-03T08:56:10Z/10"
listener_manager.lds.version_text: "2025-05-03T08:56:10Z/10"
cluster.xds-grpc.assignment_stale: 0
cluster.xds-grpc.assignment_timeout_received: 0
cluster.xds-grpc.bind_errors: 0
cluster.xds-grpc.circuit_breakers.default.cx_open: 0
cluster.xds-grpc.circuit_breakers.default.cx_pool_open: 0
cluster.xds-grpc.circuit_breakers.default.rq_open: 0
cluster.xds-grpc.circuit_breakers.default.rq_pending_open: 0
cluster.xds-grpc.circuit_breakers.default.rq_retry_open: 0
cluster.xds-grpc.circuit_breakers.high.cx_open: 0
cluster.xds-grpc.circuit_breakers.high.cx_pool_open: 0
cluster.xds-grpc.circuit_breakers.high.rq_open: 0
cluster.xds-grpc.circuit_breakers.high.rq_pending_open: 0
cluster.xds-grpc.circuit_breakers.high.rq_retry_open: 0
cluster.xds-grpc.default.total_match_count: 1
cluster.xds-grpc.http2.deferred_stream_close: 0
cluster.xds-grpc.http2.dropped_headers_with_underscores: 0
cluster.xds-grpc.http2.header_overflow: 0
cluster.xds-grpc.http2.headers_cb_no_stream: 0
cluster.xds-grpc.http2.inbound_empty_frames_flood: 0
cluster.xds-grpc.http2.inbound_priority_frames_flood: 0
cluster.xds-grpc.http2.inbound_window_update_frames_flood: 0
cluster.xds-grpc.http2.keepalive_timeout: 0
cluster.xds-grpc.http2.metadata_empty_frames: 0
cluster.xds-grpc.http2.outbound_control_flood: 0
cluster.xds-grpc.http2.outbound_flood: 0
cluster.xds-grpc.http2.pending_send_bytes: 0
cluster.xds-grpc.http2.requests_rejected_with_underscores_in_headers: 0
cluster.xds-grpc.http2.rx_messaging_error: 0
cluster.xds-grpc.http2.rx_reset: 0
cluster.xds-grpc.http2.stream_refused_errors: 0
cluster.xds-grpc.http2.streams_active: 1
cluster.xds-grpc.http2.trailers: 0
cluster.xds-grpc.http2.tx_flush_timeout: 0
cluster.xds-grpc.http2.tx_reset: 0
cluster.xds-grpc.internal.upstream_rq_200: 1
cluster.xds-grpc.internal.upstream_rq_2xx: 1
cluster.xds-grpc.internal.upstream_rq_completed: 1
cluster.xds-grpc.lb_healthy_panic: 0
cluster.xds-grpc.lb_local_cluster_not_ok: 0
cluster.xds-grpc.lb_recalculate_zone_structures: 0
cluster.xds-grpc.lb_subsets_active: 0
cluster.xds-grpc.lb_subsets_created: 0
cluster.xds-grpc.lb_subsets_fallback: 0
cluster.xds-grpc.lb_subsets_fallback_panic: 0
cluster.xds-grpc.lb_subsets_removed: 0
cluster.xds-grpc.lb_subsets_selected: 0
cluster.xds-grpc.lb_zone_cluster_too_small: 0
cluster.xds-grpc.lb_zone_no_capacity_left: 0
cluster.xds-grpc.lb_zone_number_differs: 0
cluster.xds-grpc.lb_zone_routing_all_directly: 0
cluster.xds-grpc.lb_zone_routing_cross_zone: 0
cluster.xds-grpc.lb_zone_routing_sampled: 0
cluster.xds-grpc.max_host_weight: 0
cluster.xds-grpc.membership_change: 1
cluster.xds-grpc.membership_degraded: 0
cluster.xds-grpc.membership_excluded: 0
cluster.xds-grpc.membership_healthy: 1
cluster.xds-grpc.membership_total: 1
cluster.xds-grpc.original_dst_host_invalid: 0
cluster.xds-grpc.retry_or_shadow_abandoned: 0
cluster.xds-grpc.update_attempt: 0
cluster.xds-grpc.update_empty: 0
cluster.xds-grpc.update_failure: 0
cluster.xds-grpc.update_no_rebuild: 0
cluster.xds-grpc.update_success: 0
cluster.xds-grpc.upstream_cx_active: 1
cluster.xds-grpc.upstream_cx_close_notify: 0
cluster.xds-grpc.upstream_cx_connect_attempts_exceeded: 0
cluster.xds-grpc.upstream_cx_connect_fail: 0
cluster.xds-grpc.upstream_cx_connect_timeout: 0
cluster.xds-grpc.upstream_cx_connect_with_0_rtt: 0
cluster.xds-grpc.upstream_cx_destroy: 0
cluster.xds-grpc.upstream_cx_destroy_local: 0
cluster.xds-grpc.upstream_cx_destroy_local_with_active_rq: 0
cluster.xds-grpc.upstream_cx_destroy_remote: 0
cluster.xds-grpc.upstream_cx_destroy_remote_with_active_rq: 0
cluster.xds-grpc.upstream_cx_destroy_with_active_rq: 0
cluster.xds-grpc.upstream_cx_http1_total: 0
cluster.xds-grpc.upstream_cx_http2_total: 1
cluster.xds-grpc.upstream_cx_http3_total: 0
cluster.xds-grpc.upstream_cx_idle_timeout: 0
cluster.xds-grpc.upstream_cx_max_duration_reached: 0
cluster.xds-grpc.upstream_cx_max_requests: 1
cluster.xds-grpc.upstream_cx_none_healthy: 0
cluster.xds-grpc.upstream_cx_overflow: 0
cluster.xds-grpc.upstream_cx_pool_overflow: 0
cluster.xds-grpc.upstream_cx_protocol_error: 0
cluster.xds-grpc.upstream_cx_rx_bytes_buffered: 17
cluster.xds-grpc.upstream_cx_rx_bytes_total: 168737
cluster.xds-grpc.upstream_cx_total: 1
cluster.xds-grpc.upstream_cx_tx_bytes_buffered: 0
cluster.xds-grpc.upstream_cx_tx_bytes_total: 38181
cluster.xds-grpc.upstream_flow_control_backed_up_total: 0
cluster.xds-grpc.upstream_flow_control_drained_total: 0
cluster.xds-grpc.upstream_flow_control_paused_reading_total: 0
cluster.xds-grpc.upstream_flow_control_resumed_reading_total: 0
cluster.xds-grpc.upstream_http3_broken: 0
cluster.xds-grpc.upstream_internal_redirect_failed_total: 0
cluster.xds-grpc.upstream_internal_redirect_succeeded_total: 0
cluster.xds-grpc.upstream_rq_0rtt: 0
cluster.xds-grpc.upstream_rq_200: 1
cluster.xds-grpc.upstream_rq_2xx: 1
cluster.xds-grpc.upstream_rq_active: 1
cluster.xds-grpc.upstream_rq_cancelled: 0
cluster.xds-grpc.upstream_rq_completed: 1
cluster.xds-grpc.upstream_rq_maintenance_mode: 0
cluster.xds-grpc.upstream_rq_max_duration_reached: 0
cluster.xds-grpc.upstream_rq_pending_active: 0
cluster.xds-grpc.upstream_rq_pending_failure_eject: 0
cluster.xds-grpc.upstream_rq_pending_overflow: 0
cluster.xds-grpc.upstream_rq_pending_total: 1
cluster.xds-grpc.upstream_rq_per_try_idle_timeout: 0
cluster.xds-grpc.upstream_rq_per_try_timeout: 0
cluster.xds-grpc.upstream_rq_retry: 0
cluster.xds-grpc.upstream_rq_retry_backoff_exponential: 0
cluster.xds-grpc.upstream_rq_retry_backoff_ratelimited: 0
cluster.xds-grpc.upstream_rq_retry_limit_exceeded: 0
cluster.xds-grpc.upstream_rq_retry_overflow: 0
cluster.xds-grpc.upstream_rq_retry_success: 0
cluster.xds-grpc.upstream_rq_rx_reset: 0
cluster.xds-grpc.upstream_rq_timeout: 0
cluster.xds-grpc.upstream_rq_total: 1
cluster.xds-grpc.upstream_rq_tx_reset: 0
cluster.xds-grpc.version: 0
cluster_manager.active_clusters: 22
cluster_manager.cds.init_fetch_timeout: 0
cluster_manager.cds.update_attempt: 4
cluster_manager.cds.update_failure: 0
cluster_manager.cds.update_rejected: 0
cluster_manager.cds.update_success: 3
cluster_manager.cds.update_time: 1746262570459
cluster_manager.cds.version: 11248753202267649098
cluster_manager.cluster_added: 22
cluster_manager.cluster_modified: 1
cluster_manager.cluster_removed: 0
cluster_manager.cluster_updated: 4
cluster_manager.cluster_updated_via_merge: 0
cluster_manager.update_merge_cancelled: 0
cluster_manager.update_out_of_merge_window: 0
cluster_manager.warming_clusters: 0
istiocustom.istio_build.component.proxy.tag.1.17.8: 1
istiocustom.istio_requests_total.reporter.destination.source_workload.webapp.source_canonical_service.webapp.source_canonical_revision.latest.source_workload_namespace.istioinaction.source_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.source_app.webapp.source_version.unknown.source_cluster.Kubernetes.destination_workload.catalog.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/catalog.destination_app.catalog.destination_version.v1.destination_service.catalog.istioinaction.svc.cluster.local.destination_canonical_service.catalog.destination_canonical_revision.v1.destination_service_name.catalog.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.mutual_tls: 2
listener_manager.lds.init_fetch_timeout: 0
listener_manager.lds.update_attempt: 4
listener_manager.lds.update_failure: 0
listener_manager.lds.update_rejected: 0
listener_manager.lds.update_success: 3
listener_manager.lds.update_time: 1746262570460
listener_manager.lds.version: 11248753202267649098
listener_manager.listener_added: 15
listener_manager.listener_create_failure: 0
listener_manager.listener_create_success: 30
listener_manager.listener_in_place_updated: 0
listener_manager.listener_modified: 0
listener_manager.listener_removed: 0
listener_manager.listener_stopped: 0
listener_manager.total_filter_chains_draining: 0
listener_manager.total_listeners_active: 15
listener_manager.total_listeners_draining: 0
listener_manager.total_listeners_warming: 0
listener_manager.workers_started: 1
server.compilation_settings.fips_mode: 0
server.concurrency: 2
server.days_until_first_cert_expiring: 0
server.debug_assertion_failures: 0
server.dropped_stat_flushes: 0
server.dynamic_unknown_fields: 0
server.envoy_bug_failures: 0
server.hot_restart_epoch: 0
server.live: 1
server.main_thread.watchdog_mega_miss: 0
server.main_thread.watchdog_miss: 0
server.memory_allocated: 9582616
server.memory_heap_size: 11534336
server.memory_physical_size: 14155776
server.parent_connections: 0
server.seconds_until_first_ocsp_response_expiring: 0
server.state: 0
server.static_unknown_fields: 0
server.stats_recent_lookups: 4578
server.total_connections: 1
server.uptime: 245
server.version: 14754347
server.wip_protos: 1
server.worker_0.watchdog_mega_miss: 0
server.worker_0.watchdog_miss: 0
server.worker_1.watchdog_mega_miss: 0
server.worker_1.watchdog_miss: 0
wasm.envoy.wasm.runtime.null.active: 4
wasm.envoy.wasm.runtime.null.created: 6
wasm.remote_load_cache_entries: 0
wasm.remote_load_cache_hits: 0
wasm.remote_load_cache_misses: 0
wasm.remote_load_cache_negative_hits: 0
wasm.remote_load_fetch_failures: 0
wasm.remote_load_fetch_successes: 0
cluster.xds-grpc.upstream_cx_connect_ms: P0(nan,4) P25(nan,4.025) P50(nan,4.05) P75(nan,4.075) P90(nan,4.09) P95(nan,4.095) P99(nan,4.099) P99.5(nan,4.0995) P99.9(nan,4.0999) P100(nan,4.1)
cluster.xds-grpc.upstream_cx_length_ms: No recorded values
cluster_manager.cds.update_duration: P0(nan,0) P25(nan,0) P50(nan,1.05) P75(nan,3.025) P90(nan,3.07) P95(nan,3.085) P99(nan,3.097) P99.5(nan,3.0985) P99.9(nan,3.0997) P100(nan,3.1)
istiocustom.istio_request_bytes.reporter.destination.source_workload.webapp.source_canonical_service.webapp.source_canonical_revision.latest.source_workload_namespace.istioinaction.source_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.source_app.webapp.source_version.unknown.source_cluster.Kubernetes.destination_workload.catalog.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/catalog.destination_app.catalog.destination_version.v1.destination_service.catalog.istioinaction.svc.cluster.local.destination_canonical_service.catalog.destination_canonical_revision.v1.destination_service_name.catalog.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.mutual_tls: P0(nan,1200) P25(nan,1225) P50(nan,1250) P75(nan,1275) P90(nan,1290) P95(nan,1295) P99(nan,1299) P99.5(nan,1299.5) P99.9(nan,1299.9) P100(nan,1300)
istiocustom.istio_request_duration_milliseconds.reporter.destination.source_workload.webapp.source_canonical_service.webapp.source_canonical_revision.latest.source_workload_namespace.istioinaction.source_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.source_app.webapp.source_version.unknown.source_cluster.Kubernetes.destination_workload.catalog.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/catalog.destination_app.catalog.destination_version.v1.destination_service.catalog.istioinaction.svc.cluster.local.destination_canonical_service.catalog.destination_canonical_revision.v1.destination_service_name.catalog.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.mutual_tls: P0(nan,1) P25(nan,1.05) P50(nan,1.1) P75(nan,8.05) P90(nan,8.08) P95(nan,8.09) P99(nan,8.098) P99.5(nan,8.099) P99.9(nan,8.0998) P100(nan,8.1)
istiocustom.istio_response_bytes.reporter.destination.source_workload.webapp.source_canonical_service.webapp.source_canonical_revision.latest.source_workload_namespace.istioinaction.source_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.source_app.webapp.source_version.unknown.source_cluster.Kubernetes.destination_workload.catalog.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/catalog.destination_app.catalog.destination_version.v1.destination_service.catalog.istioinaction.svc.cluster.local.destination_canonical_service.catalog.destination_canonical_revision.v1.destination_service_name.catalog.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.mutual_tls: P0(nan,1700) P25(nan,1725) P50(nan,1750) P75(nan,1775) P90(nan,1790) P95(nan,1795) P99(nan,1799) P99.5(nan,1799.5) P99.9(nan,1799.9) P100(nan,1800)
listener_manager.lds.update_duration: P0(nan,0) P25(nan,0) P50(nan,1.05) P75(nan,10.25) P90(nan,10.7) P95(nan,10.85) P99(nan,10.969999999999999) P99.5(nan,10.985) P99.9(nan,10.997) P100(nan,11)
server.initialization_time_ms: P0(nan,150) P25(nan,152.5) P50(nan,155) P75(nan,157.5) P90(nan,159) P95(nan,159.5) P99(nan,159.9) P99.5(nan,159.95) P99.9(nan,159.99) P100(nan,160)
(4) webapp 서비스 엔보이 메트릭 전체 조회
1
kubectl exec -it deploy/webapp -c istio-proxy -n istioinaction -- curl localhost:15000/stats
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
cluster_manager.cds.version_text: "2025-05-03T08:56:10Z/10"
listener_manager.lds.version_text: "2025-05-03T08:56:10Z/10"
cluster.xds-grpc.assignment_stale: 0
cluster.xds-grpc.assignment_timeout_received: 0
cluster.xds-grpc.bind_errors: 0
cluster.xds-grpc.circuit_breakers.default.cx_open: 0
cluster.xds-grpc.circuit_breakers.default.cx_pool_open: 0
cluster.xds-grpc.circuit_breakers.default.rq_open: 0
cluster.xds-grpc.circuit_breakers.default.rq_pending_open: 0
cluster.xds-grpc.circuit_breakers.default.rq_retry_open: 0
cluster.xds-grpc.circuit_breakers.high.cx_open: 0
cluster.xds-grpc.circuit_breakers.high.cx_pool_open: 0
cluster.xds-grpc.circuit_breakers.high.rq_open: 0
cluster.xds-grpc.circuit_breakers.high.rq_pending_open: 0
cluster.xds-grpc.circuit_breakers.high.rq_retry_open: 0
cluster.xds-grpc.default.total_match_count: 1
cluster.xds-grpc.http2.deferred_stream_close: 0
cluster.xds-grpc.http2.dropped_headers_with_underscores: 0
cluster.xds-grpc.http2.header_overflow: 0
cluster.xds-grpc.http2.headers_cb_no_stream: 0
cluster.xds-grpc.http2.inbound_empty_frames_flood: 0
cluster.xds-grpc.http2.inbound_priority_frames_flood: 0
cluster.xds-grpc.http2.inbound_window_update_frames_flood: 0
cluster.xds-grpc.http2.keepalive_timeout: 0
cluster.xds-grpc.http2.metadata_empty_frames: 0
cluster.xds-grpc.http2.outbound_control_flood: 0
cluster.xds-grpc.http2.outbound_flood: 0
cluster.xds-grpc.http2.pending_send_bytes: 0
cluster.xds-grpc.http2.requests_rejected_with_underscores_in_headers: 0
cluster.xds-grpc.http2.rx_messaging_error: 0
cluster.xds-grpc.http2.rx_reset: 0
cluster.xds-grpc.http2.stream_refused_errors: 0
cluster.xds-grpc.http2.streams_active: 1
cluster.xds-grpc.http2.trailers: 0
cluster.xds-grpc.http2.tx_flush_timeout: 0
cluster.xds-grpc.http2.tx_reset: 0
cluster.xds-grpc.internal.upstream_rq_200: 1
cluster.xds-grpc.internal.upstream_rq_2xx: 1
cluster.xds-grpc.internal.upstream_rq_completed: 1
cluster.xds-grpc.lb_healthy_panic: 0
cluster.xds-grpc.lb_local_cluster_not_ok: 0
cluster.xds-grpc.lb_recalculate_zone_structures: 0
cluster.xds-grpc.lb_subsets_active: 0
cluster.xds-grpc.lb_subsets_created: 0
cluster.xds-grpc.lb_subsets_fallback: 0
cluster.xds-grpc.lb_subsets_fallback_panic: 0
cluster.xds-grpc.lb_subsets_removed: 0
cluster.xds-grpc.lb_subsets_selected: 0
cluster.xds-grpc.lb_zone_cluster_too_small: 0
cluster.xds-grpc.lb_zone_no_capacity_left: 0
cluster.xds-grpc.lb_zone_number_differs: 0
cluster.xds-grpc.lb_zone_routing_all_directly: 0
cluster.xds-grpc.lb_zone_routing_cross_zone: 0
cluster.xds-grpc.lb_zone_routing_sampled: 0
cluster.xds-grpc.max_host_weight: 0
cluster.xds-grpc.membership_change: 1
cluster.xds-grpc.membership_degraded: 0
cluster.xds-grpc.membership_excluded: 0
cluster.xds-grpc.membership_healthy: 1
cluster.xds-grpc.membership_total: 1
cluster.xds-grpc.original_dst_host_invalid: 0
cluster.xds-grpc.retry_or_shadow_abandoned: 0
cluster.xds-grpc.update_attempt: 0
cluster.xds-grpc.update_empty: 0
cluster.xds-grpc.update_failure: 0
cluster.xds-grpc.update_no_rebuild: 0
cluster.xds-grpc.update_success: 0
cluster.xds-grpc.upstream_cx_active: 1
cluster.xds-grpc.upstream_cx_close_notify: 0
cluster.xds-grpc.upstream_cx_connect_attempts_exceeded: 0
cluster.xds-grpc.upstream_cx_connect_fail: 0
cluster.xds-grpc.upstream_cx_connect_timeout: 0
cluster.xds-grpc.upstream_cx_connect_with_0_rtt: 0
cluster.xds-grpc.upstream_cx_destroy: 0
cluster.xds-grpc.upstream_cx_destroy_local: 0
cluster.xds-grpc.upstream_cx_destroy_local_with_active_rq: 0
cluster.xds-grpc.upstream_cx_destroy_remote: 0
cluster.xds-grpc.upstream_cx_destroy_remote_with_active_rq: 0
cluster.xds-grpc.upstream_cx_destroy_with_active_rq: 0
cluster.xds-grpc.upstream_cx_http1_total: 0
cluster.xds-grpc.upstream_cx_http2_total: 1
cluster.xds-grpc.upstream_cx_http3_total: 0
cluster.xds-grpc.upstream_cx_idle_timeout: 0
cluster.xds-grpc.upstream_cx_max_duration_reached: 0
cluster.xds-grpc.upstream_cx_max_requests: 1
cluster.xds-grpc.upstream_cx_none_healthy: 0
cluster.xds-grpc.upstream_cx_overflow: 0
cluster.xds-grpc.upstream_cx_pool_overflow: 0
cluster.xds-grpc.upstream_cx_protocol_error: 0
cluster.xds-grpc.upstream_cx_rx_bytes_buffered: 17
cluster.xds-grpc.upstream_cx_rx_bytes_total: 222555
cluster.xds-grpc.upstream_cx_total: 1
cluster.xds-grpc.upstream_cx_tx_bytes_buffered: 0
cluster.xds-grpc.upstream_cx_tx_bytes_total: 40476
cluster.xds-grpc.upstream_flow_control_backed_up_total: 0
cluster.xds-grpc.upstream_flow_control_drained_total: 0
cluster.xds-grpc.upstream_flow_control_paused_reading_total: 0
cluster.xds-grpc.upstream_flow_control_resumed_reading_total: 0
cluster.xds-grpc.upstream_http3_broken: 0
cluster.xds-grpc.upstream_internal_redirect_failed_total: 0
cluster.xds-grpc.upstream_internal_redirect_succeeded_total: 0
cluster.xds-grpc.upstream_rq_0rtt: 0
cluster.xds-grpc.upstream_rq_200: 1
cluster.xds-grpc.upstream_rq_2xx: 1
cluster.xds-grpc.upstream_rq_active: 1
cluster.xds-grpc.upstream_rq_cancelled: 0
cluster.xds-grpc.upstream_rq_completed: 1
cluster.xds-grpc.upstream_rq_maintenance_mode: 0
cluster.xds-grpc.upstream_rq_max_duration_reached: 0
cluster.xds-grpc.upstream_rq_pending_active: 0
cluster.xds-grpc.upstream_rq_pending_failure_eject: 0
cluster.xds-grpc.upstream_rq_pending_overflow: 0
cluster.xds-grpc.upstream_rq_pending_total: 1
cluster.xds-grpc.upstream_rq_per_try_idle_timeout: 0
cluster.xds-grpc.upstream_rq_per_try_timeout: 0
cluster.xds-grpc.upstream_rq_retry: 0
cluster.xds-grpc.upstream_rq_retry_backoff_exponential: 0
cluster.xds-grpc.upstream_rq_retry_backoff_ratelimited: 0
cluster.xds-grpc.upstream_rq_retry_limit_exceeded: 0
cluster.xds-grpc.upstream_rq_retry_overflow: 0
cluster.xds-grpc.upstream_rq_retry_success: 0
cluster.xds-grpc.upstream_rq_rx_reset: 0
cluster.xds-grpc.upstream_rq_timeout: 0
cluster.xds-grpc.upstream_rq_total: 1
cluster.xds-grpc.upstream_rq_tx_reset: 0
cluster.xds-grpc.version: 0
cluster_manager.active_clusters: 22
cluster_manager.cds.init_fetch_timeout: 0
cluster_manager.cds.update_attempt: 5
cluster_manager.cds.update_failure: 0
cluster_manager.cds.update_rejected: 0
cluster_manager.cds.update_success: 4
cluster_manager.cds.update_time: 1746262570460
cluster_manager.cds.version: 11248753202267649098
cluster_manager.cluster_added: 22
cluster_manager.cluster_modified: 2
cluster_manager.cluster_removed: 0
cluster_manager.cluster_updated: 1
cluster_manager.cluster_updated_via_merge: 0
cluster_manager.update_merge_cancelled: 0
cluster_manager.update_out_of_merge_window: 0
cluster_manager.warming_clusters: 0
istiocustom.istio_build.component.proxy.tag.1.17.8: 1
istiocustom.istio_requests_total.reporter.destination.source_workload.istio-ingressgateway.source_canonical_service.istio-ingressgateway.source_canonical_revision.latest.source_workload_namespace.istio-system.source_principal.spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account.source_app.istio-ingressgateway.source_version.unknown.source_cluster.Kubernetes.destination_workload.webapp.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.destination_app.webapp.destination_version.destination_service.webapp.istioinaction.svc.cluster.local.destination_canonical_service.webapp.destination_canonical_revision.latest.destination_service_name.webapp.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.mutual_tls: 2
istiocustom.istio_requests_total.reporter.source.source_workload.webapp.source_canonical_service.webapp.source_canonical_revision.latest.source_workload_namespace.istioinaction.source_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.source_app.webapp.source_version.source_cluster.Kubernetes.destination_workload.catalog.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/catalog.destination_app.catalog.destination_version.v1.destination_service.catalog.istioinaction.svc.cluster.local.destination_canonical_service.catalog.destination_canonical_revision.v1.destination_service_name.catalog.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.unknown: 2
listener_manager.lds.init_fetch_timeout: 0
listener_manager.lds.update_attempt: 5
listener_manager.lds.update_failure: 0
listener_manager.lds.update_rejected: 0
listener_manager.lds.update_success: 4
listener_manager.lds.update_time: 1746262570462
listener_manager.lds.version: 11248753202267649098
listener_manager.listener_added: 15
listener_manager.listener_create_failure: 0
listener_manager.listener_create_success: 30
listener_manager.listener_in_place_updated: 0
listener_manager.listener_modified: 0
listener_manager.listener_removed: 0
listener_manager.listener_stopped: 0
listener_manager.total_filter_chains_draining: 0
listener_manager.total_listeners_active: 15
listener_manager.total_listeners_draining: 0
listener_manager.total_listeners_warming: 0
listener_manager.workers_started: 1
server.compilation_settings.fips_mode: 0
server.concurrency: 2
server.days_until_first_cert_expiring: 0
server.debug_assertion_failures: 0
server.dropped_stat_flushes: 0
server.dynamic_unknown_fields: 0
server.envoy_bug_failures: 0
server.hot_restart_epoch: 0
server.live: 1
server.main_thread.watchdog_mega_miss: 0
server.main_thread.watchdog_miss: 0
server.memory_allocated: 9697048
server.memory_heap_size: 11534336
server.memory_physical_size: 14155776
server.parent_connections: 0
server.seconds_until_first_ocsp_response_expiring: 0
server.state: 0
server.static_unknown_fields: 0
server.stats_recent_lookups: 4768
server.total_connections: 2
server.uptime: 310
server.version: 14754347
server.wip_protos: 1
server.worker_0.watchdog_mega_miss: 0
server.worker_0.watchdog_miss: 0
server.worker_1.watchdog_mega_miss: 0
server.worker_1.watchdog_miss: 0
wasm.envoy.wasm.runtime.null.active: 4
wasm.envoy.wasm.runtime.null.created: 6
wasm.remote_load_cache_entries: 0
wasm.remote_load_cache_hits: 0
wasm.remote_load_cache_misses: 0
wasm.remote_load_cache_negative_hits: 0
wasm.remote_load_fetch_failures: 0
wasm.remote_load_fetch_successes: 0
cluster.xds-grpc.upstream_cx_connect_ms: P0(nan,3) P25(nan,3.025) P50(nan,3.05) P75(nan,3.075) P90(nan,3.09) P95(nan,3.095) P99(nan,3.099) P99.5(nan,3.0995) P99.9(nan,3.0999) P100(nan,3.1)
cluster.xds-grpc.upstream_cx_length_ms: No recorded values
cluster_manager.cds.update_duration: P0(nan,0) P25(nan,0) P50(nan,2.05) P75(nan,2.1) P90(nan,4.06) P95(nan,4.08) P99(nan,4.096) P99.5(nan,4.098) P99.9(nan,4.0996) P100(nan,4.1)
istiocustom.istio_request_bytes.reporter.destination.source_workload.istio-ingressgateway.source_canonical_service.istio-ingressgateway.source_canonical_revision.latest.source_workload_namespace.istio-system.source_principal.spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account.source_app.istio-ingressgateway.source_version.unknown.source_cluster.Kubernetes.destination_workload.webapp.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.destination_app.webapp.destination_version.destination_service.webapp.istioinaction.svc.cluster.local.destination_canonical_service.webapp.destination_canonical_revision.latest.destination_service_name.webapp.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.mutual_tls: P0(nan,1600) P25(nan,1625) P50(nan,1650) P75(nan,1675) P90(nan,1690) P95(nan,1695) P99(nan,1699) P99.5(nan,1699.5) P99.9(nan,1699.9) P100(nan,1700)
istiocustom.istio_request_bytes.reporter.source.source_workload.webapp.source_canonical_service.webapp.source_canonical_revision.latest.source_workload_namespace.istioinaction.source_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.source_app.webapp.source_version.source_cluster.Kubernetes.destination_workload.catalog.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/catalog.destination_app.catalog.destination_version.v1.destination_service.catalog.istioinaction.svc.cluster.local.destination_canonical_service.catalog.destination_canonical_revision.v1.destination_service_name.catalog.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.unknown: P0(nan,630) P25(nan,632.5) P50(nan,635) P75(nan,637.5) P90(nan,639) P95(nan,639.5) P99(nan,639.9) P99.5(nan,639.95) P99.9(nan,639.99) P100(nan,640)
istiocustom.istio_request_duration_milliseconds.reporter.destination.source_workload.istio-ingressgateway.source_canonical_service.istio-ingressgateway.source_canonical_revision.latest.source_workload_namespace.istio-system.source_principal.spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account.source_app.istio-ingressgateway.source_version.unknown.source_cluster.Kubernetes.destination_workload.webapp.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.destination_app.webapp.destination_version.destination_service.webapp.istioinaction.svc.cluster.local.destination_canonical_service.webapp.destination_canonical_revision.latest.destination_service_name.webapp.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.mutual_tls: P0(nan,4) P25(nan,4.05) P50(nan,4.1) P75(nan,79.5) P90(nan,79.8) P95(nan,79.9) P99(nan,79.98) P99.5(nan,79.99) P99.9(nan,79.998) P100(nan,80)
istiocustom.istio_request_duration_milliseconds.reporter.source.source_workload.webapp.source_canonical_service.webapp.source_canonical_revision.latest.source_workload_namespace.istioinaction.source_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.source_app.webapp.source_version.source_cluster.Kubernetes.destination_workload.catalog.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/catalog.destination_app.catalog.destination_version.v1.destination_service.catalog.istioinaction.svc.cluster.local.destination_canonical_service.catalog.destination_canonical_revision.v1.destination_service_name.catalog.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.unknown: P0(nan,2) P25(nan,2.05) P50(nan,2.1) P75(nan,18.5) P90(nan,18.8) P95(nan,18.9) P99(nan,18.98) P99.5(nan,18.99) P99.9(nan,18.998) P100(nan,19)
istiocustom.istio_response_bytes.reporter.destination.source_workload.istio-ingressgateway.source_canonical_service.istio-ingressgateway.source_canonical_revision.latest.source_workload_namespace.istio-system.source_principal.spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account.source_app.istio-ingressgateway.source_version.unknown.source_cluster.Kubernetes.destination_workload.webapp.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.destination_app.webapp.destination_version.destination_service.webapp.istioinaction.svc.cluster.local.destination_canonical_service.webapp.destination_canonical_revision.latest.destination_service_name.webapp.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.mutual_tls: P0(nan,1300) P25(nan,1350) P50(nan,1400) P75(nan,6550) P90(nan,6580) P95(nan,6590) P99(nan,6598) P99.5(nan,6599) P99.9(nan,6599.8) P100(nan,6600)
istiocustom.istio_response_bytes.reporter.source.source_workload.webapp.source_canonical_service.webapp.source_canonical_revision.latest.source_workload_namespace.istioinaction.source_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.source_app.webapp.source_version.source_cluster.Kubernetes.destination_workload.catalog.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/catalog.destination_app.catalog.destination_version.v1.destination_service.catalog.istioinaction.svc.cluster.local.destination_canonical_service.catalog.destination_canonical_revision.v1.destination_service_name.catalog.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.unknown: P0(nan,860) P25(nan,862.5) P50(nan,865) P75(nan,867.5) P90(nan,869) P95(nan,869.5) P99(nan,869.9) P99.5(nan,869.95) P99.9(nan,869.99) P100(nan,870)
listener_manager.lds.update_duration: P0(nan,0) P25(nan,0) P50(nan,0) P75(nan,1.1) P90(nan,11.6) P95(nan,11.8) P99(nan,11.96) P99.5(nan,11.98) P99.9(nan,11.996) P100(nan,12)
server.initialization_time_ms: P0(nan,150) P25(nan,152.5) P50(nan,155) P75(nan,157.5) P90(nan,159) P95(nan,159.5) P99(nan,159.9) P99.5(nan,159.95) P99.9(nan,159.99) P100(nan,160)
📊 프록시가 엔보이 통계를 더 많이 보고하도록 설정하기
1. 프록시 통계 확대 전 기본 API 호출 테스트
1
curl -s http://webapp.istioinaction.io:30000/api/catalog | jq
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
[
{
"id": 1,
"color": "amber",
"department": "Eyewear",
"name": "Elinor Glasses",
"price": "282.00"
},
{
"id": 2,
"color": "cyan",
"department": "Clothing",
"name": "Atlas Shirt",
"price": "127.00"
},
{
"id": 3,
"color": "teal",
"department": "Clothing",
"name": "Small Metal Shoes",
"price": "232.00"
},
{
"id": 4,
"color": "red",
"department": "Watches",
"name": "Red Dragon Watch",
"price": "232.00"
}
]
2. 기존 프록시 메트릭 확인
1
kubectl exec -it deploy/webapp -c istio-proxy -n istioinaction -- curl localhost:15000/stats | grep catalog
✅ 출력
1
2
3
4
istiocustom.istio_requests_total.reporter.source.source_workload.webapp.source_canonical_service.webapp.source_canonical_revision.latest.source_workload_namespace.istioinaction.source_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.source_app.webapp.source_version.source_cluster.Kubernetes.destination_workload.catalog.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/catalog.destination_app.catalog.destination_version.v1.destination_service.catalog.istioinaction.svc.cluster.local.destination_canonical_service.catalog.destination_canonical_revision.v1.destination_service_name.catalog.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.unknown: 3
istiocustom.istio_request_bytes.reporter.source.source_workload.webapp.source_canonical_service.webapp.source_canonical_revision.latest.source_workload_namespace.istioinaction.source_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.source_app.webapp.source_version.source_cluster.Kubernetes.destination_workload.catalog.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/catalog.destination_app.catalog.destination_version.v1.destination_service.catalog.istioinaction.svc.cluster.local.destination_canonical_service.catalog.destination_canonical_revision.v1.destination_service_name.catalog.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.unknown: P0(nan,630) P25(nan,632.5) P50(nan,635) P75(nan,637.5) P90(nan,639) P95(nan,639.5) P99(nan,639.9) P99.5(nan,639.95) P99.9(nan,639.99) P100(nan,640)
istiocustom.istio_request_duration_milliseconds.reporter.source.source_workload.webapp.source_canonical_service.webapp.source_canonical_revision.latest.source_workload_namespace.istioinaction.source_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.source_app.webapp.source_version.source_cluster.Kubernetes.destination_workload.catalog.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/catalog.destination_app.catalog.destination_version.v1.destination_service.catalog.istioinaction.svc.cluster.local.destination_canonical_service.catalog.destination_canonical_revision.v1.destination_service_name.catalog.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.unknown: P0(nan,2) P25(nan,2.075) P50(nan,7.05) P75(nan,18.25) P90(nan,18.7) P95(nan,18.85) P99(nan,18.97) P99.5(nan,18.985) P99.9(nan,18.997) P100(nan,19)
istiocustom.istio_response_bytes.reporter.source.source_workload.webapp.source_canonical_service.webapp.source_canonical_revision.latest.source_workload_namespace.istioinaction.source_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.source_app.webapp.source_version.source_cluster.Kubernetes.destination_workload.catalog.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/catalog.destination_app.catalog.destination_version.v1.destination_service.catalog.istioinaction.svc.cluster.local.destination_canonical_service.catalog.destination_canonical_revision.v1.destination_service_name.catalog.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.unknown: P0(nan,860) P25(nan,862.5) P50(nan,865) P75(nan,867.5) P90(nan,869) P95(nan,869.5) P99(nan,869.9) P99.5(nan,869.95) P99.9(nan,869.99) P100(nan,870)
3. 설정 파일 확인 및 배포 - webapp-deployment-stats-inclusion.yaml
1
2
cat ch7/webapp-deployment-stats-inclusion.yaml
kubectl apply -n istioinaction -f ch7/webapp-deployment-stats-inclusion.yaml
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: webapp
name: webapp
spec:
replicas: 1
selector:
matchLabels:
app: webapp
template:
metadata:
annotations:
proxy.istio.io/config: |-
proxyStatsMatcher:
inclusionPrefixes:
- "cluster.outbound|80||catalog.istioinaction"
labels:
app: webapp
spec:
containers:
- env:
- name: KUBERNETES_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: CATALOG_SERVICE_HOST
value: catalog.istioinaction
- name: CATALOG_SERVICE_PORT
value: "80"
- name: FORUM_SERVICE_HOST
value: forum.istioinaction
- name: FORUM_SERVICE_PORT
value: "80"
image: istioinaction/webapp:latest
imagePullPolicy: IfNotPresent
name: webapp
ports:
- containerPort: 8080
name: http
protocol: TCP
securityContext:
privileged: false
deployment.apps/webapp configured
proxyStatsMatcher에 의한 메트릭 포함 조건 정의 확인
4. 설정 적용 후 API 호출테스트
1
curl -s http://webapp.istioinaction.io:30000/api/catalog | jq
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
[
{
"id": 1,
"color": "amber",
"department": "Eyewear",
"name": "Elinor Glasses",
"price": "282.00"
},
{
"id": 2,
"color": "cyan",
"department": "Clothing",
"name": "Atlas Shirt",
"price": "127.00"
},
{
"id": 3,
"color": "teal",
"department": "Clothing",
"name": "Small Metal Shoes",
"price": "232.00"
},
{
"id": 4,
"color": "red",
"department": "Watches",
"name": "Red Dragon Watch",
"price": "232.00"
}
]
5. 적용 후 확인 - catalog.istioinaction 에 대한 metrics 추가
(1) catalog 메트릭 확인
1
kubectl exec -it deploy/webapp -c istio-proxy -n istioinaction -- curl localhost:15000/stats | grep catalog
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.version_text: "2025-05-03T08:56:10Z/10"
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.assignment_stale: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.assignment_timeout_received: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.bind_errors: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.circuit_breakers.default.cx_open: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.circuit_breakers.default.cx_pool_open: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.circuit_breakers.default.remaining_cx: 4294967294
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.circuit_breakers.default.remaining_cx_pools: 18446744073709551614
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.circuit_breakers.default.remaining_pending: 4294967295
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.circuit_breakers.default.remaining_retries: 4294967295
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.circuit_breakers.default.remaining_rq: 4294967295
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.circuit_breakers.default.rq_open: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.circuit_breakers.default.rq_pending_open: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.circuit_breakers.default.rq_retry_open: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.circuit_breakers.high.cx_open: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.circuit_breakers.high.cx_pool_open: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.circuit_breakers.high.rq_open: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.circuit_breakers.high.rq_pending_open: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.circuit_breakers.high.rq_retry_open: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.client_ssl_socket_factory.downstream_context_secrets_not_ready: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.client_ssl_socket_factory.ssl_context_update_by_sds: 2
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.client_ssl_socket_factory.upstream_context_secrets_not_ready: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.default.total_match_count: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.http1.dropped_headers_with_underscores: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.http1.metadata_not_supported_error: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.http1.requests_rejected_with_underscores_in_headers: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.http1.response_flood: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.init_fetch_timeout: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.internal.upstream_rq_200: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.internal.upstream_rq_2xx: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.internal.upstream_rq_completed: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_healthy_panic: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_local_cluster_not_ok: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_recalculate_zone_structures: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_subsets_active: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_subsets_created: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_subsets_fallback: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_subsets_fallback_panic: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_subsets_removed: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_subsets_selected: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_zone_cluster_too_small: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_zone_no_capacity_left: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_zone_number_differs: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_zone_routing_all_directly: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_zone_routing_cross_zone: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_zone_routing_sampled: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.max_host_weight: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.membership_change: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.membership_degraded: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.membership_excluded: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.membership_healthy: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.membership_total: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.metadata_exchange.alpn_protocol_found: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.metadata_exchange.alpn_protocol_not_found: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.metadata_exchange.header_not_found: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.metadata_exchange.initial_header_not_found: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.metadata_exchange.metadata_added: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.original_dst_host_invalid: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.retry_or_shadow_abandoned: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.ciphers.TLS_AES_128_GCM_SHA256: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.connection_error: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.curves.X25519: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.fail_verify_cert_hash: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.fail_verify_error: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.fail_verify_no_cert: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.fail_verify_san: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.handshake: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.no_certificate: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.ocsp_staple_failed: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.ocsp_staple_omitted: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.ocsp_staple_requests: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.ocsp_staple_responses: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.session_reused: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.sigalgs.rsa_pss_rsae_sha256: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.versions.TLSv1.3: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.tlsMode-disabled.total_match_count: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.tlsMode-istio.total_match_count: 2
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.update_attempt: 3
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.update_empty: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.update_failure: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.update_no_rebuild: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.update_rejected: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.update_success: 2
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.update_time: 1746263640628
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_active: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_close_notify: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_connect_attempts_exceeded: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_connect_fail: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_connect_timeout: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_connect_with_0_rtt: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_destroy: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_destroy_local: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_destroy_local_with_active_rq: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_destroy_remote: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_destroy_remote_with_active_rq: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_destroy_with_active_rq: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_http1_total: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_http2_total: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_http3_total: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_idle_timeout: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_max_duration_reached: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_max_requests: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_none_healthy: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_overflow: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_pool_overflow: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_protocol_error: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_rx_bytes_buffered: 1709
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_rx_bytes_total: 1709
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_total: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_tx_bytes_buffered: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_tx_bytes_total: 1284
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_flow_control_backed_up_total: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_flow_control_drained_total: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_flow_control_paused_reading_total: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_flow_control_resumed_reading_total: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_http3_broken: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_internal_redirect_failed_total: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_internal_redirect_succeeded_total: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_0rtt: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_200: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_2xx: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_active: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_cancelled: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_completed: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_maintenance_mode: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_max_duration_reached: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_pending_active: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_pending_failure_eject: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_pending_overflow: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_pending_total: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_per_try_idle_timeout: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_per_try_timeout: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_retry: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_retry_backoff_exponential: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_retry_backoff_ratelimited: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_retry_limit_exceeded: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_retry_overflow: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_retry_success: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_rx_reset: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_timeout: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_total: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_tx_reset: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.version: 11248753202267649098
istiocustom.istio_requests_total.reporter.source.source_workload.webapp.source_canonical_service.webapp.source_canonical_revision.latest.source_workload_namespace.istioinaction.source_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.source_app.webapp.source_version.source_cluster.Kubernetes.destination_workload.catalog.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/catalog.destination_app.catalog.destination_version.v1.destination_service.catalog.istioinaction.svc.cluster.local.destination_canonical_service.catalog.destination_canonical_revision.v1.destination_service_name.catalog.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.unknown: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.internal.upstream_rq_time: P0(nan,8) P25(nan,8.025) P50(nan,8.05) P75(nan,8.075) P90(nan,8.09) P95(nan,8.095) P99(nan,8.099) P99.5(nan,8.0995) P99.9(nan,8.0999) P100(nan,8.1)
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.update_duration: P0(nan,0) P25(nan,0) P50(nan,0) P75(nan,0) P90(nan,0) P95(nan,0) P99(nan,0) P99.5(nan,0) P99.9(nan,0) P100(nan,0)
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_connect_ms: P0(nan,6) P25(nan,6.025) P50(nan,6.05) P75(nan,6.075) P90(nan,6.09) P95(nan,6.095) P99(nan,6.099) P99.5(nan,6.0995) P99.9(nan,6.0999) P100(nan,6.1)
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_length_ms: No recorded values
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_time: P0(nan,8) P25(nan,8.025) P50(nan,8.05) P75(nan,8.075) P90(nan,8.09) P95(nan,8.095) P99(nan,8.099) P99.5(nan,8.0995) P99.9(nan,8.0999) P100(nan,8.1)
istiocustom.istio_request_bytes.reporter.source.source_workload.webapp.source_canonical_service.webapp.source_canonical_revision.latest.source_workload_namespace.istioinaction.source_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.source_app.webapp.source_version.source_cluster.Kubernetes.destination_workload.catalog.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/catalog.destination_app.catalog.destination_version.v1.destination_service.catalog.istioinaction.svc.cluster.local.destination_canonical_service.catalog.destination_canonical_revision.v1.destination_service_name.catalog.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.unknown: P0(nan,630) P25(nan,632.5) P50(nan,635) P75(nan,637.5) P90(nan,639) P95(nan,639.5) P99(nan,639.9) P99.5(nan,639.95) P99.9(nan,639.99) P100(nan,640)
istiocustom.istio_request_duration_milliseconds.reporter.source.source_workload.webapp.source_canonical_service.webapp.source_canonical_revision.latest.source_workload_namespace.istioinaction.source_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.source_app.webapp.source_version.source_cluster.Kubernetes.destination_workload.catalog.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/catalog.destination_app.catalog.destination_version.v1.destination_service.catalog.istioinaction.svc.cluster.local.destination_canonical_service.catalog.destination_canonical_revision.v1.destination_service_name.catalog.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.unknown: P0(nan,11) P25(nan,11.25) P50(nan,11.5) P75(nan,11.75) P90(nan,11.9) P95(nan,11.95) P99(nan,11.99) P99.5(nan,11.995) P99.9(nan,11.999) P100(nan,12)
istiocustom.istio_response_bytes.reporter.source.source_workload.webapp.source_canonical_service.webapp.source_canonical_revision.latest.source_workload_namespace.istioinaction.source_principal.spiffe://cluster.local/ns/istioinaction/sa/webapp.source_app.webapp.source_version.source_cluster.Kubernetes.destination_workload.catalog.destination_workload_namespace.istioinaction.destination_principal.spiffe://cluster.local/ns/istioinaction/sa/catalog.destination_app.catalog.destination_version.v1.destination_service.catalog.istioinaction.svc.cluster.local.destination_canonical_service.catalog.destination_canonical_revision.v1.destination_service_name.catalog.destination_service_namespace.istioinaction.destination_cluster.Kubernetes.request_protocol.http.response_code.200.grpc_response_status.response_flags.-.connection_security_policy.unknown: P0(nan,860) P25(nan,862.5) P50(nan,865) P75(nan,867.5) P90(nan,869) P95(nan,869.5) P99(nan,869.9) P99.5(nan,869.95) P99.9(nan,869.99) P100(nan,870)
(2) 내부 요청 메트릭 확인
1
kubectl exec -it deploy/webapp -c istio-proxy -n istioinaction -- curl localhost:15000/stats | grep catalog | grep internal
✅ 출력
1
2
3
4
5
6
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.internal.upstream_rq_200: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.internal.upstream_rq_2xx: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.internal.upstream_rq_completed: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_internal_redirect_failed_total: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_internal_redirect_succeeded_total: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.internal.upstream_rq_time: P0(nan,8) P25(nan,8.025) P50(nan,8.05) P75(nan,8.075) P90(nan,8.09) P95(nan,8.095) P99(nan,8.099) P99.5(nan,8.0995) P99.9(nan,8.0999) P100(nan,8.1)
(3) TLS 관련 메트릭 확인
1
kubectl exec -it deploy/webapp -c istio-proxy -n istioinaction -- curl localhost:15000/stats | grep catalog | grep ssl
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.client_ssl_socket_factory.downstream_context_secrets_not_ready: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.client_ssl_socket_factory.ssl_context_update_by_sds: 2
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.client_ssl_socket_factory.upstream_context_secrets_not_ready: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.ciphers.TLS_AES_128_GCM_SHA256: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.connection_error: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.curves.X25519: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.fail_verify_cert_hash: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.fail_verify_error: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.fail_verify_no_cert: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.fail_verify_san: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.handshake: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.no_certificate: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.ocsp_staple_failed: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.ocsp_staple_omitted: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.ocsp_staple_requests: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.ocsp_staple_responses: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.session_reused: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.sigalgs.rsa_pss_rsae_sha256: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.ssl.versions.TLSv1.3: 1
(4) 업스트림 커넥션/요청 관련 메트릭 확인
1
kubectl exec -it deploy/webapp -c istio-proxy -n istioinaction -- curl localhost:15000/stats | grep catalog | egrep 'local.upstream_cx|local.upstream_rq'
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_active: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_close_notify: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_connect_attempts_exceeded: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_connect_fail: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_connect_timeout: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_connect_with_0_rtt: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_destroy: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_destroy_local: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_destroy_local_with_active_rq: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_destroy_remote: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_destroy_remote_with_active_rq: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_destroy_with_active_rq: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_http1_total: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_http2_total: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_http3_total: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_idle_timeout: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_max_duration_reached: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_max_requests: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_none_healthy: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_overflow: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_pool_overflow: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_protocol_error: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_rx_bytes_buffered: 1709
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_rx_bytes_total: 1709
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_total: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_tx_bytes_buffered: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_tx_bytes_total: 1284
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_0rtt: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_200: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_2xx: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_active: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_cancelled: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_completed: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_maintenance_mode: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_max_duration_reached: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_pending_active: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_pending_failure_eject: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_pending_overflow: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_pending_total: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_per_try_idle_timeout: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_per_try_timeout: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_retry: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_retry_backoff_exponential: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_retry_backoff_ratelimited: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_retry_limit_exceeded: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_retry_overflow: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_retry_success: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_rx_reset: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_timeout: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_total: 1
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_tx_reset: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_connect_ms: P0(nan,6) P25(nan,6.025) P50(nan,6.05) P75(nan,6.075) P90(nan,6.09) P95(nan,6.095) P99(nan,6.099) P99.5(nan,6.0995) P99.9(nan,6.0999) P100(nan,6.1)
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_cx_length_ms: No recorded values
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.upstream_rq_time: P0(nan,8) P25(nan,8.025) P50(nan,8.05) P75(nan,8.075) P90(nan,8.09) P95(nan,8.095) P99(nan,8.099) P99.5(nan,8.0995) P99.9(nan,8.0999) P100(nan,8.1)
(5) 로드밸런싱 관련 메트릭 확인
1
kubectl exec -it deploy/webapp -c istio-proxy -n istioinaction -- curl localhost:15000/stats | grep catalog | grep lb
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_healthy_panic: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_local_cluster_not_ok: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_recalculate_zone_structures: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_subsets_active: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_subsets_created: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_subsets_fallback: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_subsets_fallback_panic: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_subsets_removed: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_subsets_selected: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_zone_cluster_too_small: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_zone_no_capacity_left: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_zone_number_differs: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_zone_routing_all_directly: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_zone_routing_cross_zone: 0
cluster.outbound|80||catalog.istioinaction.svc.cluster.local.lb_zone_routing_sampled: 0
(6) 클러스터 정보 및 엔드포인트 상태 확인
1
kubectl exec -it deploy/webapp -c istio-proxy -n istioinaction -- curl localhost:15000/clusters | grep catalog
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
outbound|80||catalog.istioinaction.svc.cluster.local::observability_name::outbound|80||catalog.istioinaction.svc.cluster.local
outbound|80||catalog.istioinaction.svc.cluster.local::default_priority::max_connections::4294967295
outbound|80||catalog.istioinaction.svc.cluster.local::default_priority::max_pending_requests::4294967295
outbound|80||catalog.istioinaction.svc.cluster.local::default_priority::max_requests::4294967295
outbound|80||catalog.istioinaction.svc.cluster.local::default_priority::max_retries::4294967295
outbound|80||catalog.istioinaction.svc.cluster.local::high_priority::max_connections::1024
outbound|80||catalog.istioinaction.svc.cluster.local::high_priority::max_pending_requests::1024
outbound|80||catalog.istioinaction.svc.cluster.local::high_priority::max_requests::1024
outbound|80||catalog.istioinaction.svc.cluster.local::high_priority::max_retries::3
outbound|80||catalog.istioinaction.svc.cluster.local::added_via_api::true
outbound|80||catalog.istioinaction.svc.cluster.local::eds_service_name::outbound|80||catalog.istioinaction.svc.cluster.local
outbound|80||catalog.istioinaction.svc.cluster.local::10.10.0.9:3000::cx_active::1
outbound|80||catalog.istioinaction.svc.cluster.local::10.10.0.9:3000::cx_connect_fail::0
outbound|80||catalog.istioinaction.svc.cluster.local::10.10.0.9:3000::cx_total::1
outbound|80||catalog.istioinaction.svc.cluster.local::10.10.0.9:3000::rq_active::0
outbound|80||catalog.istioinaction.svc.cluster.local::10.10.0.9:3000::rq_error::0
outbound|80||catalog.istioinaction.svc.cluster.local::10.10.0.9:3000::rq_success::1
outbound|80||catalog.istioinaction.svc.cluster.local::10.10.0.9:3000::rq_timeout::0
outbound|80||catalog.istioinaction.svc.cluster.local::10.10.0.9:3000::rq_total::1
outbound|80||catalog.istioinaction.svc.cluster.local::10.10.0.9:3000::hostname::
outbound|80||catalog.istioinaction.svc.cluster.local::10.10.0.9:3000::health_flags::healthy
outbound|80||catalog.istioinaction.svc.cluster.local::10.10.0.9:3000::weight::1
outbound|80||catalog.istioinaction.svc.cluster.local::10.10.0.9:3000::region::
outbound|80||catalog.istioinaction.svc.cluster.local::10.10.0.9:3000::zone::
outbound|80||catalog.istioinaction.svc.cluster.local::10.10.0.9:3000::sub_zone::
outbound|80||catalog.istioinaction.svc.cluster.local::10.10.0.9:3000::canary::false
outbound|80||catalog.istioinaction.svc.cluster.local::10.10.0.9:3000::priority::0
outbound|80||catalog.istioinaction.svc.cluster.local::10.10.0.9:3000::success_rate::-1
outbound|80||catalog.istioinaction.svc.cluster.local::10.10.0.9:3000::local_origin_success_rate::-1
📡 컨트롤 플레인의 메트릭 - citadel, pilot, xds 메트릭 확인
1. istiod 포트 리스닝 상태 확인
1
kubectl exec -it deploy/istiod -n istio-system -- netstat -tnl
✅ 출력
1
2
3
4
5
6
7
8
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 127.0.0.1:9876 0.0.0.0:* LISTEN
tcp6 0 0 :::15014 :::* LISTEN
tcp6 0 0 :::15012 :::* LISTEN
tcp6 0 0 :::15010 :::* LISTEN
tcp6 0 0 :::15017 :::* LISTEN
tcp6 0 0 :::8080 :::* LISTEN
2. citadel 메트릭 확인
1
kubectl exec -it -n istio-system deploy/istiod -n istio-system -- curl localhost:15014/metrics | grep citadel
✅ 출력
1
2
3
4
5
6
7
8
9
# HELP citadel_server_csr_count The number of CSRs received by Citadel server.
# TYPE citadel_server_csr_count counter
citadel_server_csr_count 4
# HELP citadel_server_root_cert_expiry_timestamp The unix timestamp, in seconds, when Citadel root cert will expire. A negative time indicates the cert is expired.
# TYPE citadel_server_root_cert_expiry_timestamp gauge
citadel_server_root_cert_expiry_timestamp 2.061605423e+09
# HELP citadel_server_success_cert_issuance_count The number of certificates issuances that have succeeded.
# TYPE citadel_server_success_cert_issuance_count counter
citadel_server_success_cert_issuance_count 4
3. pilot 프록시 수렴 시간 메트릭 확인
1
kubectl exec -it -n istio-system deploy/istiod -n istio-system -- curl localhost:15014/metrics | grep convergence
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
# HELP pilot_proxy_convergence_time Delay in seconds between config change and a proxy receiving all required configuration.
# TYPE pilot_proxy_convergence_time histogram
pilot_proxy_convergence_time_bucket{le="0.1"} 26 **#** 0.1초 내에 26개의 업데이트가 프록시에 배포됐다
pilot_proxy_convergence_time_bucket{le="0.5"} 26
pilot_proxy_convergence_time_bucket{le="1"} 26
pilot_proxy_convergence_time_bucket{le="3"} 26
pilot_proxy_convergence_time_bucket{le="5"} 26
pilot_proxy_convergence_time_bucket{le="10"} 26
pilot_proxy_convergence_time_bucket{le="20"} 26
pilot_proxy_convergence_time_bucket{le="30"} 26
pilot_proxy_convergence_time_bucket{le="+Inf"} 26
pilot_proxy_convergence_time_sum 0.019110517
pilot_proxy_convergence_time_count 26
4. pilot 리소스 상태 및 서비스 수량 확인
1
kubectl exec -it -n istio-system deploy/istiod -n istio-system -- curl localhost:15014/metrics | grep pilot | egrep 'service|^pilot_xds'
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
# HELP pilot_duplicate_envoy_clusters Duplicate envoy clusters caused by service entries with same hostname
pilot_push_triggers{type="service"} 3
# HELP pilot_services Total services known to pilot.
# TYPE pilot_services gauge
pilot_services 7
# HELP pilot_virt_services Total virtual services known to pilot.
# TYPE pilot_virt_services gauge
pilot_virt_services 1
# HELP pilot_vservice_dup_domain Virtual services with dup domains.
# TYPE pilot_vservice_dup_domain gauge
pilot_vservice_dup_domain 0
pilot_xds{version="1.17.8"} 3
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.cluster.v3.Cluster",le="1"} 0
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.cluster.v3.Cluster",le="10000"} 0
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.cluster.v3.Cluster",le="1e+06"} 29
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.cluster.v3.Cluster",le="4e+06"} 29
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.cluster.v3.Cluster",le="1e+07"} 29
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.cluster.v3.Cluster",le="4e+07"} 29
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.cluster.v3.Cluster",le="+Inf"} 29
pilot_xds_config_size_bytes_sum{type="type.googleapis.com/envoy.config.cluster.v3.Cluster"} 375781.00000000006
pilot_xds_config_size_bytes_count{type="type.googleapis.com/envoy.config.cluster.v3.Cluster"} 29
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment",le="1"} 0
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment",le="10000"} 40
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment",le="1e+06"} 40
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment",le="4e+06"} 40
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment",le="1e+07"} 40
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment",le="4e+07"} 40
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment",le="+Inf"} 40
pilot_xds_config_size_bytes_sum{type="type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment"} 44570.99999999999
pilot_xds_config_size_bytes_count{type="type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment"} 40
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.listener.v3.Listener",le="1"} 15
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.listener.v3.Listener",le="10000"} 19
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.listener.v3.Listener",le="1e+06"} 29
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.listener.v3.Listener",le="4e+06"} 29
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.listener.v3.Listener",le="1e+07"} 29
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.listener.v3.Listener",le="4e+07"} 29
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.listener.v3.Listener",le="+Inf"} 29
pilot_xds_config_size_bytes_sum{type="type.googleapis.com/envoy.config.listener.v3.Listener"} 320916.00000000006
pilot_xds_config_size_bytes_count{type="type.googleapis.com/envoy.config.listener.v3.Listener"} 29
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.route.v3.RouteConfiguration",le="1"} 0
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.route.v3.RouteConfiguration",le="10000"} 14
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.route.v3.RouteConfiguration",le="1e+06"} 14
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.route.v3.RouteConfiguration",le="4e+06"} 14
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.route.v3.RouteConfiguration",le="1e+07"} 14
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.route.v3.RouteConfiguration",le="4e+07"} 14
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.route.v3.RouteConfiguration",le="+Inf"} 14
pilot_xds_config_size_bytes_sum{type="type.googleapis.com/envoy.config.route.v3.RouteConfiguration"} 46877.99999999999
pilot_xds_config_size_bytes_count{type="type.googleapis.com/envoy.config.route.v3.RouteConfiguration"} 14
pilot_xds_expired_nonce{type="eds"} 6
pilot_xds_push_time_bucket{type="cds",le="0.01"} 29
pilot_xds_push_time_bucket{type="cds",le="0.1"} 29
pilot_xds_push_time_bucket{type="cds",le="1"} 29
pilot_xds_push_time_bucket{type="cds",le="3"} 29
pilot_xds_push_time_bucket{type="cds",le="5"} 29
pilot_xds_push_time_bucket{type="cds",le="10"} 29
pilot_xds_push_time_bucket{type="cds",le="20"} 29
pilot_xds_push_time_bucket{type="cds",le="30"} 29
pilot_xds_push_time_bucket{type="cds",le="+Inf"} 29
pilot_xds_push_time_sum{type="cds"} 0.009880015999999998
pilot_xds_push_time_count{type="cds"} 29
pilot_xds_push_time_bucket{type="eds",le="0.01"} 40
pilot_xds_push_time_bucket{type="eds",le="0.1"} 40
pilot_xds_push_time_bucket{type="eds",le="1"} 40
pilot_xds_push_time_bucket{type="eds",le="3"} 40
pilot_xds_push_time_bucket{type="eds",le="5"} 40
pilot_xds_push_time_bucket{type="eds",le="10"} 40
pilot_xds_push_time_bucket{type="eds",le="20"} 40
pilot_xds_push_time_bucket{type="eds",le="30"} 40
pilot_xds_push_time_bucket{type="eds",le="+Inf"} 40
pilot_xds_push_time_sum{type="eds"} 0.005206036000000001
pilot_xds_push_time_count{type="eds"} 40
pilot_xds_push_time_bucket{type="lds",le="0.01"} 29
pilot_xds_push_time_bucket{type="lds",le="0.1"} 29
pilot_xds_push_time_bucket{type="lds",le="1"} 29
pilot_xds_push_time_bucket{type="lds",le="3"} 29
pilot_xds_push_time_bucket{type="lds",le="5"} 29
pilot_xds_push_time_bucket{type="lds",le="10"} 29
pilot_xds_push_time_bucket{type="lds",le="20"} 29
pilot_xds_push_time_bucket{type="lds",le="30"} 29
pilot_xds_push_time_bucket{type="lds",le="+Inf"} 29
pilot_xds_push_time_sum{type="lds"} 0.012618381
pilot_xds_push_time_count{type="lds"} 29
pilot_xds_push_time_bucket{type="rds",le="0.01"} 14
pilot_xds_push_time_bucket{type="rds",le="0.1"} 14
pilot_xds_push_time_bucket{type="rds",le="1"} 14
pilot_xds_push_time_bucket{type="rds",le="3"} 14
pilot_xds_push_time_bucket{type="rds",le="5"} 14
pilot_xds_push_time_bucket{type="rds",le="10"} 14
pilot_xds_push_time_bucket{type="rds",le="20"} 14
pilot_xds_push_time_bucket{type="rds",le="30"} 14
pilot_xds_push_time_bucket{type="rds",le="+Inf"} 14
pilot_xds_push_time_sum{type="rds"} 0.0031356699999999993
pilot_xds_push_time_count{type="rds"} 14
pilot_xds_pushes{type="cds"} 29
pilot_xds_pushes{type="eds"} 40
pilot_xds_pushes{type="lds"} 29
pilot_xds_pushes{type="rds"} 14
pilot_xds_send_time_bucket{le="0.01"} 112
pilot_xds_send_time_bucket{le="0.1"} 112
pilot_xds_send_time_bucket{le="1"} 112
pilot_xds_send_time_bucket{le="3"} 112
pilot_xds_send_time_bucket{le="5"} 112
pilot_xds_send_time_bucket{le="10"} 112
pilot_xds_send_time_bucket{le="20"} 112
pilot_xds_send_time_bucket{le="30"} 112
pilot_xds_send_time_bucket{le="+Inf"} 112
pilot_xds_send_time_sum 0.001285499999999999
pilot_xds_send_time_count 112
5. 전체 네임스페이스의 서비스 리소스 확인
1
kubectl get svc -A -oname
✅ 출력
1
2
3
4
5
6
7
service/kubernetes
service/istio-ingressgateway
service/istiod
service/catalog
service/webapp
service/kube-dns
service/kube-ops-view
6. 전체 네임스페이스의 VirtualService 리소스 확인
1
kubectl get vs -A -oname
✅ 출력
1
virtualservice.networking.istio.io/webapp-virtualservice
7. 프록시 동기화 상태 확인
1
docker exec -it myk8s-control-plane istioctl proxy-status
✅ 출력
1
2
3
4
NAME CLUSTER CDS LDS EDS RDS ECDS ISTIOD VERSION
catalog-6cf4b97d-5kkz8.istioinaction Kubernetes SYNCED SYNCED SYNCED SYNCED NOT SENT istiod-7df6ffc78d-t5kbh 1.17.8
istio-ingressgateway-996bc6bb6-crh8b.istio-system Kubernetes SYNCED SYNCED SYNCED SYNCED NOT SENT istiod-7df6ffc78d-t5kbh 1.17.8
webapp-8578f44594-pj6pg.istioinaction Kubernetes SYNCED SYNCED SYNCED SYNCED NOT SENT istiod-7df6ffc78d-t5kbh 1.17.8
📥 Scraping Istio metrics with Prometheus
1. webapp 파드 정보 확인
1
kubectl exec -it deploy/webapp -n istioinaction -c istio-proxy -- netstat -tnl
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:15090 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:15090 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:15006 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:15006 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:15001 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:15001 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:15021 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:15021 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:15000 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:15004 0.0.0.0:* LISTEN
tcp6 0 0 :::8080 :::* LISTEN
tcp6 0 0 :::15020 :::* LISTEN
2. 사이드카 프록시 메트릭 엔드포인트 확인
1
kubectl exec -it deploy/webapp -c istio-proxy -n istioinaction -- curl localhost:15020/metrics
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
...
istio_response_bytes_bucket{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",le="+Inf"} 1
istio_response_bytes_sum{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown"} 865
istio_response_bytes_count{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown"} 1
# TYPE envoy_listener_manager_lds_update_duration histogram
envoy_listener_manager_lds_update_duration_bucket{le="0.5"} 0
envoy_listener_manager_lds_update_duration_bucket{le="1"} 0
envoy_listener_manager_lds_update_duration_bucket{le="5"} 1
envoy_listener_manager_lds_update_duration_bucket{le="10"} 1
envoy_listener_manager_lds_update_duration_bucket{le="25"} 2
envoy_listener_manager_lds_update_duration_bucket{le="50"} 2
envoy_listener_manager_lds_update_duration_bucket{le="100"} 2
envoy_listener_manager_lds_update_duration_bucket{le="250"} 2
envoy_listener_manager_lds_update_duration_bucket{le="500"} 2
envoy_listener_manager_lds_update_duration_bucket{le="1000"} 2
envoy_listener_manager_lds_update_duration_bucket{le="2500"} 2
envoy_listener_manager_lds_update_duration_bucket{le="5000"} 2
envoy_listener_manager_lds_update_duration_bucket{le="10000"} 2
envoy_listener_manager_lds_update_duration_bucket{le="30000"} 2
envoy_listener_manager_lds_update_duration_bucket{le="60000"} 2
envoy_listener_manager_lds_update_duration_bucket{le="300000"} 2
envoy_listener_manager_lds_update_duration_bucket{le="600000"} 2
envoy_listener_manager_lds_update_duration_bucket{le="1800000"} 2
envoy_listener_manager_lds_update_duration_bucket{le="3600000"} 2
envoy_listener_manager_lds_update_duration_bucket{le="+Inf"} 2
envoy_listener_manager_lds_update_duration_sum{} 14.5500000000000007105427357601
envoy_listener_manager_lds_update_duration_count{} 2
# TYPE envoy_server_initialization_time_ms histogram
envoy_server_initialization_time_ms_bucket{le="0.5"} 0
envoy_server_initialization_time_ms_bucket{le="1"} 0
envoy_server_initialization_time_ms_bucket{le="5"} 0
envoy_server_initialization_time_ms_bucket{le="10"} 0
envoy_server_initialization_time_ms_bucket{le="25"} 0
envoy_server_initialization_time_ms_bucket{le="50"} 0
envoy_server_initialization_time_ms_bucket{le="100"} 1
envoy_server_initialization_time_ms_bucket{le="250"} 1
envoy_server_initialization_time_ms_bucket{le="500"} 1
envoy_server_initialization_time_ms_bucket{le="1000"} 1
envoy_server_initialization_time_ms_bucket{le="2500"} 1
envoy_server_initialization_time_ms_bucket{le="5000"} 1
envoy_server_initialization_time_ms_bucket{le="10000"} 1
envoy_server_initialization_time_ms_bucket{le="30000"} 1
envoy_server_initialization_time_ms_bucket{le="60000"} 1
envoy_server_initialization_time_ms_bucket{le="300000"} 1
envoy_server_initialization_time_ms_bucket{le="600000"} 1
envoy_server_initialization_time_ms_bucket{le="1800000"} 1
envoy_server_initialization_time_ms_bucket{le="3600000"} 1
envoy_server_initialization_time_ms_bucket{le="+Inf"} 1
envoy_server_initialization_time_ms_sum{} 89.5
envoy_server_initialization_time_ms_count{} 1
⚙️ 프로메테우스, 그라파나 설정
1. Helm 리포지토리 추가 및 업데이트
1
2
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
"prometheus-community" already exists with the same configuration, skipping
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "metrics-server" chart repository
...Successfully got an update from the "kedacore" chart repository
...Successfully got an update from the "eks" chart repository
...Successfully got an update from the "hashicorp" chart repository
...Successfully got an update from the "flagger" chart repository
...Successfully got an update from the "argo" chart repository
...Successfully got an update from the "grafana-charts" chart repository
...Successfully got an update from the "prometheus-community" chart repository
...Successfully got an update from the "geek-cookbook" chart repository
...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈
2. Prometheus 및 Grafana 설정값 정의
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
cat << EOF > prom-values-2.yaml
prometheusOperator:
tls:
enabled: false
admissionWebhooks:
patch:
enabled: false
prometheus:
service:
type: NodePort
nodePort: 30001
grafana:
service:
type: NodePort
nodePort: 30002
EOF
3. Prometheus 네임스페이스 생성 및 Helm 차트 설치
1
2
3
kubectl create ns prometheus
helm install prom prometheus-community/kube-prometheus-stack --version 13.13.1 \
-n prometheus -f ch7/prom-values.yaml -f prom-values-2.yaml
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
namespace/prometheus created
NAME: prom
LAST DEPLOYED: Sat May 3 18:59:23 2025
NAMESPACE: prometheus
STATUS: deployed
REVISION: 1
NOTES:
kube-prometheus-stack has been installed. Check its status by running:
kubectl --namespace prometheus get pods -l "release=prom"
Visit https://github.com/prometheus-operator/kube-prometheus for instructions on how to create & configure Alertmanager and Prometheus instances using the Operator.
4. 설치된 Prometheus/Grafana 구성 요소 조회
1
kubectl get sts,deploy,pod,svc,ep,cm,secret -n prometheus
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
NAME READY AGE
statefulset.apps/prometheus-prom-kube-prometheus-stack-prometheus 0/1 3s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/prom-grafana 0/1 1 0 37s
deployment.apps/prom-kube-prometheus-stack-operator 1/1 1 1 37s
NAME READY STATUS RESTARTS AGE
pod/prom-grafana-d7f5cb646-bvnh8 2/2 Running 0 37s
pod/prom-kube-prometheus-stack-operator-6f4f9f9d49-mzp4c 1/1 Running 0 37s
pod/prometheus-prom-kube-prometheus-stack-prometheus-0 2/2 Running 0 3s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/prom-grafana NodePort 10.200.1.123 <none> 80:30002/TCP 37s
service/prom-kube-prometheus-stack-operator ClusterIP 10.200.1.161 <none> 8080/TCP 37s
service/prom-kube-prometheus-stack-prometheus NodePort 10.200.1.129 <none> 9090:30001/TCP 37s
service/prometheus-operated ClusterIP None <none> 9090/TCP 3s
NAME ENDPOINTS AGE
endpoints/prom-grafana 37s
endpoints/prom-kube-prometheus-stack-operator 10.10.0.13:8080 37s
endpoints/prom-kube-prometheus-stack-prometheus <none> 37s
endpoints/prometheus-operated <none> 3s
NAME DATA AGE
configmap/istio-ca-root-cert 1 41s
configmap/kube-root-ca.crt 1 41s
configmap/prom-grafana 1 37s
configmap/prom-grafana-config-dashboards 1 37s
configmap/prom-grafana-test 1 37s
configmap/prom-kube-prometheus-stack-cluster-total 1 37s
configmap/prom-kube-prometheus-stack-controller-manager 1 37s
configmap/prom-kube-prometheus-stack-grafana-datasource 1 37s
configmap/prom-kube-prometheus-stack-k8s-resources-cluster 1 37s
configmap/prom-kube-prometheus-stack-k8s-resources-namespace 1 37s
configmap/prom-kube-prometheus-stack-k8s-resources-node 1 37s
configmap/prom-kube-prometheus-stack-k8s-resources-pod 1 37s
configmap/prom-kube-prometheus-stack-k8s-resources-workload 1 37s
configmap/prom-kube-prometheus-stack-k8s-resources-workloads-namespace 1 37s
configmap/prom-kube-prometheus-stack-namespace-by-pod 1 37s
configmap/prom-kube-prometheus-stack-namespace-by-workload 1 37s
configmap/prom-kube-prometheus-stack-nodes 1 37s
configmap/prom-kube-prometheus-stack-persistentvolumesusage 1 37s
configmap/prom-kube-prometheus-stack-pod-total 1 37s
configmap/prom-kube-prometheus-stack-prometheus 1 37s
configmap/prom-kube-prometheus-stack-statefulset 1 37s
configmap/prom-kube-prometheus-stack-workload-total 1 37s
configmap/prometheus-prom-kube-prometheus-stack-prometheus-rulefiles-0 0 3s
NAME TYPE DATA AGE
secret/default-token-rv64d kubernetes.io/service-account-token 3 41s
secret/prom-grafana Opaque 3 37s
secret/prom-grafana-test-token-zz9lg kubernetes.io/service-account-token 3 37s
secret/prom-grafana-token-z74gb kubernetes.io/service-account-token 3 37s
secret/prom-kube-prometheus-stack-operator-token-2hpzw kubernetes.io/service-account-token 3 37s
secret/prom-kube-prometheus-stack-prometheus-token-7xq6t kubernetes.io/service-account-token 3 37s
secret/prometheus-prom-kube-prometheus-stack-prometheus Opaque 1 3s
secret/prometheus-prom-kube-prometheus-stack-prometheus-tls-assets Opaque 0 3s
secret/sh.helm.release.v1.prom.v1 helm.sh/release.v1 1 37s
5. Monitoring 관련 CRD 확인
1
kubectl get crd | grep monitoring
✅ 출력
1
2
3
4
5
6
7
8
alertmanagerconfigs.monitoring.coreos.com 2025-05-03T09:43:37Z
alertmanagers.monitoring.coreos.com 2025-05-03T09:43:37Z
podmonitors.monitoring.coreos.com 2025-05-03T09:43:37Z
probes.monitoring.coreos.com 2025-05-03T09:43:37Z
prometheuses.monitoring.coreos.com 2025-05-03T09:43:37Z
prometheusrules.monitoring.coreos.com 2025-05-03T09:43:37Z
servicemonitors.monitoring.coreos.com 2025-05-03T09:43:37Z
thanosrulers.monitoring.coreos.com 2025-05-03T09:43:37Z
6. Prometheus 웹 UI 접속
http://127.0.0.1:30001/ 
7. 설치된 ServiceMonitor 리소스 확인
1
kubectl get servicemonitors -n prometheus
✅ 출력
1
2
3
4
5
NAME AGE
prom-kube-prometheus-stack-grafana 2m45s
prom-kube-prometheus-stack-kube-controller-manager 2m45s
prom-kube-prometheus-stack-operator 2m45s
prom-kube-prometheus-stack-prometheus 2m45s
8. Prometheus 및 ServiceMonitor 상세 조회
1
kubectl get prometheus,servicemonitors -n prometheus
✅ 출력
1
2
3
4
5
6
7
8
NAME VERSION REPLICAS AGE
prometheus.monitoring.coreos.com/prom-kube-prometheus-stack-prometheus v2.24.0 1 3m58s
NAME AGE
servicemonitor.monitoring.coreos.com/prom-kube-prometheus-stack-grafana 3m58s
servicemonitor.monitoring.coreos.com/prom-kube-prometheus-stack-kube-controller-manager 3m58s
servicemonitor.monitoring.coreos.com/prom-kube-prometheus-stack-operator 3m58s
servicemonitor.monitoring.coreos.com/prom-kube-prometheus-stack-prometheus 3m58s
9. Prometheus 버전 확인
1
kubectl exec -it sts/prometheus-prom-kube-prometheus-stack-prometheus -n prometheus -c prometheus -- prometheus --version
✅ 출력
1
2
3
4
5
prometheus, version 2.24.0 (branch: HEAD, revision: 02e92236a8bad3503ff5eec3e04ac205a3b8e4fe)
build user: root@d9f90f0b1f76
build date: 20210106-13:48:37
go version: go1.15.6
platform: linux/amd64
10. Grafana 웹 UI 접속
http://127.0.0.1:30002/ 
🎛️ Configuring the Prometheus Operator to scrape the Istio control plane and workloads
1. Istiod 서비스 세부 정보 조회
1
kubectl describe svc istiod -n istio-system
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
Name: istiod
Namespace: istio-system
Labels: app=istiod
install.operator.istio.io/owning-resource=unknown
install.operator.istio.io/owning-resource-namespace=istio-system
istio=pilot
istio.io/rev=default
operator.istio.io/component=Pilot
operator.istio.io/managed=Reconcile
operator.istio.io/version=1.17.8
release=istio
Annotations: <none>
Selector: app=istiod,istio=pilot
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.200.1.155
IPs: 10.200.1.155
Port: grpc-xds 15010/TCP
TargetPort: 15010/TCP
Endpoints: 10.10.0.6:15010
Port: https-dns 15012/TCP
TargetPort: 15012/TCP
Endpoints: 10.10.0.6:15012
Port: https-webhook 443/TCP
TargetPort: 15017/TCP
Endpoints: 10.10.0.6:15017
Port: http-monitoring 15014/TCP
TargetPort: 15014/TCP
Endpoints: 10.10.0.6:15014
Session Affinity: None
Internal Traffic Policy: Cluster
Events: <none>
2. Istiod 파드 상태 확인
1
kubectl get pod -n istio-system -l istio=pilot
✅ 출력
1
2
NAME READY STATUS RESTARTS AGE
istiod-7df6ffc78d-t5kbh 1/1 Running 0 6h28m
3. Istio Control Plane용 ServiceMonitor 리소스 생성
1
2
3
4
kubectl apply -f ch7/service-monitor-cp.yaml -n prometheus
# 결과
servicemonitor.monitoring.coreos.com/istio-component-monitor created
4. ServiceMonitor 리소스 목록 확인
1
kubectl get servicemonitor -n prometheus
✅ 출력
1
2
3
4
5
6
NAME AGE
istio-component-monitor 20s
prom-kube-prometheus-stack-grafana 40m
prom-kube-prometheus-stack-kube-controller-manager 40m
prom-kube-prometheus-stack-operator 40m
prom-kube-prometheus-stack-prometheus 40m
5. Istiod 서비스 및 엔드포인트 확인
1
kubectl get svc,ep istiod -n istio-system
✅ 출력
1
2
3
4
5
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/istiod ClusterIP 10.200.1.155 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP 6h30m
NAME ENDPOINTS AGE
endpoints/istiod 10.10.0.6:15012,10.10.0.6:15010,10.10.0.6:15017 + 1 more... 6h30m
6. Istiod 메트릭 엔드포인트 직접 조회
1
kubectl exec -it netshoot -- curl -s istiod.istio-system:15014/metrics
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
# HELP citadel_server_csr_count The number of CSRs received by Citadel server.
# TYPE citadel_server_csr_count counter
citadel_server_csr_count 4
# HELP citadel_server_root_cert_expiry_timestamp The unix timestamp, in seconds, when Citadel root cert will expire. A negative time indicates the cert is expired.
# TYPE citadel_server_root_cert_expiry_timestamp gauge
citadel_server_root_cert_expiry_timestamp 2.061605423e+09
# HELP citadel_server_success_cert_issuance_count The number of certificates issuances that have succeeded.
# TYPE citadel_server_success_cert_issuance_count counter
citadel_server_success_cert_issuance_count 4
# HELP endpoint_no_pod Endpoints without an associated pod.
# TYPE endpoint_no_pod gauge
endpoint_no_pod 0
# HELP galley_validation_config_updates k8s webhook configuration updates
# TYPE galley_validation_config_updates counter
galley_validation_config_updates 3
# HELP galley_validation_failed Resource validation failed
# TYPE galley_validation_failed counter
galley_validation_failed{group="networking.istio.io",reason="invalid_resource",resource="gateways",version="v1alpha3"} 1
# HELP galley_validation_passed Resource is valid
# TYPE galley_validation_passed counter
galley_validation_passed{group="networking.istio.io",resource="gateways",version="v1alpha3"} 1
galley_validation_passed{group="networking.istio.io",resource="virtualservices",version="v1alpha3"} 1
# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 2.3637e-05
go_gc_duration_seconds{quantile="0.25"} 6.0486e-05
go_gc_duration_seconds{quantile="0.5"} 8.388e-05
go_gc_duration_seconds{quantile="0.75"} 0.000119786
go_gc_duration_seconds{quantile="1"} 0.000228043
go_gc_duration_seconds_sum 0.019207779
go_gc_duration_seconds_count 205
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 396
# HELP go_info Information about the Go environment.
# TYPE go_info gauge
go_info{version="go1.20.10"} 1
# HELP go_memstats_alloc_bytes Number of bytes allocated and still in use.
# TYPE go_memstats_alloc_bytes gauge
go_memstats_alloc_bytes 2.6236696e+07
# HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed.
# TYPE go_memstats_alloc_bytes_total counter
go_memstats_alloc_bytes_total 8.8673392e+08
# HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table.
# TYPE go_memstats_buck_hash_sys_bytes gauge
go_memstats_buck_hash_sys_bytes 1.764099e+06
# HELP go_memstats_frees_total Total number of frees.
# TYPE go_memstats_frees_total counter
go_memstats_frees_total 1.0896008e+07
# HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata.
# TYPE go_memstats_gc_sys_bytes gauge
go_memstats_gc_sys_bytes 9.536768e+06
# HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use.
# TYPE go_memstats_heap_alloc_bytes gauge
go_memstats_heap_alloc_bytes 2.6236696e+07
# HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used.
# TYPE go_memstats_heap_idle_bytes gauge
go_memstats_heap_idle_bytes 1.8538496e+07
# HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use.
# TYPE go_memstats_heap_inuse_bytes gauge
go_memstats_heap_inuse_bytes 3.2776192e+07
# HELP go_memstats_heap_objects Number of allocated objects.
# TYPE go_memstats_heap_objects gauge
go_memstats_heap_objects 142661
# HELP go_memstats_heap_released_bytes Number of heap bytes released to OS.
# TYPE go_memstats_heap_released_bytes gauge
go_memstats_heap_released_bytes 1.6113664e+07
# HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system.
# TYPE go_memstats_heap_sys_bytes gauge
go_memstats_heap_sys_bytes 5.1314688e+07
# HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection.
# TYPE go_memstats_last_gc_time_seconds gauge
go_memstats_last_gc_time_seconds 1.7462689286184554e+09
# HELP go_memstats_lookups_total Total number of pointer lookups.
# TYPE go_memstats_lookups_total counter
go_memstats_lookups_total 0
# HELP go_memstats_mallocs_total Total number of mallocs.
# TYPE go_memstats_mallocs_total counter
go_memstats_mallocs_total 1.1038669e+07
# HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures.
# TYPE go_memstats_mcache_inuse_bytes gauge
go_memstats_mcache_inuse_bytes 21600
# HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system.
# TYPE go_memstats_mcache_sys_bytes gauge
go_memstats_mcache_sys_bytes 31200
# HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures.
# TYPE go_memstats_mspan_inuse_bytes gauge
go_memstats_mspan_inuse_bytes 528960
# HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system.
# TYPE go_memstats_mspan_sys_bytes gauge
go_memstats_mspan_sys_bytes 669120
# HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place.
# TYPE go_memstats_next_gc_bytes gauge
go_memstats_next_gc_bytes 4.6391008e+07
# HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations.
# TYPE go_memstats_other_sys_bytes gauge
go_memstats_other_sys_bytes 3.396965e+06
# HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator.
# TYPE go_memstats_stack_inuse_bytes gauge
go_memstats_stack_inuse_bytes 3.211264e+06
# HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator.
# TYPE go_memstats_stack_sys_bytes gauge
go_memstats_stack_sys_bytes 3.211264e+06
# HELP go_memstats_sys_bytes Number of bytes obtained from system.
# TYPE go_memstats_sys_bytes gauge
go_memstats_sys_bytes 6.9924104e+07
# HELP go_threads Number of OS threads created.
# TYPE go_threads gauge
go_threads 21
# HELP grpc_server_handled_total Total number of RPCs completed on the server, regardless of success or failure.
# TYPE grpc_server_handled_total counter
grpc_server_handled_total{grpc_code="OK",grpc_method="CreateCertificate",grpc_service="istio.v1.auth.IstioCertificateService",grpc_type="unary"} 4
# HELP grpc_server_handling_seconds Histogram of response latency (seconds) of gRPC that had been application-level handled by the server.
# TYPE grpc_server_handling_seconds histogram
grpc_server_handling_seconds_bucket{grpc_method="CreateCertificate",grpc_service="istio.v1.auth.IstioCertificateService",grpc_type="unary",le="0.005"} 0
grpc_server_handling_seconds_bucket{grpc_method="CreateCertificate",grpc_service="istio.v1.auth.IstioCertificateService",grpc_type="unary",le="0.01"} 4
grpc_server_handling_seconds_bucket{grpc_method="CreateCertificate",grpc_service="istio.v1.auth.IstioCertificateService",grpc_type="unary",le="0.025"} 4
grpc_server_handling_seconds_bucket{grpc_method="CreateCertificate",grpc_service="istio.v1.auth.IstioCertificateService",grpc_type="unary",le="0.05"} 4
grpc_server_handling_seconds_bucket{grpc_method="CreateCertificate",grpc_service="istio.v1.auth.IstioCertificateService",grpc_type="unary",le="0.1"} 4
grpc_server_handling_seconds_bucket{grpc_method="CreateCertificate",grpc_service="istio.v1.auth.IstioCertificateService",grpc_type="unary",le="0.25"} 4
grpc_server_handling_seconds_bucket{grpc_method="CreateCertificate",grpc_service="istio.v1.auth.IstioCertificateService",grpc_type="unary",le="0.5"} 4
grpc_server_handling_seconds_bucket{grpc_method="CreateCertificate",grpc_service="istio.v1.auth.IstioCertificateService",grpc_type="unary",le="1"} 4
grpc_server_handling_seconds_bucket{grpc_method="CreateCertificate",grpc_service="istio.v1.auth.IstioCertificateService",grpc_type="unary",le="2.5"} 4
grpc_server_handling_seconds_bucket{grpc_method="CreateCertificate",grpc_service="istio.v1.auth.IstioCertificateService",grpc_type="unary",le="5"} 4
grpc_server_handling_seconds_bucket{grpc_method="CreateCertificate",grpc_service="istio.v1.auth.IstioCertificateService",grpc_type="unary",le="10"} 4
grpc_server_handling_seconds_bucket{grpc_method="CreateCertificate",grpc_service="istio.v1.auth.IstioCertificateService",grpc_type="unary",le="+Inf"} 4
grpc_server_handling_seconds_sum{grpc_method="CreateCertificate",grpc_service="istio.v1.auth.IstioCertificateService",grpc_type="unary"} 0.03276689
grpc_server_handling_seconds_count{grpc_method="CreateCertificate",grpc_service="istio.v1.auth.IstioCertificateService",grpc_type="unary"} 4
# HELP grpc_server_msg_received_total Total number of RPC stream messages received on the server.
# TYPE grpc_server_msg_received_total counter
grpc_server_msg_received_total{grpc_method="CreateCertificate",grpc_service="istio.v1.auth.IstioCertificateService",grpc_type="unary"} 4
# HELP grpc_server_msg_sent_total Total number of gRPC stream messages sent by the server.
# TYPE grpc_server_msg_sent_total counter
grpc_server_msg_sent_total{grpc_method="CreateCertificate",grpc_service="istio.v1.auth.IstioCertificateService",grpc_type="unary"} 4
# HELP grpc_server_started_total Total number of RPCs started on the server.
# TYPE grpc_server_started_total counter
grpc_server_started_total{grpc_method="CreateCertificate",grpc_service="istio.v1.auth.IstioCertificateService",grpc_type="unary"} 4
# HELP istio_build Istio component build info
# TYPE istio_build gauge
istio_build{component="pilot",tag="1.17.8"} 1
# HELP istiod_managed_clusters Number of clusters managed by istiod
# TYPE istiod_managed_clusters gauge
istiod_managed_clusters{cluster_type="local"} 1
istiod_managed_clusters{cluster_type="remote"} 0
# HELP istiod_uptime_seconds Current istiod server uptime in seconds
# TYPE istiod_uptime_seconds gauge
istiod_uptime_seconds 23613.38740838
# HELP pilot_conflict_inbound_listener Number of conflicting inbound listeners.
# TYPE pilot_conflict_inbound_listener gauge
pilot_conflict_inbound_listener 0
# HELP pilot_conflict_outbound_listener_http_over_current_tcp Number of conflicting wildcard http listeners with current wildcard tcp listener.
# TYPE pilot_conflict_outbound_listener_http_over_current_tcp gauge
pilot_conflict_outbound_listener_http_over_current_tcp 0
# HELP pilot_conflict_outbound_listener_tcp_over_current_http Number of conflicting wildcard tcp listeners with current wildcard http listener.
# TYPE pilot_conflict_outbound_listener_tcp_over_current_http gauge
pilot_conflict_outbound_listener_tcp_over_current_http 0
# HELP pilot_conflict_outbound_listener_tcp_over_current_tcp Number of conflicting tcp listeners with current tcp listener.
# TYPE pilot_conflict_outbound_listener_tcp_over_current_tcp gauge
pilot_conflict_outbound_listener_tcp_over_current_tcp 0
# HELP pilot_debounce_time Delay in seconds between the first config enters debouncing and the merged push request is pushed into the push queue.
# TYPE pilot_debounce_time histogram
pilot_debounce_time_bucket{le="0.01"} 0
pilot_debounce_time_bucket{le="0.1"} 0
pilot_debounce_time_bucket{le="1"} 40
pilot_debounce_time_bucket{le="3"} 40
pilot_debounce_time_bucket{le="5"} 40
pilot_debounce_time_bucket{le="10"} 40
pilot_debounce_time_bucket{le="20"} 40
pilot_debounce_time_bucket{le="30"} 40
pilot_debounce_time_bucket{le="+Inf"} 40
pilot_debounce_time_sum 4.936899179
pilot_debounce_time_count 40
# HELP pilot_destrule_subsets Duplicate subsets across destination rules for same host
# TYPE pilot_destrule_subsets gauge
pilot_destrule_subsets 0
# HELP pilot_dns_cluster_without_endpoints DNS clusters without endpoints caused by the endpoint field in STRICT_DNS type cluster is not set or the corresponding subset cannot select any endpoint
# TYPE pilot_dns_cluster_without_endpoints gauge
pilot_dns_cluster_without_endpoints 0
# HELP pilot_duplicate_envoy_clusters Duplicate envoy clusters caused by service entries with same hostname
# TYPE pilot_duplicate_envoy_clusters gauge
pilot_duplicate_envoy_clusters 0
# HELP pilot_eds_no_instances Number of clusters without instances.
# TYPE pilot_eds_no_instances gauge
pilot_eds_no_instances 0
# HELP pilot_endpoint_not_ready Endpoint found in unready state.
# TYPE pilot_endpoint_not_ready gauge
pilot_endpoint_not_ready 0
# HELP pilot_inbound_updates Total number of updates received by pilot.
# TYPE pilot_inbound_updates counter
pilot_inbound_updates{type="config"} 174
pilot_inbound_updates{type="eds"} 94
pilot_inbound_updates{type="svc"} 31
pilot_inbound_updates{type="svcdelete"} 10
# HELP pilot_info Pilot version and build information.
# TYPE pilot_info gauge
pilot_info{version="1.17.8-a781f9ee6c511d8f22140d8990c31e577b2a9676-Clean"} 1
# HELP pilot_k8s_cfg_events Events from k8s config.
# TYPE pilot_k8s_cfg_events counter
pilot_k8s_cfg_events{event="add",type="EnvoyFilter"} 10
pilot_k8s_cfg_events{event="add",type="Gateway"} 1
pilot_k8s_cfg_events{event="add",type="VirtualService"} 1
# HELP pilot_k8s_endpoints_pending_pod Number of endpoints that do not currently have any corresponding pods.
# TYPE pilot_k8s_endpoints_pending_pod gauge
pilot_k8s_endpoints_pending_pod 0
# HELP pilot_k8s_reg_events Events from k8s registry.
# TYPE pilot_k8s_reg_events counter
pilot_k8s_reg_events{event="add",type="EndpointSlice"} 23
pilot_k8s_reg_events{event="add",type="Namespaces"} 10
pilot_k8s_reg_events{event="add",type="Nodes"} 1
pilot_k8s_reg_events{event="add",type="Pods"} 25
pilot_k8s_reg_events{event="add",type="Services"} 23
pilot_k8s_reg_events{event="delete",type="EndpointSlice"} 10
pilot_k8s_reg_events{event="delete",type="Namespaces"} 1
pilot_k8s_reg_events{event="delete",type="Pods"} 7
pilot_k8s_reg_events{event="delete",type="Services"} 10
pilot_k8s_reg_events{event="update",type="EndpointSlice"} 50
pilot_k8s_reg_events{event="update",type="Namespaces"} 3
pilot_k8s_reg_events{event="update",type="Nodes"} 82
pilot_k8s_reg_events{event="update",type="Pods"} 161
pilot_k8s_reg_events{event="update",type="Services"} 4
# HELP pilot_no_ip Pods not found in the endpoint table, possibly invalid.
# TYPE pilot_no_ip gauge
pilot_no_ip 0
# HELP pilot_proxy_convergence_time Delay in seconds between config change and a proxy receiving all required configuration.
# TYPE pilot_proxy_convergence_time histogram
pilot_proxy_convergence_time_bucket{le="0.1"} 95
pilot_proxy_convergence_time_bucket{le="0.5"} 95
pilot_proxy_convergence_time_bucket{le="1"} 95
pilot_proxy_convergence_time_bucket{le="3"} 95
pilot_proxy_convergence_time_bucket{le="5"} 95
pilot_proxy_convergence_time_bucket{le="10"} 95
pilot_proxy_convergence_time_bucket{le="20"} 95
pilot_proxy_convergence_time_bucket{le="30"} 95
pilot_proxy_convergence_time_bucket{le="+Inf"} 95
pilot_proxy_convergence_time_sum 0.11708706899999997
pilot_proxy_convergence_time_count 95
# HELP pilot_proxy_queue_time Time in seconds, a proxy is in the push queue before being dequeued.
# TYPE pilot_proxy_queue_time histogram
pilot_proxy_queue_time_bucket{le="0.1"} 95
pilot_proxy_queue_time_bucket{le="0.5"} 95
pilot_proxy_queue_time_bucket{le="1"} 95
pilot_proxy_queue_time_bucket{le="3"} 95
pilot_proxy_queue_time_bucket{le="5"} 95
pilot_proxy_queue_time_bucket{le="10"} 95
pilot_proxy_queue_time_bucket{le="20"} 95
pilot_proxy_queue_time_bucket{le="30"} 95
pilot_proxy_queue_time_bucket{le="+Inf"} 95
pilot_proxy_queue_time_sum 0.0030975040000000014
pilot_proxy_queue_time_count 95
# HELP pilot_push_triggers Total number of times a push was triggered, labeled by reason for the push.
# TYPE pilot_push_triggers counter
pilot_push_triggers{type="config"} 2
pilot_push_triggers{type="endpoint"} 210
pilot_push_triggers{type="headlessendpoint"} 51
pilot_push_triggers{type="proxy"} 4
pilot_push_triggers{type="secret"} 48
pilot_push_triggers{type="service"} 84
# HELP pilot_pushcontext_init_seconds Total time in seconds Pilot takes to init pushContext.
# TYPE pilot_pushcontext_init_seconds histogram
pilot_pushcontext_init_seconds_bucket{le="0.01"} 27
pilot_pushcontext_init_seconds_bucket{le="0.1"} 27
pilot_pushcontext_init_seconds_bucket{le="0.5"} 27
pilot_pushcontext_init_seconds_bucket{le="1"} 27
pilot_pushcontext_init_seconds_bucket{le="3"} 27
pilot_pushcontext_init_seconds_bucket{le="5"} 27
pilot_pushcontext_init_seconds_bucket{le="+Inf"} 27
pilot_pushcontext_init_seconds_sum 0.012643411000000002
pilot_pushcontext_init_seconds_count 27
# HELP pilot_services Total services known to pilot.
# TYPE pilot_services gauge
pilot_services 13
# HELP pilot_virt_services Total virtual services known to pilot.
# TYPE pilot_virt_services gauge
pilot_virt_services 1
# HELP pilot_vservice_dup_domain Virtual services with dup domains.
# TYPE pilot_vservice_dup_domain gauge
pilot_vservice_dup_domain 0
# HELP pilot_xds Number of endpoints connected to this pilot using XDS.
# TYPE pilot_xds gauge
pilot_xds{version="1.17.8"} 3
# HELP pilot_xds_config_size_bytes Distribution of configuration sizes pushed to clients
# TYPE pilot_xds_config_size_bytes histogram
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.cluster.v3.Cluster",le="1"} 0
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.cluster.v3.Cluster",le="10000"} 0
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.cluster.v3.Cluster",le="1e+06"} 87
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.cluster.v3.Cluster",le="4e+06"} 87
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.cluster.v3.Cluster",le="1e+07"} 87
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.cluster.v3.Cluster",le="4e+07"} 87
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.cluster.v3.Cluster",le="+Inf"} 87
pilot_xds_config_size_bytes_sum{type="type.googleapis.com/envoy.config.cluster.v3.Cluster"} 1.5201149999999995e+06
pilot_xds_config_size_bytes_count{type="type.googleapis.com/envoy.config.cluster.v3.Cluster"} 87
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment",le="1"} 0
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment",le="10000"} 110
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment",le="1e+06"} 110
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment",le="4e+06"} 110
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment",le="1e+07"} 110
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment",le="4e+07"} 110
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment",le="+Inf"} 110
pilot_xds_config_size_bytes_sum{type="type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment"} 76903.00000000001
pilot_xds_config_size_bytes_count{type="type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment"} 110
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.listener.v3.Listener",le="1"} 15
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.listener.v3.Listener",le="10000"} 38
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.listener.v3.Listener",le="1e+06"} 87
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.listener.v3.Listener",le="4e+06"} 87
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.listener.v3.Listener",le="1e+07"} 87
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.listener.v3.Listener",le="4e+07"} 87
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.listener.v3.Listener",le="+Inf"} 87
pilot_xds_config_size_bytes_sum{type="type.googleapis.com/envoy.config.listener.v3.Listener"} 2.0994560000000005e+06
pilot_xds_config_size_bytes_count{type="type.googleapis.com/envoy.config.listener.v3.Listener"} 87
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.route.v3.RouteConfiguration",le="1"} 0
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.route.v3.RouteConfiguration",le="10000"} 84
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.route.v3.RouteConfiguration",le="1e+06"} 84
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.route.v3.RouteConfiguration",le="4e+06"} 84
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.route.v3.RouteConfiguration",le="1e+07"} 84
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.route.v3.RouteConfiguration",le="4e+07"} 84
pilot_xds_config_size_bytes_bucket{type="type.googleapis.com/envoy.config.route.v3.RouteConfiguration",le="+Inf"} 84
pilot_xds_config_size_bytes_sum{type="type.googleapis.com/envoy.config.route.v3.RouteConfiguration"} 407473.99999999994
pilot_xds_config_size_bytes_count{type="type.googleapis.com/envoy.config.route.v3.RouteConfiguration"} 84
# HELP pilot_xds_expired_nonce Total number of XDS requests with an expired nonce.
# TYPE pilot_xds_expired_nonce counter
pilot_xds_expired_nonce{type="eds"} 27
pilot_xds_expired_nonce{type="rds"} 12
# HELP pilot_xds_push_time Total time in seconds Pilot takes to push lds, rds, cds and eds.
# TYPE pilot_xds_push_time histogram
pilot_xds_push_time_bucket{type="cds",le="0.01"} 87
pilot_xds_push_time_bucket{type="cds",le="0.1"} 87
pilot_xds_push_time_bucket{type="cds",le="1"} 87
pilot_xds_push_time_bucket{type="cds",le="3"} 87
pilot_xds_push_time_bucket{type="cds",le="5"} 87
pilot_xds_push_time_bucket{type="cds",le="10"} 87
pilot_xds_push_time_bucket{type="cds",le="20"} 87
pilot_xds_push_time_bucket{type="cds",le="30"} 87
pilot_xds_push_time_bucket{type="cds",le="+Inf"} 87
pilot_xds_push_time_sum{type="cds"} 0.030460150999999994
pilot_xds_push_time_count{type="cds"} 87
pilot_xds_push_time_bucket{type="eds",le="0.01"} 110
pilot_xds_push_time_bucket{type="eds",le="0.1"} 110
pilot_xds_push_time_bucket{type="eds",le="1"} 110
pilot_xds_push_time_bucket{type="eds",le="3"} 110
pilot_xds_push_time_bucket{type="eds",le="5"} 110
pilot_xds_push_time_bucket{type="eds",le="10"} 110
pilot_xds_push_time_bucket{type="eds",le="20"} 110
pilot_xds_push_time_bucket{type="eds",le="30"} 110
pilot_xds_push_time_bucket{type="eds",le="+Inf"} 110
pilot_xds_push_time_sum{type="eds"} 0.010252860999999999
pilot_xds_push_time_count{type="eds"} 110
pilot_xds_push_time_bucket{type="lds",le="0.01"} 87
pilot_xds_push_time_bucket{type="lds",le="0.1"} 87
pilot_xds_push_time_bucket{type="lds",le="1"} 87
pilot_xds_push_time_bucket{type="lds",le="3"} 87
pilot_xds_push_time_bucket{type="lds",le="5"} 87
pilot_xds_push_time_bucket{type="lds",le="10"} 87
pilot_xds_push_time_bucket{type="lds",le="20"} 87
pilot_xds_push_time_bucket{type="lds",le="30"} 87
pilot_xds_push_time_bucket{type="lds",le="+Inf"} 87
pilot_xds_push_time_sum{type="lds"} 0.076346487
pilot_xds_push_time_count{type="lds"} 87
pilot_xds_push_time_bucket{type="rds",le="0.01"} 84
pilot_xds_push_time_bucket{type="rds",le="0.1"} 84
pilot_xds_push_time_bucket{type="rds",le="1"} 84
pilot_xds_push_time_bucket{type="rds",le="3"} 84
pilot_xds_push_time_bucket{type="rds",le="5"} 84
pilot_xds_push_time_bucket{type="rds",le="10"} 84
pilot_xds_push_time_bucket{type="rds",le="20"} 84
pilot_xds_push_time_bucket{type="rds",le="30"} 84
pilot_xds_push_time_bucket{type="rds",le="+Inf"} 84
pilot_xds_push_time_sum{type="rds"} 0.012056376999999998
pilot_xds_push_time_count{type="rds"} 84
# HELP pilot_xds_pushes Pilot build and send errors for lds, rds, cds and eds.
# TYPE pilot_xds_pushes counter
pilot_xds_pushes{type="cds"} 87
pilot_xds_pushes{type="eds"} 110
pilot_xds_pushes{type="lds"} 87
pilot_xds_pushes{type="rds"} 84
# HELP pilot_xds_send_time Total time in seconds Pilot takes to send generated configuration.
# TYPE pilot_xds_send_time histogram
pilot_xds_send_time_bucket{le="0.01"} 368
pilot_xds_send_time_bucket{le="0.1"} 368
pilot_xds_send_time_bucket{le="1"} 368
pilot_xds_send_time_bucket{le="3"} 368
pilot_xds_send_time_bucket{le="5"} 368
pilot_xds_send_time_bucket{le="10"} 368
pilot_xds_send_time_bucket{le="20"} 368
pilot_xds_send_time_bucket{le="30"} 368
pilot_xds_send_time_bucket{le="+Inf"} 368
pilot_xds_send_time_sum 0.005986710999999994
pilot_xds_send_time_count 368
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 27.66
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 1.073741816e+09
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 18
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 1.30940928e+08
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.74624542251e+09
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 5.12464896e+09
# HELP process_virtual_memory_max_bytes Maximum amount of virtual memory available in bytes.
# TYPE process_virtual_memory_max_bytes gauge
process_virtual_memory_max_bytes 1.8446744073709552e+19
# HELP sidecar_injection_requests_total Total number of sidecar injection requests.
# TYPE sidecar_injection_requests_total counter
sidecar_injection_requests_total 3
# HELP sidecar_injection_success_total Total number of successful sidecar injection requests.
# TYPE sidecar_injection_success_total counter
sidecar_injection_success_total 3
# HELP webhook_patch_attempts_total Webhook patching attempts
# TYPE webhook_patch_attempts_total counter
webhook_patch_attempts_total{name="istio-revision-tag-default"} 1
webhook_patch_attempts_total{name="istio-sidecar-injector"} 3
7. PodMonitor 매니페스트 확인
1
cat ch7/pod-monitor-dp.yaml
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
name: envoy-stats-monitor
namespace: prometheus
labels:
monitoring: istio-proxies
release: prom
spec:
selector:
matchExpressions:
- {key: istio-prometheus-ignore, operator: DoesNotExist}
namespaceSelector:
any: true
jobLabel: envoy-stats
podMetricsEndpoints:
- path: /stats/prometheus
interval: 15s
relabelings:
- action: keep
sourceLabels: [__meta_kubernetes_pod_container_name]
regex: "istio-proxy"
- action: keep
sourceLabels: [__meta_kubernetes_pod_annotationpresent_prometheus_io_scrape]
- sourceLabels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
action: replace
regex: ([^:]+)(?::\d+)?;(\d+)
replacement: $1:$2
targetLabel: __address__
- action: labeldrop
regex: "__meta_kubernetes_pod_label_(.+)"
- sourceLabels: [__meta_kubernetes_namespace]
action: replace
targetLabel: namespace
- sourceLabels: [__meta_kubernetes_pod_name]
action: replace
targetLabel: pod_name
8. Istio 데이터 플레인용 PodMonitor 리소스 적용
1
2
3
4
kubectl apply -f ch7/pod-monitor-dp.yaml -n prometheus
# 결과
podmonitor.monitoring.coreos.com/envoy-stats-monitor created
9. PodMonitor 리소스 목록 확인
1
kubectl get podmonitor -n prometheus
✅ 출력
1
2
NAME AGE
envoy-stats-monitor 17s
10. 웹앱 서비스에 반복 요청 전송
1
for in in {1..10}; do curl -s http://webapp.istioinaction.io:30000/api/catalog ; sleep 0.5; done
✅ 출력
1
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}][{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}][{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}][{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}][{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}][{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}][{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}][{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}][{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}][{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]
11. 웹앱 서비스에 지속적인 부하 생성
1
while true; do curl -s http://webapp.istioinaction.io:30000/api/catalog ; date "+%Y-%m-%d %H:%M:%S" ; sleep 1; echo; done
✅ 출력 
12. 웹앱 파드의 Envoy 메트릭 직접 조회
1
2
WEBAPP=$(kubectl get pod -n istioinaction -l app=webapp -o jsonpath='{.items[0].status.podIP}')
kubectl exec -it netshoot -- curl -s $WEBAPP:15020/stats/prometheus
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
...
istio_response_bytes_bucket{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",le="25"} 0
istio_response_bytes_bucket{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",le="50"} 0
istio_response_bytes_bucket{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",le="100"} 0
istio_response_bytes_bucket{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",le="250"} 0
istio_response_bytes_bucket{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",le="500"} 0
istio_response_bytes_bucket{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",le="1000"} 483
istio_response_bytes_bucket{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",le="2500"} 483
istio_response_bytes_bucket{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",le="5000"} 483
istio_response_bytes_bucket{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",le="10000"} 483
istio_response_bytes_bucket{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",le="30000"} 483
istio_response_bytes_bucket{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",le="60000"} 483
istio_response_bytes_bucket{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",le="300000"} 483
istio_response_bytes_bucket{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",le="600000"} 483
istio_response_bytes_bucket{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",le="1800000"} 483
istio_response_bytes_bucket{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",le="3600000"} 483
istio_response_bytes_bucket{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",le="+Inf"} 483
istio_response_bytes_sum{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown"} 417795
istio_response_bytes_count{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown"} 483
# TYPE envoy_listener_manager_lds_update_duration histogram
envoy_listener_manager_lds_update_duration_bucket{le="0.5"} 0
envoy_listener_manager_lds_update_duration_bucket{le="1"} 0
envoy_listener_manager_lds_update_duration_bucket{le="5"} 21
envoy_listener_manager_lds_update_duration_bucket{le="10"} 21
envoy_listener_manager_lds_update_duration_bucket{le="25"} 22
envoy_listener_manager_lds_update_duration_bucket{le="50"} 22
envoy_listener_manager_lds_update_duration_bucket{le="100"} 22
envoy_listener_manager_lds_update_duration_bucket{le="250"} 22
envoy_listener_manager_lds_update_duration_bucket{le="500"} 22
envoy_listener_manager_lds_update_duration_bucket{le="1000"} 22
envoy_listener_manager_lds_update_duration_bucket{le="2500"} 22
envoy_listener_manager_lds_update_duration_bucket{le="5000"} 22
envoy_listener_manager_lds_update_duration_bucket{le="10000"} 22
envoy_listener_manager_lds_update_duration_bucket{le="30000"} 22
envoy_listener_manager_lds_update_duration_bucket{le="60000"} 22
envoy_listener_manager_lds_update_duration_bucket{le="300000"} 22
envoy_listener_manager_lds_update_duration_bucket{le="600000"} 22
envoy_listener_manager_lds_update_duration_bucket{le="1800000"} 22
envoy_listener_manager_lds_update_duration_bucket{le="3600000"} 22
envoy_listener_manager_lds_update_duration_bucket{le="+Inf"} 22
envoy_listener_manager_lds_update_duration_sum{} 51.549999999999997157829056959599
envoy_listener_manager_lds_update_duration_count{} 22
# TYPE envoy_server_initialization_time_ms histogram
envoy_server_initialization_time_ms_bucket{le="0.5"} 0
envoy_server_initialization_time_ms_bucket{le="1"} 0
envoy_server_initialization_time_ms_bucket{le="5"} 0
envoy_server_initialization_time_ms_bucket{le="10"} 0
envoy_server_initialization_time_ms_bucket{le="25"} 0
envoy_server_initialization_time_ms_bucket{le="50"} 0
envoy_server_initialization_time_ms_bucket{le="100"} 1
envoy_server_initialization_time_ms_bucket{le="250"} 1
envoy_server_initialization_time_ms_bucket{le="500"} 1
envoy_server_initialization_time_ms_bucket{le="1000"} 1
envoy_server_initialization_time_ms_bucket{le="2500"} 1
envoy_server_initialization_time_ms_bucket{le="5000"} 1
envoy_server_initialization_time_ms_bucket{le="10000"} 1
envoy_server_initialization_time_ms_bucket{le="30000"} 1
envoy_server_initialization_time_ms_bucket{le="60000"} 1
envoy_server_initialization_time_ms_bucket{le="300000"} 1
envoy_server_initialization_time_ms_bucket{le="600000"} 1
envoy_server_initialization_time_ms_bucket{le="1800000"} 1
envoy_server_initialization_time_ms_bucket{le="3600000"} 1
envoy_server_initialization_time_ms_bucket{le="+Inf"} 1
envoy_server_initialization_time_ms_sum{} 89.5
envoy_server_initialization_time_ms_count{} 1
🧩 Customizing Istio’s standard metrics
1. 기존 EnvoyFilter 리소스 확인
1
kubectl get envoyfilter -n istio-system
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
NAME AGE
stats-filter-1.13 7h
stats-filter-1.14 7h
stats-filter-1.15 7h
stats-filter-1.16 7h
stats-filter-1.17 7h
tcp-stats-filter-1.13 7h
tcp-stats-filter-1.14 7h
tcp-stats-filter-1.15 7h
tcp-stats-filter-1.16 7h
tcp-stats-filter-1.17 7h
2. EnvoyFilter(stats-filter-1.13) 세부 내용 확인
1
kubectl get envoyfilter stats-filter-1.13 -n istio-system -o yaml
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"networking.istio.io/v1alpha3","kind":"EnvoyFilter","metadata":{"annotations":{},"labels":{"install.operator.istio.io/owning-resource-namespace":"istio-system","istio.io/rev":"default","operator.istio.io/component":"Pilot","operator.istio.io/managed":"Reconcile","operator.istio.io/version":"1.17.8"},"name":"stats-filter-1.13","namespace":"istio-system"},"spec":{"configPatches":[{"applyTo":"HTTP_FILTER","match":{"context":"SIDECAR_OUTBOUND","listener":{"filterChain":{"filter":{"name":"envoy.filters.network.http_connection_manager","subFilter":{"name":"envoy.filters.http.router"}}}},"proxy":{"proxyVersion":"^1\\.13.*"}},"patch":{"operation":"INSERT_BEFORE","value":{"name":"istio.stats","typed_config":{"@type":"type.googleapis.com/udpa.type.v1.TypedStruct","type_url":"type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm","value":{"config":{"configuration":{"@type":"type.googleapis.com/google.protobuf.StringValue","value":"{\n \"debug\": \"false\",\n \"stat_prefix\": \"istio\"\n}\n"},"root_id":"stats_outbound","vm_config":{"code":{"local":{"inline_string":"envoy.wasm.stats"}},"runtime":"envoy.wasm.runtime.null","vm_id":"stats_outbound"}}}}}}},{"applyTo":"HTTP_FILTER","match":{"context":"SIDECAR_INBOUND","listener":{"filterChain":{"filter":{"name":"envoy.filters.network.http_connection_manager","subFilter":{"name":"envoy.filters.http.router"}}}},"proxy":{"proxyVersion":"^1\\.13.*"}},"patch":{"operation":"INSERT_BEFORE","value":{"name":"istio.stats","typed_config":{"@type":"type.googleapis.com/udpa.type.v1.TypedStruct","type_url":"type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm","value":{"config":{"configuration":{"@type":"type.googleapis.com/google.protobuf.StringValue","value":"{\n \"debug\": \"false\",\n \"stat_prefix\": \"istio\",\n \"disable_host_header_fallback\": true\n}\n"},"root_id":"stats_inbound","vm_config":{"code":{"local":{"inline_string":"envoy.wasm.stats"}},"runtime":"envoy.wasm.runtime.null","vm_id":"stats_inbound"}}}}}}},{"applyTo":"HTTP_FILTER","match":{"context":"GATEWAY","listener":{"filterChain":{"filter":{"name":"envoy.filters.network.http_connection_manager","subFilter":{"name":"envoy.filters.http.router"}}}},"proxy":{"proxyVersion":"^1\\.13.*"}},"patch":{"operation":"INSERT_BEFORE","value":{"name":"istio.stats","typed_config":{"@type":"type.googleapis.com/udpa.type.v1.TypedStruct","type_url":"type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm","value":{"config":{"configuration":{"@type":"type.googleapis.com/google.protobuf.StringValue","value":"{\n \"debug\": \"false\",\n \"stat_prefix\": \"istio\",\n \"disable_host_header_fallback\": true\n}\n"},"root_id":"stats_outbound","vm_config":{"code":{"local":{"inline_string":"envoy.wasm.stats"}},"runtime":"envoy.wasm.runtime.null","vm_id":"stats_outbound"}}}}}}}],"priority":-1}}
creationTimestamp: "2025-05-03T04:10:08Z"
generation: 1
labels:
install.operator.istio.io/owning-resource-namespace: istio-system
istio.io/rev: default
operator.istio.io/component: Pilot
operator.istio.io/managed: Reconcile
operator.istio.io/version: 1.17.8
name: stats-filter-1.13
namespace: istio-system
resourceVersion: "1403"
uid: 49f0d4e1-9edb-47f6-8331-04eda2ec4ed0
spec:
configPatches:
- applyTo: HTTP_FILTER
match:
context: SIDECAR_OUTBOUND
listener:
filterChain:
filter:
name: envoy.filters.network.http_connection_manager
subFilter:
name: envoy.filters.http.router
proxy:
proxyVersion: ^1\.13.*
patch:
operation: INSERT_BEFORE
value:
name: istio.stats
typed_config:
'@type': type.googleapis.com/udpa.type.v1.TypedStruct
type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
value:
config:
configuration:
'@type': type.googleapis.com/google.protobuf.StringValue
value: |
{
"debug": "false",
"stat_prefix": "istio"
}
root_id: stats_outbound
vm_config:
code:
local:
inline_string: envoy.wasm.stats
runtime: envoy.wasm.runtime.null
vm_id: stats_outbound
- applyTo: HTTP_FILTER
match:
context: SIDECAR_INBOUND
listener:
filterChain:
filter:
name: envoy.filters.network.http_connection_manager
subFilter:
name: envoy.filters.http.router
proxy:
proxyVersion: ^1\.13.*
patch:
operation: INSERT_BEFORE
value:
name: istio.stats
typed_config:
'@type': type.googleapis.com/udpa.type.v1.TypedStruct
type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
value:
config:
configuration:
'@type': type.googleapis.com/google.protobuf.StringValue
value: |
{
"debug": "false",
"stat_prefix": "istio",
"disable_host_header_fallback": true
}
root_id: stats_inbound
vm_config:
code:
local:
inline_string: envoy.wasm.stats
runtime: envoy.wasm.runtime.null
vm_id: stats_inbound
- applyTo: HTTP_FILTER
match:
context: GATEWAY
listener:
filterChain:
filter:
name: envoy.filters.network.http_connection_manager
subFilter:
name: envoy.filters.http.router
proxy:
proxyVersion: ^1\.13.*
patch:
operation: INSERT_BEFORE
value:
name: istio.stats
typed_config:
'@type': type.googleapis.com/udpa.type.v1.TypedStruct
type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
value:
config:
configuration:
'@type': type.googleapis.com/google.protobuf.StringValue
value: |
{
"debug": "false",
"stat_prefix": "istio",
"disable_host_header_fallback": true
}
root_id: stats_outbound
vm_config:
code:
local:
inline_string: envoy.wasm.stats
runtime: envoy.wasm.runtime.null
vm_id: stats_outbound
priority: -1
3. IstioOperator 파일을 통한 메트릭 디멘션 커스터마이징 작성
1
cat ch7/metrics/istio-operator-new-dimensions.yaml
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
profile: demo
values:
telemetry:
v2:
prometheus:
configOverride:
inboundSidecar:
metrics:
- name: requests_total
dimensions:
upstream_proxy_version: upstream_peer.istio_version
source_mesh_id: node.metadata['MESH_ID']
tags_to_remove:
- request_protocol
outboundSidecar:
metrics:
- name: requests_total
dimensions:
upstream_proxy_version: upstream_peer.istio_version
source_mesh_id: node.metadata['MESH_ID']
tags_to_remove:
- request_protocol
gateway:
metrics:
- name: requests_total
dimensions:
upstream_proxy_version: upstream_peer.istio_version
source_mesh_id: node.metadata['MESH_ID']
tags_to_remove:
- request_protocol
4. Istio 설치 상태에서 Prometheus 텔레메트리 구성 확인
1
kubectl get istiooperator installed-state -n istio-system -o yaml | grep -E "prometheus:|telemetry:" -A2
✅ 출력
1
2
3
4
5
6
7
telemetry:
enabled: true
v2:
--
prometheus:
enabled: true
wasmEnabled: false
5. 웹앱 사이드카에서 커스터마이징된 메트릭 확인
1
2
kubectl -n istioinaction exec -it deploy/webapp -c istio-proxy \
-- curl localhost:15000/stats/prometheus | grep istio_requests_total
✅ 출력
1
2
3
# TYPE istio_requests_total counter
istio_requests_total{reporter="destination",source_workload="istio-ingressgateway",source_canonical_service="istio-ingressgateway",source_canonical_revision="latest",source_workload_namespace="istio-system",source_principal="spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account",source_app="istio-ingressgateway",source_version="unknown",source_cluster="Kubernetes",destination_workload="webapp",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",destination_app="webapp",destination_version="",destination_service="webapp.istioinaction.svc.cluster.local",destination_canonical_service="webapp",destination_canonical_revision="latest",destination_service_name="webapp",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="mutual_tls"} 1499
istio_requests_total{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown"} 1499
6. 커스터마이징 파일 작성 및 Istio 설치 확인
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
docker exec -it myk8s-control-plane bash
root@myk8s-control-plane:/# cat << EOF > istio-operator-new-dimensions.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
profile: demo
values:
telemetry:
v2:
prometheus:
configOverride:
inboundSidecar:
metrics:
- name: requests_total
dimensions:
upstream_proxy_version: upstream_peer.istio_version
source_mesh_id: node.metadata['MESH_ID']
tags_to_remove:
- request_protocol
outboundSidecar:
metrics:
- name: requests_total
dimensions:
upstream_proxy_version: upstream_peer.istio_version
source_mesh_id: node.metadata['MESH_ID']
tags_to_remove:
- request_protocol
gateway:
metrics:
- name: requests_total
dimensions:
upstream_proxy_version: upstream_peer.istio_version
source_mesh_id: node.metadata['MESH_ID']
tags_to_remove:
- request_protocol
EOF
istioctl verify-install -f istio-operator-new-dimensions.yaml
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
✔ Deployment: istio-ingressgateway.istio-system checked successfully
✔ PodDisruptionBudget: istio-ingressgateway.istio-system checked successfully
✔ Role: istio-ingressgateway-sds.istio-system checked successfully
✔ RoleBinding: istio-ingressgateway-sds.istio-system checked successfully
✔ Service: istio-ingressgateway.istio-system checked successfully
✔ ServiceAccount: istio-ingressgateway-service-account.istio-system checked successfully
✔ Deployment: istio-egressgateway.istio-system checked successfully
✔ PodDisruptionBudget: istio-egressgateway.istio-system checked successfully
✔ Role: istio-egressgateway-sds.istio-system checked successfully
✔ RoleBinding: istio-egressgateway-sds.istio-system checked successfully
✔ Service: istio-egressgateway.istio-system checked successfully
✔ ServiceAccount: istio-egressgateway-service-account.istio-system checked successfully
✔ ClusterRole: istiod-istio-system.istio-system checked successfully
✔ ClusterRole: istio-reader-istio-system.istio-system checked successfully
✔ ClusterRoleBinding: istio-reader-istio-system.istio-system checked successfully
✔ ClusterRoleBinding: istiod-istio-system.istio-system checked successfully
✔ ServiceAccount: istio-reader-service-account.istio-system checked successfully
✔ Role: istiod-istio-system.istio-system checked successfully
✔ RoleBinding: istiod-istio-system.istio-system checked successfully
✔ ServiceAccount: istiod-service-account.istio-system checked successfully
✔ CustomResourceDefinition: wasmplugins.extensions.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: destinationrules.networking.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: envoyfilters.networking.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: gateways.networking.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: proxyconfigs.networking.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: serviceentries.networking.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: sidecars.networking.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: virtualservices.networking.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: workloadentries.networking.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: workloadgroups.networking.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: authorizationpolicies.security.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: peerauthentications.security.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: requestauthentications.security.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: telemetries.telemetry.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: istiooperators.install.istio.io.istio-system checked successfully
✔ ClusterRole: istiod-clusterrole-istio-system.istio-system checked successfully
✔ ClusterRole: istiod-gateway-controller-istio-system.istio-system checked successfully
✔ ClusterRoleBinding: istiod-clusterrole-istio-system.istio-system checked successfully
✔ ClusterRoleBinding: istiod-gateway-controller-istio-system.istio-system checked successfully
✔ ConfigMap: istio.istio-system checked successfully
✔ Deployment: istiod.istio-system checked successfully
✔ ConfigMap: istio-sidecar-injector.istio-system checked successfully
✔ MutatingWebhookConfiguration: istio-sidecar-injector.istio-system checked successfully
✔ PodDisruptionBudget: istiod.istio-system checked successfully
✔ ClusterRole: istio-reader-clusterrole-istio-system.istio-system checked successfully
✔ ClusterRoleBinding: istio-reader-clusterrole-istio-system.istio-system checked successfully
✔ Role: istiod.istio-system checked successfully
✔ RoleBinding: istiod.istio-system checked successfully
✔ Service: istiod.istio-system checked successfully
✔ ServiceAccount: istiod.istio-system checked successfully
✔ EnvoyFilter: stats-filter-1.13.istio-system checked successfully
✔ EnvoyFilter: tcp-stats-filter-1.13.istio-system checked successfully
✔ EnvoyFilter: stats-filter-1.14.istio-system checked successfully
✔ EnvoyFilter: tcp-stats-filter-1.14.istio-system checked successfully
✔ EnvoyFilter: stats-filter-1.15.istio-system checked successfully
✔ EnvoyFilter: tcp-stats-filter-1.15.istio-system checked successfully
✔ EnvoyFilter: stats-filter-1.16.istio-system checked successfully
✔ EnvoyFilter: tcp-stats-filter-1.16.istio-system checked successfully
✔ EnvoyFilter: stats-filter-1.17.istio-system checked successfully
✔ EnvoyFilter: tcp-stats-filter-1.17.istio-system checked successfully
✔ ValidatingWebhookConfiguration: istio-validator-istio-system.istio-system checked successfully
✔ IstioOperator: .istio-system checked successfully
Checked 15 custom resource definitions
Checked 3 Istio Deployments
✔ Istio is installed and verified successfully
1
root@myk8s-control-plane:/# istioctl verify-install -f istio-operator-new-dimensions.yaml
✅ 출력
1
2
3
4
5
6
✔ Istio core installed
✔ Istiod installed
✔ Egress gateways installed
✔ Ingress gateways installed
✔ Installation complete
Making this installation the default for injection and validation.
1
2
root@myk8s-control-plane:/# exit
exit
7. IstioOperator 적용 후 설정 반영 상태 재확인
1
kubectl get istiooperator -n istio-system installed-state -o yaml | grep -E "prometheus:" -A9
✅ 출력
1
2
3
4
5
6
7
8
9
prometheus:
enabled: true
wasmEnabled: false
stackdriver:
configOverride: {}
enabled: false
logging: false
monitoring: false
topology: false
8. EnvoyFilter(stats-filter-1.13) 수정 후 상태 확인
1
kubectl get envoyfilter stats-filter-1.13 -n istio-system -o yaml
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"networking.istio.io/v1alpha3","kind":"EnvoyFilter","metadata":{"annotations":{},"labels":{"install.operator.istio.io/owning-resource-namespace":"istio-system","istio.io/rev":"default","operator.istio.io/component":"Pilot","operator.istio.io/managed":"Reconcile","operator.istio.io/version":"1.17.8"},"name":"stats-filter-1.13","namespace":"istio-system"},"spec":{"configPatches":[{"applyTo":"HTTP_FILTER","match":{"context":"SIDECAR_OUTBOUND","listener":{"filterChain":{"filter":{"name":"envoy.filters.network.http_connection_manager","subFilter":{"name":"envoy.filters.http.router"}}}},"proxy":{"proxyVersion":"^1\\.13.*"}},"patch":{"operation":"INSERT_BEFORE","value":{"name":"istio.stats","typed_config":{"@type":"type.googleapis.com/udpa.type.v1.TypedStruct","type_url":"type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm","value":{"config":{"configuration":{"@type":"type.googleapis.com/google.protobuf.StringValue","value":"{\"metrics\":[{\"dimensions\":{\"source_mesh_id\":\"node.metadata['MESH_ID']\",\"upstream_proxy_version\":\"upstream_peer.istio_version\"},\"name\":\"requests_total\",\"tags_to_remove\":[\"request_protocol\"]}]}\n"},"root_id":"stats_outbound","vm_config":{"code":{"local":{"inline_string":"envoy.wasm.stats"}},"runtime":"envoy.wasm.runtime.null","vm_id":"stats_outbound"}}}}}}},{"applyTo":"HTTP_FILTER","match":{"context":"SIDECAR_INBOUND","listener":{"filterChain":{"filter":{"name":"envoy.filters.network.http_connection_manager","subFilter":{"name":"envoy.filters.http.router"}}}},"proxy":{"proxyVersion":"^1\\.13.*"}},"patch":{"operation":"INSERT_BEFORE","value":{"name":"istio.stats","typed_config":{"@type":"type.googleapis.com/udpa.type.v1.TypedStruct","type_url":"type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm","value":{"config":{"configuration":{"@type":"type.googleapis.com/google.protobuf.StringValue","value":"{\"metrics\":[{\"dimensions\":{\"source_mesh_id\":\"node.metadata['MESH_ID']\",\"upstream_proxy_version\":\"upstream_peer.istio_version\"},\"name\":\"requests_total\",\"tags_to_remove\":[\"request_protocol\"]}]}\n"},"root_id":"stats_inbound","vm_config":{"code":{"local":{"inline_string":"envoy.wasm.stats"}},"runtime":"envoy.wasm.runtime.null","vm_id":"stats_inbound"}}}}}}},{"applyTo":"HTTP_FILTER","match":{"context":"GATEWAY","listener":{"filterChain":{"filter":{"name":"envoy.filters.network.http_connection_manager","subFilter":{"name":"envoy.filters.http.router"}}}},"proxy":{"proxyVersion":"^1\\.13.*"}},"patch":{"operation":"INSERT_BEFORE","value":{"name":"istio.stats","typed_config":{"@type":"type.googleapis.com/udpa.type.v1.TypedStruct","type_url":"type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm","value":{"config":{"configuration":{"@type":"type.googleapis.com/google.protobuf.StringValue","value":"{\"metrics\":[{\"dimensions\":{\"source_mesh_id\":\"node.metadata['MESH_ID']\",\"upstream_proxy_version\":\"upstream_peer.istio_version\"},\"name\":\"requests_total\",\"tags_to_remove\":[\"request_protocol\"]}]}\n"},"root_id":"stats_outbound","vm_config":{"code":{"local":{"inline_string":"envoy.wasm.stats"}},"runtime":"envoy.wasm.runtime.null","vm_id":"stats_outbound"}}}}}}}],"priority":-1}}
creationTimestamp: "2025-05-03T04:10:08Z"
generation: 2
labels:
install.operator.istio.io/owning-resource-namespace: istio-system
istio.io/rev: default
operator.istio.io/component: Pilot
operator.istio.io/managed: Reconcile
operator.istio.io/version: 1.17.8
name: stats-filter-1.13
namespace: istio-system
resourceVersion: "47569"
uid: 49f0d4e1-9edb-47f6-8331-04eda2ec4ed0
spec:
configPatches:
- applyTo: HTTP_FILTER
match:
context: SIDECAR_OUTBOUND
listener:
filterChain:
filter:
name: envoy.filters.network.http_connection_manager
subFilter:
name: envoy.filters.http.router
proxy:
proxyVersion: ^1\.13.*
patch:
operation: INSERT_BEFORE
value:
name: istio.stats
typed_config:
'@type': type.googleapis.com/udpa.type.v1.TypedStruct
type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
value:
config:
configuration:
'@type': type.googleapis.com/google.protobuf.StringValue
value: |
{"metrics":[{"dimensions":{"source_mesh_id":"node.metadata['MESH_ID']","upstream_proxy_version":"upstream_peer.istio_version"},"name":"requests_total","tags_to_remove":["request_protocol"]}]}
root_id: stats_outbound
vm_config:
code:
local:
inline_string: envoy.wasm.stats
runtime: envoy.wasm.runtime.null
vm_id: stats_outbound
- applyTo: HTTP_FILTER
match:
context: SIDECAR_INBOUND
listener:
filterChain:
filter:
name: envoy.filters.network.http_connection_manager
subFilter:
name: envoy.filters.http.router
proxy:
proxyVersion: ^1\.13.*
patch:
operation: INSERT_BEFORE
value:
name: istio.stats
typed_config:
'@type': type.googleapis.com/udpa.type.v1.TypedStruct
type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
value:
config:
configuration:
'@type': type.googleapis.com/google.protobuf.StringValue
value: |
{"metrics":[{"dimensions":{"source_mesh_id":"node.metadata['MESH_ID']","upstream_proxy_version":"upstream_peer.istio_version"},"name":"requests_total","tags_to_remove":["request_protocol"]}]}
root_id: stats_inbound
vm_config:
code:
local:
inline_string: envoy.wasm.stats
runtime: envoy.wasm.runtime.null
vm_id: stats_inbound
- applyTo: HTTP_FILTER
match:
context: GATEWAY
listener:
filterChain:
filter:
name: envoy.filters.network.http_connection_manager
subFilter:
name: envoy.filters.http.router
proxy:
proxyVersion: ^1\.13.*
patch:
operation: INSERT_BEFORE
value:
name: istio.stats
typed_config:
'@type': type.googleapis.com/udpa.type.v1.TypedStruct
type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
value:
config:
configuration:
'@type': type.googleapis.com/google.protobuf.StringValue
value: |
{"metrics":[{"dimensions":{"source_mesh_id":"node.metadata['MESH_ID']","upstream_proxy_version":"upstream_peer.istio_version"},"name":"requests_total","tags_to_remove":["request_protocol"]}]}
root_id: stats_outbound
vm_config:
code:
local:
inline_string: envoy.wasm.stats
runtime: envoy.wasm.runtime.null
vm_id: stats_outbound
priority: -1
9. 웹앱 배포 YAML에서 추가 메트릭 태그 설정 확인
1
cat ch7/metrics/webapp-deployment-extrastats.yaml
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: webapp
name: webapp
spec:
replicas: 1
selector:
matchLabels:
app: webapp
template:
metadata:
annotations:
proxy.istio.io/config: |-
extraStatTags:
- "upstream_proxy_version"
- "source_mesh_id"
labels:
app: webapp
spec:
containers:
- env:
- name: KUBERNETES_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: istioinaction/webapp:latest
imagePullPolicy: IfNotPresent
name: webapp
ports:
- containerPort: 8080
name: http
protocol: TCP
securityContext:
privileged: false
10. 웹앱 배포에 extraStatTags 설정 적용
1
2
3
4
kubectl apply -n istioinaction -f ch7/metrics/webapp-deployment-extrastats.yaml
# 출력
deployment.apps/webapp configured
11. 웹앱에 부하 생성
1
while true; do curl -s http://webapp.istioinaction.io:30000/api/catalog ; date "+%Y-%m-%d %H:%M:%S" ; sleep 1; echo; done
✅ 출력
1
2
3
4
5
6
7
8
9
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]2025-05-03 21:21:46
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]2025-05-03 21:21:47
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]2025-05-03 21:21:48
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]2025-05-03 21:21:49
...
12. 웹앱 사이드카에서 커스터마이징된 메트릭 확인
1
2
kubectl -n istioinaction exec -it deploy/webapp -c istio-proxy \
-- curl localhost:15000/stats/prometheus | grep istio_requests_total
✅ 출력
1
2
3
# TYPE istio_requests_total counter
istio_requests_total{reporter="destination",source_workload="istio-ingressgateway",source_canonical_service="istio-ingressgateway",source_canonical_revision="latest",source_workload_namespace="istio-system",source_principal="spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account",source_app="istio-ingressgateway",source_version="unknown",source_cluster="Kubernetes",destination_workload="webapp",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",destination_app="webapp",destination_version="",destination_service="webapp.istioinaction.svc.cluster.local",destination_canonical_service="webapp",destination_canonical_revision="latest",destination_service_name="webapp",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="mutual_tls",source_mesh_id="cluster.local",upstream_proxy_version="unknown"} 50
istio_requests_total{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",source_mesh_id="cluster.local",upstream_proxy_version="1.17.8"} 50
🔖 새 속성으로 호출 그룹화하기
1. 호출 조건 기반 속성 생성 필터 정의
1
cat ch7/metrics/attribute-gen.yaml
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: attribute-gen-example
namespace: istioinaction
spec:
configPatches:
## Sidecar Outbound
- applyTo: HTTP_FILTER
match:
context: SIDECAR_OUTBOUND
listener:
filterChain:
filter:
name: envoy.filters.network.http_connection_manager
subFilter:
name: istio.stats
proxy:
proxyVersion: ^1\.13.*
patch:
operation: INSERT_BEFORE
value:
name: istio.attributegen
typed_config:
'@type': type.googleapis.com/udpa.type.v1.TypedStruct
type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
value:
config:
configuration:
'@type': type.googleapis.com/google.protobuf.StringValue
value: |
{
"attributes": [
{
"output_attribute": "istio_operationId",
"match": [
{
"value": "getitems",
"condition": "request.url_path == '/items' && request.method == 'GET'"
},
{
"value": "createitem",
"condition": "request.url_path == '/items' && request.method == 'POST'"
},
{
"value": "deleteitem",
"condition": "request.url_path == '/items' && request.method == 'DELETE'"
}
]
}
]
}
vm_config:
code:
local:
inline_string: envoy.wasm.attributegen
runtime: envoy.wasm.runtime.null
2. Istio 설치 버전 확인
1
docker exec -it myk8s-control-plane istioctl version
✅ 출력
1
2
3
client version: 1.17.8
control plane version: 1.17.8
data plane version: 1.17.8 (4 proxies)
3. EnvoyFilter의 프록시 버전 매칭 조건 수정
1.13 → 1.17로 변경
1
vi ch7/metrics/attribute-gen.yaml
4. 새 속성 필터 적용 (attribute-gen-example 생성)
1
2
3
4
kubectl apply -f ch7/metrics/attribute-gen.yaml -n istioinaction
# 결과
envoyfilter.networking.istio.io/attribute-gen-example created
5. 적용된 EnvoyFilter 리소스 상세 조회
1
kubectl get envoyfilter -n istioinaction -o yaml | kubectl neat
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
apiVersion: v1
items:
- apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: attribute-gen-example
namespace: istioinaction
spec:
configPatches:
- applyTo: HTTP_FILTER
match:
context: SIDECAR_OUTBOUND
listener:
filterChain:
filter:
name: envoy.filters.network.http_connection_manager
subFilter:
name: istio.stats
proxy:
proxyVersion: ^1\.17.*
patch:
operation: INSERT_BEFORE
value:
name: istio.attributegen
typed_config:
'@type': type.googleapis.com/udpa.type.v1.TypedStruct
type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
value:
config:
configuration:
'@type': type.googleapis.com/google.protobuf.StringValue
value: "{\n \"attributes\": [\n {\n \"output_attribute\":
\"istio_operationId\",\n \"match\": [\n {\n \"value\":
\"getitems\",\n \"condition\": \"request.url_path == '/items'
&& request.method == 'GET'\"\n },\n {\n \"value\":
\"createitem\",\n \"condition\": \"request.url_path ==
'/items' && request.method == 'POST'\"\n }, \n {\n
\ \"value\": \"deleteitem\",\n \"condition\": \"request.url_path
== '/items' && request.method == 'DELETE'\"\n } \n
\ ]\n }\n ]\n}\n"
vm_config:
code:
local:
inline_string: envoy.wasm.attributegen
runtime: envoy.wasm.runtime.null
kind: List
metadata: {}
6. EnvoyFilter 목록 확인
1
kubectl get envoyfilter -n istioinaction
✅ 출력
1
2
NAME AGE
attribute-gen-example 43s
7. IstioOperator를 통한 디멘션 설정 파일 생성 및 정상적용 확인
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
docker exec -it myk8s-control-plane bash
root@myk8s-control-plane:/# cat << EOF > istio-operator-new-attribute.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
profile: demo
values:
telemetry:
v2:
prometheus:
configOverride:
outboundSidecar:
metrics:
- name: requests_total
dimensions:
upstream_operation: istio_operationId # 새 디멘션
EOF
istioctl verify-install -f istio-operator-new-attribute.yaml
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
✔ Deployment: istio-ingressgateway.istio-system checked successfully
✔ PodDisruptionBudget: istio-ingressgateway.istio-system checked successfully
✔ Role: istio-ingressgateway-sds.istio-system checked successfully
✔ RoleBinding: istio-ingressgateway-sds.istio-system checked successfully
✔ Service: istio-ingressgateway.istio-system checked successfully
✔ ServiceAccount: istio-ingressgateway-service-account.istio-system checked successfully
✔ Deployment: istio-egressgateway.istio-system checked successfully
✔ PodDisruptionBudget: istio-egressgateway.istio-system checked successfully
✔ Role: istio-egressgateway-sds.istio-system checked successfully
✔ RoleBinding: istio-egressgateway-sds.istio-system checked successfully
✔ Service: istio-egressgateway.istio-system checked successfully
✔ ServiceAccount: istio-egressgateway-service-account.istio-system checked successfully
✔ ClusterRole: istiod-istio-system.istio-system checked successfully
✔ ClusterRole: istio-reader-istio-system.istio-system checked successfully
✔ ClusterRoleBinding: istio-reader-istio-system.istio-system checked successfully
✔ ClusterRoleBinding: istiod-istio-system.istio-system checked successfully
✔ ServiceAccount: istio-reader-service-account.istio-system checked successfully
✔ Role: istiod-istio-system.istio-system checked successfully
✔ RoleBinding: istiod-istio-system.istio-system checked successfully
✔ ServiceAccount: istiod-service-account.istio-system checked successfully
✔ CustomResourceDefinition: wasmplugins.extensions.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: destinationrules.networking.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: envoyfilters.networking.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: gateways.networking.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: proxyconfigs.networking.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: serviceentries.networking.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: sidecars.networking.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: virtualservices.networking.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: workloadentries.networking.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: workloadgroups.networking.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: authorizationpolicies.security.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: peerauthentications.security.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: requestauthentications.security.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: telemetries.telemetry.istio.io.istio-system checked successfully
✔ CustomResourceDefinition: istiooperators.install.istio.io.istio-system checked successfully
✔ ClusterRole: istiod-clusterrole-istio-system.istio-system checked successfully
✔ ClusterRole: istiod-gateway-controller-istio-system.istio-system checked successfully
✔ ClusterRoleBinding: istiod-clusterrole-istio-system.istio-system checked successfully
✔ ClusterRoleBinding: istiod-gateway-controller-istio-system.istio-system checked successfully
✔ ConfigMap: istio.istio-system checked successfully
✔ Deployment: istiod.istio-system checked successfully
✔ ConfigMap: istio-sidecar-injector.istio-system checked successfully
✔ MutatingWebhookConfiguration: istio-sidecar-injector.istio-system checked successfully
✔ PodDisruptionBudget: istiod.istio-system checked successfully
✔ ClusterRole: istio-reader-clusterrole-istio-system.istio-system checked successfully
✔ ClusterRoleBinding: istio-reader-clusterrole-istio-system.istio-system checked successfully
✔ Role: istiod.istio-system checked successfully
✔ RoleBinding: istiod.istio-system checked successfully
✔ Service: istiod.istio-system checked successfully
✔ ServiceAccount: istiod.istio-system checked successfully
✔ EnvoyFilter: stats-filter-1.13.istio-system checked successfully
✔ EnvoyFilter: tcp-stats-filter-1.13.istio-system checked successfully
✔ EnvoyFilter: stats-filter-1.14.istio-system checked successfully
✔ EnvoyFilter: tcp-stats-filter-1.14.istio-system checked successfully
✔ EnvoyFilter: stats-filter-1.15.istio-system checked successfully
✔ EnvoyFilter: tcp-stats-filter-1.15.istio-system checked successfully
✔ EnvoyFilter: stats-filter-1.16.istio-system checked successfully
✔ EnvoyFilter: tcp-stats-filter-1.16.istio-system checked successfully
✔ EnvoyFilter: stats-filter-1.17.istio-system checked successfully
✔ EnvoyFilter: tcp-stats-filter-1.17.istio-system checked successfully
✔ ValidatingWebhookConfiguration: istio-validator-istio-system.istio-system checked successfully
✔ IstioOperator: .istio-system checked successfully
Checked 15 custom resource definitions
Checked 3 Istio Deployments
✔ Istio is installed and verified successfully
8. Istio 설치 재적용 및 불필요 리소스 정리
1
2
3
4
5
6
7
8
9
10
root@myk8s-control-plane:/# istioctl install -f istio-operator-new-attribute.yaml -y
✔ Istio core installed
✔ Istiod installed
✔ Egress gateways installed
✔ Ingress gateways installed
- Pruning removed resources Removed HorizontalPodAutoscaler:istio-system:istio-ingressgateway.
Removed HorizontalPodAutoscaler:istio-system:istiod.
✔ Installation complete Making this installation the default for injection and validation.
Thank you for installing Istio 1.17. Please take a few minutes to tell us about your install/upgrade experience! https://forms.gle/hMHGiwZHPU7UQRWe9
9. IstioOperator 리소스 내 istio_operationId 디멘션 확인
1
2
3
4
root@myk8s-control-plane:/# exit
exit
kubectl get istiooperator -n istio-system installed-state -o yaml | grep -B2 -A1 istio_operationId$
✅ 출력
1
2
3
4
metrics:
- dimensions:
upstream_operation: istio_operationId
name: requests_total
10. stats-filter에 반영된 디멘션 확인 (v1.17 기준)
1
kubectl get envoyfilter -n istio-system stats-filter-1.17 -o yaml | kubectl neat
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
labels:
install.operator.istio.io/owning-resource-namespace: istio-system
istio.io/rev: default
operator.istio.io/component: Pilot
operator.istio.io/managed: Reconcile
operator.istio.io/version: 1.17.8
name: stats-filter-1.17
namespace: istio-system
spec:
configPatches:
- applyTo: HTTP_FILTER
match:
context: SIDECAR_OUTBOUND
listener:
filterChain:
filter:
name: envoy.filters.network.http_connection_manager
subFilter:
name: envoy.filters.http.router
proxy:
proxyVersion: ^1\.17.*
patch:
operation: INSERT_BEFORE
value:
name: istio.stats
typed_config:
'@type': type.googleapis.com/udpa.type.v1.TypedStruct
type_url: type.googleapis.com/stats.PluginConfig
value:
metrics:
- dimensions:
upstream_operation: istio_operationId
name: requests_total
- applyTo: HTTP_FILTER
match:
context: SIDECAR_INBOUND
listener:
filterChain:
filter:
name: envoy.filters.network.http_connection_manager
subFilter:
name: envoy.filters.http.router
proxy:
proxyVersion: ^1\.17.*
patch:
operation: INSERT_BEFORE
value:
name: istio.stats
typed_config:
'@type': type.googleapis.com/udpa.type.v1.TypedStruct
type_url: type.googleapis.com/stats.PluginConfig
value:
disable_host_header_fallback: true
- applyTo: HTTP_FILTER
match:
context: GATEWAY
listener:
filterChain:
filter:
name: envoy.filters.network.http_connection_manager
subFilter:
name: envoy.filters.http.router
proxy:
proxyVersion: ^1\.17.*
patch:
operation: INSERT_BEFORE
value:
name: istio.stats
typed_config:
'@type': type.googleapis.com/udpa.type.v1.TypedStruct
type_url: type.googleapis.com/stats.PluginConfig
value:
disable_host_header_fallback: true
priority: -1
11. 이전 버전 필터 (v1.16)에 디멘션 반영 여부 확인
1
kubectl get envoyfilter -n istio-system stats-filter-1.16 -o yaml | grep istio_operationId -B15 -A5
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"networking.istio.io/v1alpha3","kind":"EnvoyFilter","metadata":{"annotations":{},"labels":{"install.operator.istio.io/owning-resource-namespace":"istio-system","istio.io/rev":"default","operator.istio.io/component":"Pilot","operator.istio.io/managed":"Reconcile","operator.istio.io/version":"1.17.8"},"name":"stats-filter-1.16","namespace":"istio-system"},"spec":{"configPatches":[{"applyTo":"HTTP_FILTER","match":{"context":"SIDECAR_OUTBOUND","listener":{"filterChain":{"filter":{"name":"envoy.filters.network.http_connection_manager","subFilter":{"name":"envoy.filters.http.router"}}}},"proxy":{"proxyVersion":"^1\\.16.*"}},"patch":{"operation":"INSERT_BEFORE","value":{"name":"istio.stats","typed_config":{"@type":"type.googleapis.com/udpa.type.v1.TypedStruct","type_url":"type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm","value":{"config":{"configuration":{"@type":"type.googleapis.com/google.protobuf.StringValue","value":"{\"metrics\":[{\"dimensions\":{\"upstream_operation\":\"istio_operationId\"},\"name\":\"requests_total\"}]}\n"},"root_id":"stats_outbound","vm_config":{"code":{"local":{"inline_string":"envoy.wasm.stats"}},"runtime":"envoy.wasm.runtime.null","vm_id":"stats_outbound"}}}}}}},{"applyTo":"HTTP_FILTER","match":{"context":"SIDECAR_INBOUND","listener":{"filterChain":{"filter":{"name":"envoy.filters.network.http_connection_manager","subFilter":{"name":"envoy.filters.http.router"}}}},"proxy":{"proxyVersion":"^1\\.16.*"}},"patch":{"operation":"INSERT_BEFORE","value":{"name":"istio.stats","typed_config":{"@type":"type.googleapis.com/udpa.type.v1.TypedStruct","type_url":"type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm","value":{"config":{"configuration":{"@type":"type.googleapis.com/google.protobuf.StringValue","value":"{\n \"debug\": \"false\",\n \"stat_prefix\": \"istio\",\n \"disable_host_header_fallback\": true\n}\n"},"root_id":"stats_inbound","vm_config":{"code":{"local":{"inline_string":"envoy.wasm.stats"}},"runtime":"envoy.wasm.runtime.null","vm_id":"stats_inbound"}}}}}}},{"applyTo":"HTTP_FILTER","match":{"context":"GATEWAY","listener":{"filterChain":{"filter":{"name":"envoy.filters.network.http_connection_manager","subFilter":{"name":"envoy.filters.http.router"}}}},"proxy":{"proxyVersion":"^1\\.16.*"}},"patch":{"operation":"INSERT_BEFORE","value":{"name":"istio.stats","typed_config":{"@type":"type.googleapis.com/udpa.type.v1.TypedStruct","type_url":"type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm","value":{"config":{"configuration":{"@type":"type.googleapis.com/google.protobuf.StringValue","value":"{\n \"debug\": \"false\",\n \"stat_prefix\": \"istio\",\n \"disable_host_header_fallback\": true\n}\n"},"root_id":"stats_outbound","vm_config":{"code":{"local":{"inline_string":"envoy.wasm.stats"}},"runtime":"envoy.wasm.runtime.null","vm_id":"stats_outbound"}}}}}}}],"priority":-1}}
creationTimestamp: "2025-05-03T04:10:08Z"
generation: 4
labels:
install.operator.istio.io/owning-resource-namespace: istio-system
istio.io/rev: default
--
name: envoy.filters.http.router
proxy:
proxyVersion: ^1\.16.*
patch:
operation: INSERT_BEFORE
value:
name: istio.stats
typed_config:
'@type': type.googleapis.com/udpa.type.v1.TypedStruct
type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
value:
config:
configuration:
'@type': type.googleapis.com/google.protobuf.StringValue
value: |
{"metrics":[{"dimensions":{"upstream_operation":"istio_operationId"},"name":"requests_total"}]}
root_id: stats_outbound
vm_config:
code:
local:
inline_string: envoy.wasm.stats
12. 호출 트래픽 생성 (catalog 호출 반복)
1
while true; do curl -s http://webapp.istioinaction.io:30000/api/catalog ; date "+%Y-%m-%d %H:%M:%S" ; sleep 1; echo; done
✅ 출력
1
2
3
4
5
6
7
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]2025-05-04 00:51:49
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]2025-05-04 00:51:50
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]2025-05-04 00:51:51
...
13. metrics 확인 – 새 디멘션 적용 확인
1
kubectl -n istioinaction exec -it deploy/webapp -c istio-proxy -- curl localhost:15000/stats/prometheus | grep istio_requests_total
✅ 출력
1
2
3
# TYPE istio_requests_total counter
istio_requests_total{reporter="destination",source_workload="istio-ingressgateway",source_canonical_service="istio-ingressgateway",source_canonical_revision="latest",source_workload_namespace="istio-system",source_principal="spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account",source_app="istio-ingressgateway",source_version="unknown",source_cluster="Kubernetes",destination_workload="webapp",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",destination_app="webapp",destination_version="",destination_service="webapp.istioinaction.svc.cluster.local",destination_canonical_service="webapp",destination_canonical_revision="latest",destination_service_name="webapp",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="mutual_tls"} 49
istio_requests_total{reporter="source",source_workload="webapp",source_canonical_service="webapp",source_canonical_revision="latest",source_workload_namespace="istioinaction",source_principal="spiffe://cluster.local/ns/istioinaction/sa/webapp",source_app="webapp",source_version="",source_cluster="Kubernetes",destination_workload="catalog",destination_workload_namespace="istioinaction",destination_principal="spiffe://cluster.local/ns/istioinaction/sa/catalog",destination_app="catalog",destination_version="v1",destination_service="catalog.istioinaction.svc.cluster.local",destination_canonical_service="catalog",destination_canonical_revision="v1",destination_service_name="catalog",destination_service_namespace="istioinaction",destination_cluster="Kubernetes",request_protocol="http",response_code="200",grpc_response_status="",response_flags="-",connection_security_policy="unknown",upstream_operation="getitems"} 49
http://127.0.0.1:30001/classic/graph?g0.range_input=1h&g0.expr=istio_requests_total&g0.tab=0 
🕵️♂️ 그라파나 대시보드 설정하기
1. 반복 호출로 메트릭 생성하기
1
while true; do curl -s http://webapp.istioinaction.io:30000/api/catalog ; date "+%Y-%m-%d %H:%M:%S" ; sleep 1; echo; done
✅ 출력
1
2
3
4
5
6
7
8
9
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]2025-05-03 21:48:50
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]2025-05-03 21:48:51
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]2025-05-03 21:48:52
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]2025-05-03 21:48:53
....
2. 그라파나 웹 UI 접속
3. Istio 관련 Grafana 대시보드 ConfigMap 생성
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
cd ch8
kubectl -n prometheus create cm istio-dashboards \
--from-file=pilot-dashboard.json=dashboards/\
pilot-dashboard.json \
--from-file=istio-workload-dashboard.json=dashboards/\
istio-workload-dashboard.json \
--from-file=istio-service-dashboard.json=dashboards/\
istio-service-dashboard.json \
--from-file=istio-performance-dashboard.json=dashboards/\
istio-performance-dashboard.json \
--from-file=istio-mesh-dashboard.json=dashboards/\
istio-mesh-dashboard.json \
--from-file=istio-extension-dashboard.json=dashboards/\
istio-extension-dashboard.json
# 결과
configmap/istio-dashboards created
4. ConfigMap 상세 정보 확인
1
2
cd ..
kubectl describe cm -n prometheus istio-dashboards
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
...
"refresh": "5s",
"schemaVersion": 18,
"style": "dark",
"tags": [],
"templating": {
"list": [
{
"current": {
"selected": true,
"text": "default",
"value": "default"
},
"hide": 0,
"includeAll": false,
"label": null,
"multi": false,
"name": "datasource",
"options": [],
"query": "prometheus",
"queryValue": "",
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"type": "datasource"
}
]
},
"time": {
"from": "now-5m",
"to": "now"
},
"timepicker": {
"refresh_intervals": [
"5s",
"10s",
"30s",
"1m",
"5m",
"15m",
"30m",
"1h",
"2h",
"1d"
],
"time_options": [
"5m",
"15m",
"1h",
"6h",
"12h",
"24h",
"2d",
"7d",
"30d"
]
},
"timezone": "browser",
"title": "Istio Control Plane Dashboard",
"uid": "3--MLVZZk",
"version": 11
}
BinaryData
====
Events: <none>
5. Grafana 오퍼레이터가 대시보드 인식하도록 레이블 설정
1
2
3
4
kubectl label -n prometheus cm istio-dashboards grafana_dashboard=1
# 결과
configmap/istio-dashboards labeled
6. Grafana에서 Istio 대시보드 시각화 확인
Istio Control Plane Dashboard 
Istio Service Dashboard 
🧵 분산 트레이싱 시스템 설치하기
1. 컨트롤 플레인 컨테이너에 진입하기
1
2
3
docker exec -it myk8s-control-plane bash
root@myk8s-control-plane:/#
2. 예거 설치 파일 디렉토리 확인
1
root@myk8s-control-plane:/# ls istio-$ISTIOV/samples/addons
✅ 출력
1
2
README.md grafana.yaml kiali.yaml
extras jaeger.yaml prometheus.yaml
3. 예거 통합 배포 리소스 파일 내용 확인
1
root@myk8s-control-plane:/# cat istio-$ISTIOV/samples/addons/jaeger.yaml
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
apiVersion: apps/v1
kind: Deployment
metadata:
name: jaeger
namespace: istio-system
labels:
app: jaeger
spec:
selector:
matchLabels:
app: jaeger
template:
metadata:
labels:
app: jaeger
sidecar.istio.io/inject: "false"
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "14269"
spec:
containers:
- name: jaeger
image: "docker.io/jaegertracing/all-in-one:1.35"
env:
- name: BADGER_EPHEMERAL
value: "false"
- name: SPAN_STORAGE_TYPE
value: "badger"
- name: BADGER_DIRECTORY_VALUE
value: "/badger/data"
- name: BADGER_DIRECTORY_KEY
value: "/badger/key"
- name: COLLECTOR_ZIPKIN_HOST_PORT
value: ":9411"
- name: MEMORY_MAX_TRACES
value: "50000"
- name: QUERY_BASE_PATH
value: /jaeger
livenessProbe:
httpGet:
path: /
port: 14269
readinessProbe:
httpGet:
path: /
port: 14269
volumeMounts:
- name: data
mountPath: /badger
resources:
requests:
cpu: 10m
volumes:
- name: data
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: tracing
namespace: istio-system
labels:
app: jaeger
spec:
type: ClusterIP
ports:
- name: http-query
port: 80
protocol: TCP
targetPort: 16686
# Note: Change port name if you add '--query.grpc.tls.enabled=true'
- name: grpc-query
port: 16685
protocol: TCP
targetPort: 16685
selector:
app: jaeger
---
# Jaeger implements the Zipkin API. To support swapping out the tracing backend, we use a Service named Zipkin.
apiVersion: v1
kind: Service
metadata:
labels:
name: zipkin
name: zipkin
namespace: istio-system
spec:
ports:
- port: 9411
targetPort: 9411
name: http-query
selector:
app: jaeger
---
apiVersion: v1
kind: Service
metadata:
name: jaeger-collector
namespace: istio-system
labels:
app: jaeger
spec:
type: ClusterIP
ports:
- name: jaeger-collector-http
port: 14268
targetPort: 14268
protocol: TCP
- name: jaeger-collector-grpc
port: 14250
targetPort: 14250
protocol: TCP
- port: 9411
targetPort: 9411
name: http-zipkin
selector:
app: jaeger
4. 예거 리소스 배포 실행
1
2
3
4
5
6
7
root@myk8s-control-plane:/# kubectl apply -f istio-$ISTIOV/samples/addons/jaeger.yaml
# 결과
deployment.apps/jaeger created
service/tracing created
service/zipkin created
service/jaeger-collector created
5. 컨트롤 플레인 컨테이너에서 빠져나오기
1
2
root@myk8s-control-plane:/# exit
exit
6. 예거 리소스 배포 상태 확인
1
kubectl get deploy,pod,svc,ep -n istio-system
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/istio-egressgateway 1/1 1 1 135m
deployment.apps/istio-ingressgateway 1/1 1 1 9h
deployment.apps/istiod 1/1 1 1 9h
deployment.apps/jaeger 1/1 1 1 48s
NAME READY STATUS RESTARTS AGE
pod/istio-egressgateway-85df6b84b7-7p9l9 1/1 Running 0 135m
pod/istio-ingressgateway-6bb8fb6549-ctmvz 1/1 Running 0 135m
pod/istiod-8d74787f-tqgxx 1/1 Running 0 135m
pod/jaeger-5556cd8fcf-wrbxg 1/1 Running 0 48s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/istio-egressgateway ClusterIP 10.200.1.163 <none> 80/TCP,443/TCP 135m
service/istio-ingressgateway LoadBalancer 10.200.1.153 <pending> 15021:31958/TCP,80:30000/TCP,443:30005/TCP,31400:31053/TCP,15443:31607/TCP 9h
service/istiod ClusterIP 10.200.1.155 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP 9h
service/jaeger-collector ClusterIP 10.200.1.130 <none> 14268/TCP,14250/TCP,9411/TCP 48s
service/tracing ClusterIP 10.200.1.196 <none> 80/TCP,16685/TCP 48s
service/zipkin ClusterIP 10.200.1.183 <none> 9411/TCP 48s
NAME ENDPOINTS AGE
endpoints/istio-egressgateway 10.10.0.14:8080,10.10.0.14:8443 135m
endpoints/istio-ingressgateway 10.10.0.13:15443,10.10.0.13:15021,10.10.0.13:31400 + 2 more... 9h
endpoints/istiod 10.10.0.5:15012,10.10.0.5:15010,10.10.0.5:15017 + 1 more... 9h
endpoints/jaeger-collector 10.10.0.18:9411,10.10.0.18:14250,10.10.0.18:14268 48s
endpoints/tracing 10.10.0.18:16685,10.10.0.18:16686 48s
endpoints/zipkin 10.10.0.18:9411 48s
7. tracing 서비스의 상세 설정 확인
1
kubectl describe svc -n istio-system tracing
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Name: tracing
Namespace: istio-system
Labels: app=jaeger
Annotations: <none>
Selector: app=jaeger
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.200.1.196
IPs: 10.200.1.196
Port: http-query 80/TCP
TargetPort: 16686/TCP
Endpoints: 10.10.0.18:16686
Port: grpc-query 16685/TCP
TargetPort: 16685/TCP
Endpoints: 10.10.0.18:16685
Session Affinity: None
Internal Traffic Policy: Cluster
Events: <none>
8. tracing 서비스 타입을 NodePort로 변경 및 포트 지정
1
2
3
4
kubectl patch svc -n istio-system tracing -p '{"spec": {"type": "NodePort", "ports": [{"port": 80, "targetPort": 16686, "nodePort": 30004}]}}'
# 결과
service/tracing patched
9. 트레이싱 기능이 포함된 IstioOperator 사양 확인
1
cat ch8/install-istio-tracing-zipkin.yaml
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
namespace: istio-system
spec:
meshConfig:
defaultConfig:
tracing:
sampling: 100
zipkin:
address: zipkin.istio-system:9411
10. 기존 IstioOperator 설정에서 트레이서 설정 확인
1
kubectl get IstioOperator -n istio-system installed-state -o json
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
{
"apiVersion": "install.istio.io/v1alpha1",
"kind": "IstioOperator",
"metadata": {
"annotations": {
"install.istio.io/ignoreReconcile": "true",
"kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"install.istio.io/v1alpha1\",\"kind\":\"IstioOperator\",\"metadata\":{\"annotations\":{\"install.istio.io/ignoreReconcile\":\"true\"},\"creationTimestamp\":null,\"name\":\"installed-state\",\"namespace\":\"istio-system\"},\"spec\":{\"components\":{\"base\":{\"enabled\":true},\"cni\":{\"enabled\":false},\"egressGateways\":[{\"enabled\":false,\"name\":\"istio-egressgateway\"}],\"ingressGateways\":[{\"enabled\":true,\"name\":\"istio-ingressgateway\"}],\"istiodRemote\":{\"enabled\":false},\"pilot\":{\"enabled\":true}},\"hub\":\"docker.io/istio\",\"meshConfig\":{\"defaultConfig\":{\"proxyMetadata\":{}},\"enablePrometheusMerge\":true},\"profile\":\"default\",\"tag\":\"1.17.8\",\"values\":{\"base\":{\"enableCRDTemplates\":false,\"validationURL\":\"\"},\"defaultRevision\":\"\",\"gateways\":{\"istio-egressgateway\":{\"autoscaleEnabled\":true,\"env\":{},\"name\":\"istio-egressgateway\",\"secretVolumes\":[{\"mountPath\":\"/etc/istio/egressgateway-certs\",\"name\":\"egressgateway-certs\",\"secretName\":\"istio-egressgateway-certs\"},{\"mountPath\":\"/etc/istio/egressgateway-ca-certs\",\"name\":\"egressgateway-ca-certs\",\"secretName\":\"istio-egressgateway-ca-certs\"}],\"type\":\"ClusterIP\"},\"istio-ingressgateway\":{\"autoscaleEnabled\":true,\"env\":{},\"name\":\"istio-ingressgateway\",\"secretVolumes\":[{\"mountPath\":\"/etc/istio/ingressgateway-certs\",\"name\":\"ingressgateway-certs\",\"secretName\":\"istio-ingressgateway-certs\"},{\"mountPath\":\"/etc/istio/ingressgateway-ca-certs\",\"name\":\"ingressgateway-ca-certs\",\"secretName\":\"istio-ingressgateway-ca-certs\"}],\"type\":\"LoadBalancer\"}},\"global\":{\"configValidation\":true,\"defaultNodeSelector\":{},\"defaultPodDisruptionBudget\":{\"enabled\":true},\"defaultResources\":{\"requests\":{\"cpu\":\"10m\"}},\"imagePullPolicy\":\"\",\"imagePullSecrets\":[],\"istioNamespace\":\"istio-system\",\"istiod\":{\"enableAnalysis\":false},\"jwtPolicy\":\"third-party-jwt\",\"logAsJson\":false,\"logging\":{\"level\":\"default:info\"},\"meshNetworks\":{},\"mountMtlsCerts\":false,\"multiCluster\":{\"clusterName\":\"\",\"enabled\":false},\"network\":\"\",\"omitSidecarInjectorConfigMap\":false,\"oneNamespace\":false,\"operatorManageWebhooks\":false,\"pilotCertProvider\":\"istiod\",\"priorityClassName\":\"\",\"proxy\":{\"autoInject\":\"enabled\",\"clusterDomain\":\"cluster.local\",\"componentLogLevel\":\"misc:error\",\"enableCoreDump\":false,\"excludeIPRanges\":\"\",\"excludeInboundPorts\":\"\",\"excludeOutboundPorts\":\"\",\"image\":\"proxyv2\",\"includeIPRanges\":\"*\",\"logLevel\":\"warning\",\"privileged\":false,\"readinessFailureThreshold\":30,\"readinessInitialDelaySeconds\":1,\"readinessPeriodSeconds\":2,\"resources\":{\"limits\":{\"cpu\":\"2000m\",\"memory\":\"1024Mi\"},\"requests\":{\"cpu\":\"100m\",\"memory\":\"128Mi\"}},\"statusPort\":15020,\"tracer\":\"zipkin\"},\"proxy_init\":{\"image\":\"proxyv2\",\"resources\":{\"limits\":{\"cpu\":\"2000m\",\"memory\":\"1024Mi\"},\"requests\":{\"cpu\":\"10m\",\"memory\":\"10Mi\"}}},\"sds\":{\"token\":{\"aud\":\"istio-ca\"}},\"sts\":{\"servicePort\":0},\"tracer\":{\"datadog\":{},\"lightstep\":{},\"stackdriver\":{},\"zipkin\":{}},\"useMCP\":false},\"istiodRemote\":{\"injectionURL\":\"\"},\"pilot\":{\"autoscaleEnabled\":true,\"autoscaleMax\":5,\"autoscaleMin\":1,\"configMap\":true,\"cpu\":{\"targetAverageUtilization\":80},\"deploymentLabels\":null,\"enableProtocolSniffingForInbound\":true,\"enableProtocolSniffingForOutbound\":true,\"env\":{},\"image\":\"pilot\",\"keepaliveMaxServerConnectionAge\":\"30m\",\"nodeSelector\":{},\"podLabels\":{},\"replicaCount\":1,\"traceSampling\":1},\"telemetry\":{\"enabled\":true,\"v2\":{\"enabled\":true,\"metadataExchange\":{\"wasmEnabled\":false},\"prometheus\":{\"enabled\":true,\"wasmEnabled\":false},\"stackdriver\":{\"configOverride\":{},\"enabled\":false,\"logging\":false,\"monitoring\":false,\"topology\":false}}}}}}\n"
},
"creationTimestamp": "2025-05-03T04:10:39Z",
"generation": 1,
"name": "installed-state",
"namespace": "istio-system",
"resourceVersion": "1560",
"uid": "733a1444-e86c-40d1-8140-4a273a6181c7"
},
"spec": {
"components": {
"base": {
"enabled": true
},
"cni": {
"enabled": false
},
"egressGateways": [
{
"enabled": false,
"name": "istio-egressgateway"
}
],
"ingressGateways": [
{
"enabled": true,
"name": "istio-ingressgateway"
}
],
"istiodRemote": {
"enabled": false
},
"pilot": {
"enabled": true
}
},
"hub": "docker.io/istio",
"meshConfig": {
"defaultConfig": {
"proxyMetadata": {}
},
"enablePrometheusMerge": true
},
"profile": "default",
"tag": "1.17.8",
"values": {
"base": {
"enableCRDTemplates": false,
"validationURL": ""
},
"defaultRevision": "",
"gateways": {
"istio-egressgateway": {
"autoscaleEnabled": true,
"env": {},
"name": "istio-egressgateway",
"secretVolumes": [
{
"mountPath": "/etc/istio/egressgateway-certs",
"name": "egressgateway-certs",
"secretName": "istio-egressgateway-certs"
},
{
"mountPath": "/etc/istio/egressgateway-ca-certs",
"name": "egressgateway-ca-certs",
"secretName": "istio-egressgateway-ca-certs"
}
],
"type": "ClusterIP"
},
"istio-ingressgateway": {
"autoscaleEnabled": true,
"env": {},
"name": "istio-ingressgateway",
"secretVolumes": [
{
"mountPath": "/etc/istio/ingressgateway-certs",
"name": "ingressgateway-certs",
"secretName": "istio-ingressgateway-certs"
},
{
"mountPath": "/etc/istio/ingressgateway-ca-certs",
"name": "ingressgateway-ca-certs",
"secretName": "istio-ingressgateway-ca-certs"
}
],
"type": "LoadBalancer"
}
},
"global": {
"configValidation": true,
"defaultNodeSelector": {},
"defaultPodDisruptionBudget": {
"enabled": true
},
"defaultResources": {
"requests": {
"cpu": "10m"
}
},
"imagePullPolicy": "",
"imagePullSecrets": [],
"istioNamespace": "istio-system",
"istiod": {
"enableAnalysis": false
},
"jwtPolicy": "third-party-jwt",
"logAsJson": false,
"logging": {
"level": "default:info"
},
"meshNetworks": {},
"mountMtlsCerts": false,
"multiCluster": {
"clusterName": "",
"enabled": false
},
"network": "",
"omitSidecarInjectorConfigMap": false,
"oneNamespace": false,
"operatorManageWebhooks": false,
"pilotCertProvider": "istiod",
"priorityClassName": "",
"proxy": {
"autoInject": "enabled",
"clusterDomain": "cluster.local",
"componentLogLevel": "misc:error",
"enableCoreDump": false,
"excludeIPRanges": "",
"excludeInboundPorts": "",
"excludeOutboundPorts": "",
"image": "proxyv2",
"includeIPRanges": "*",
"logLevel": "warning",
"privileged": false,
"readinessFailureThreshold": 30,
"readinessInitialDelaySeconds": 1,
"readinessPeriodSeconds": 2,
"resources": {
"limits": {
"cpu": "2000m",
"memory": "1024Mi"
},
"requests": {
"cpu": "100m",
"memory": "128Mi"
}
},
"statusPort": 15020,
"tracer": "zipkin"
},
"proxy_init": {
"image": "proxyv2",
"resources": {
"limits": {
"cpu": "2000m",
"memory": "1024Mi"
},
"requests": {
"cpu": "10m",
"memory": "10Mi"
}
}
},
"sds": {
"token": {
"aud": "istio-ca"
}
},
"sts": {
"servicePort": 0
},
"tracer": {
"datadog": {},
"lightstep": {},
"stackdriver": {},
"zipkin": {}
},
"useMCP": false
},
"istiodRemote": {
"injectionURL": ""
},
"pilot": {
"autoscaleEnabled": true,
"autoscaleMax": 5,
"autoscaleMin": 1,
"configMap": true,
"cpu": {
"targetAverageUtilization": 80
},
"deploymentLabels": null,
"enableProtocolSniffingForInbound": true,
"enableProtocolSniffingForOutbound": true,
"env": {},
"image": "pilot",
"keepaliveMaxServerConnectionAge": "30m",
"nodeSelector": {},
"podLabels": {},
"replicaCount": 1,
"traceSampling": 1
},
"telemetry": {
"enabled": true,
"v2": {
"enabled": true,
"metadataExchange": {
"wasmEnabled": false
},
"prometheus": {
"enabled": true,
"wasmEnabled": false
},
"stackdriver": {
"configOverride": {},
"enabled": false,
"logging": false,
"monitoring": false,
"topology": false
}
}
}
}
}
}
11. 트레이싱 설정 적용을 위한 Istio 재설치
1
root@myk8s-control-plane:/# istioctl install -y -f install-istio-tracing-zipkin.yaml
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
✔ Istio core installed
✔ Istiod installed
✔ Ingress gateways installed
- Pruning removed resources Removed Deployment:istio-system:istio-egressgateway.
Removed Service:istio-system:istio-egressgateway.
Removed ServiceAccount:istio-system:istio-egressgateway-service-account.
Removed RoleBinding:istio-system:istio-egressgateway-sds.
Removed Role:istio-system:istio-egressgateway-sds.
Removed PodDisruptionBudget:istio-system:istio-egressgateway.
✔ Installation complete Making this installation the default for injection and validation.
Thank you for installing Istio 1.17. Please take a few minutes to tell us about your install/upgrade experience! https://forms.gle/hMHGiwZHPU7UQRWe9
12. Istio ConfigMap을 통해 최종 트레이싱 설정 확인
1
2
3
4
root@myk8s-control-plane:/# exit
exit
kubectl describe cm -n istio-system istio
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
Name: istio
Namespace: istio-system
Labels: install.operator.istio.io/owning-resource=unknown
install.operator.istio.io/owning-resource-namespace=istio-system
istio.io/rev=default
operator.istio.io/component=Pilot
operator.istio.io/managed=Reconcile
operator.istio.io/version=1.17.8
release=istio
Annotations: <none>
Data
====
mesh:
----
defaultConfig:
discoveryAddress: istiod.istio-system.svc:15012
proxyMetadata: {}
tracing:
sampling: 100
zipkin:
address: zipkin.istio-system:9411
enablePrometheusMerge: true
rootNamespace: istio-system
trustDomain: cluster.local
meshNetworks:
----
networks: {}
BinaryData
====
Events: <none>
🧬 기본 트레이싱 헤더 살펴보기 : 헤더와 ID 자동 주입 확인
1. Istio Gateway 및 VirtualService 정의 파일 내용 확인
1
cat ch8/tracing/thin-httpbin-virtualservice.yaml
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: coolstore-gateway
spec:
selector:
istio: ingressgateway # use istio default controller
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "webapp.istioinaction.io"
- "httpbin.istioinaction.io"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: thin-httbin-virtualservice
spec:
hosts:
- "httpbin.istioinaction.io"
gateways:
- coolstore-gateway
http:
- route:
- destination:
host: httpbin.org
---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: external-httpbin-org
spec:
hosts:
- httpbin.org
ports:
- number: 80
name: http
protocol: HTTP
location: MESH_EXTERNAL
resolution: DNS
2. Gateway, VirtualService, ServiceEntry 리소스 배포
1
2
3
4
5
6
kubectl apply -n istioinaction -f ch8/tracing/thin-httpbin-virtualservice.yaml
# 결과
gateway.networking.istio.io/coolstore-gateway configured
virtualservice.networking.istio.io/thin-httbin-virtualservice created
serviceentry.networking.istio.io/external-httpbin-org created
3. httpbin 테스트용 도메인 /etc/hosts에 등록
1
2
3
4
echo "127.0.0.1 httpbin.istioinaction.io" | sudo tee -a /etc/hosts
# 결과
127.0.0.1 httpbin.istioinaction.io
1
cat /etc/hosts | tail -n 5
✅ 출력
1
2
3
4
5
# Test
127.0.0.1 catalog.istioinaction.io
127.0.0.1 webapp.istioinaction.io
127.0.0.1 simple-web.istioinaction.io
127.0.0.1 httpbin.istioinaction.io
4. httpbin으로의 요청 시 전달된 헤더 확인
1
curl -s http://httpbin.istioinaction.io:30000/headers | jq
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
{
"headers": {
"Accept": "*/*",
"Host": "httpbin.istioinaction.io",
"User-Agent": "curl/8.13.0",
"X-Amzn-Trace-Id": "Root=1-68162090-76ae2aa6484988dc118ada3e",
"X-B3-Sampled": "1",
"X-B3-Spanid": "f53aa7b3c1e107cf",
"X-B3-Traceid": "85e2b6c83d3d05e4f53aa7b3c1e107cf",
"X-Envoy-Attempt-Count": "1",
"X-Envoy-Decorator-Operation": "httpbin.org:80/*",
"X-Envoy-Internal": "true",
"X-Envoy-Peer-Metadata": "ChQKDkFQUF9DT05UQUlORVJTEgIaAAoaCgpDTFVTVEVSX0lEEgwaCkt1YmVybmV0ZXMKHAoMSU5TVEFOQ0VfSVBTEgwaCjEwLjEwLjAuMTcKGQoNSVNUSU9fVkVSU0lPThIIGgYxLjE3LjgKnAMKBkxBQkVMUxKRAyqOAwodCgNhcHASFhoUaXN0aW8taW5ncmVzc2dhdGV3YXkKEwoFY2hhcnQSChoIZ2F0ZXdheXMKFAoIaGVyaXRhZ2USCBoGVGlsbGVyCjYKKWluc3RhbGwub3BlcmF0b3IuaXN0aW8uaW8vb3duaW5nLXJlc291cmNlEgkaB3Vua25vd24KGQoFaXN0aW8SEBoOaW5ncmVzc2dhdGV3YXkKGQoMaXN0aW8uaW8vcmV2EgkaB2RlZmF1bHQKMAobb3BlcmF0b3IuaXN0aW8uaW8vY29tcG9uZW50EhEaD0luZ3Jlc3NHYXRld2F5cwoSCgdyZWxlYXNlEgcaBWlzdGlvCjkKH3NlcnZpY2UuaXN0aW8uaW8vY2Fub25pY2FsLW5hbWUSFhoUaXN0aW8taW5ncmVzc2dhdGV3YXkKLwojc2VydmljZS5pc3Rpby5pby9jYW5vbmljYWwtcmV2aXNpb24SCBoGbGF0ZXN0CiIKF3NpZGVjYXIuaXN0aW8uaW8vaW5qZWN0EgcaBWZhbHNlChoKB01FU0hfSUQSDxoNY2x1c3Rlci5sb2NhbAouCgROQU1FEiYaJGlzdGlvLWluZ3Jlc3NnYXRld2F5LTk5NmJjNmJiNi04bnM3NAobCglOQU1FU1BBQ0USDhoMaXN0aW8tc3lzdGVtCl0KBU9XTkVSElQaUmt1YmVybmV0ZXM6Ly9hcGlzL2FwcHMvdjEvbmFtZXNwYWNlcy9pc3Rpby1zeXN0ZW0vZGVwbG95bWVudHMvaXN0aW8taW5ncmVzc2dhdGV3YXkKFwoRUExBVEZPUk1fTUVUQURBVEESAioACicKDVdPUktMT0FEX05BTUUSFhoUaXN0aW8taW5ncmVzc2dhdGV3YXk=",
"X-Envoy-Peer-Metadata-Id": "router~10.10.0.17~istio-ingressgateway-996bc6bb6-8ns74.istio-system~istio-system.svc.cluster.local"
}
}
5. Envoy 헤더 디코딩을 위한 base64 문자열 복호화
1
echo "ChQKDkFQUF9DT05UQUlORVJTEgIaAAoaCgpDTFVTVEVSX0lEEgwaCkt1YmVybmV0ZXMKHAoMSU5TVEFOQ0VfSVBTEgwaCjEwLjEwLjAuMTcKGQoNSVNUSU9fVkVSU0lPThIIGgYxLjE3LjgKnAMKBkxBQkVMUxKRAyqOAwodCgNhcHASFhoUaXN0aW8taW5ncmVzc2dhdGV3YXkKEwoFY2hhcnQSChoIZ2F0ZXdheXMKFAoIaGVyaXRhZ2USCBoGVGlsbGVyCjYKKWluc3RhbGwub3BlcmF0b3IuaXN0aW8uaW8vb3duaW5nLXJlc291cmNlEgkaB3Vua25vd24KGQoFaXN0aW8SEBoOaW5ncmVzc2dhdGV3YXkKGQoMaXN0aW8uaW8vcmV2EgkaB2RlZmF1bHQKMAobb3BlcmF0b3IuaXN0aW8uaW8vY29tcG9uZW50EhEaD0luZ3Jlc3NHYXRld2F5cwoSCgdyZWxlYXNlEgcaBWlzdGlvCjkKH3NlcnZpY2UuaXN0aW8uaW8vY2Fub25pY2FsLW5hbWUSFhoUaXN0aW8taW5ncmVzc2dhdGV3YXkKLwojc2VydmljZS5pc3Rpby5pby9jYW5vbmljYWwtcmV2aXNpb24SCBoGbGF0ZXN0CiIKF3NpZGVjYXIuaXN0aW8uaW8vaW5qZWN0EgcaBWZhbHNlChoKB01FU0hfSUQSDxoNY2x1c3Rlci5sb2NhbAouCgROQU1FEiYaJGlzdGlvLWluZ3Jlc3NnYXRld2F5LTk5NmJjNmJiNi04bnM3NAobCglOQU1FU1BBQ0USDhoMaXN0aW8tc3lzdGVtCl0KBU9XTkVSElQaUmt1YmVybmV0ZXM6Ly9hcGlzL2FwcHMvdjEvbmFtZXNwYWNlcy9pc3Rpby1zeXN0ZW0vZGVwbG95bWVudHMvaXN0aW8taW5ncmVzc2dhdGV3YXkKFwoRUExBVEZPUk1fTUVUQURBVEESAioACicKDVdPUktMT0FEX05BTUUSFhoUaXN0aW8taW5ncmVzc2dhdGV3YXk=" | base64 -d
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
APP_CONTAINERS
CLUSTER_ID
Kubernetes
INSTANCE_IPS
10.10.0.17
ISTIO_VERSIO1.17.8
�
LABELS�*�
appistio-ingressgateway
chart
gateways
heritagTiller
6
)install.operator.istio.io/owning-resource unknown
istioingressgateway
istio.io/rev default
0
perator.istio.io/componentIngressGateways
releaseistio
9
service.istio.io/canonical-nameistio-ingressgateway
/
#service.istio.io/canonical-revisiolatest
"
sidecar.istio.io/injectfalse
cluster.local
.
NAME&$istio-ingressgateway-996bc6bb6-8ns74
NAMESPACE
istio-system
]
OWNERTRkubernetes://apis/apps/v1/namespaces/istio-system/deployments/istio-ingressgateway
PLATFORM_METADATA*
'
WORKLOAD_NAMEistio-ingressgateway
🎯 메시의 트레이스 샘플링 비율 조정하기
1. 메시 샘플링 비율 설정 변경 (100 → 10)
1
kubectl edit -n istio-system cm istio
✅ 출력 
1
configmap/istio edited
2. 샘플링 적용을 위한 IngressGateway 재시작
1
2
3
4
kubectl rollout restart deploy -n istio-system istio-ingressgateway
# 결과
deployment.apps/istio-ingressgateway restarted
3. 샘플링 반영 후 트래픽 테스트
1
curl -s http://webapp.istioinaction.io:30000/api/catalog | jq
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
[
{
"id": 1,
"color": "amber",
"department": "Eyewear",
"name": "Elinor Glasses",
"price": "282.00"
},
{
"id": 2,
"color": "cyan",
"department": "Clothing",
"name": "Atlas Shirt",
"price": "127.00"
},
{
"id": 3,
"color": "teal",
"department": "Clothing",
"name": "Small Metal Shoes",
"price": "232.00"
},
{
"id": 4,
"color": "red",
"department": "Watches",
"name": "Red Dragon Watch",
"price": "232.00"
}
]
📌 클라이언트에서 트레이싱 강제하기
1. x-envoy-force-trace 헤더로 강제 트레이싱 요청
1
curl -s -H "x-envoy-force-trace: true" http://webapp.istioinaction.io:30000/api/catalog -v
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
* Host webapp.istioinaction.io:30000 was resolved.
* IPv6: (none)
* IPv4: 127.0.0.1
* Trying 127.0.0.1:30000...
* Connected to webapp.istioinaction.io (127.0.0.1) port 30000
* using HTTP/1.x
> GET /api/catalog HTTP/1.1
> Host: webapp.istioinaction.io:30000
> User-Agent: curl/8.13.0
> Accept: */*
> x-envoy-force-trace: true
>
* Request completely sent off
< HTTP/1.1 200 OK
< content-length: 357
< content-type: application/json; charset=utf-8
< date: Sat, 03 May 2025 14:11:15 GMT
< x-envoy-upstream-service-time: 3
< server: istio-envoy
< x-request-id: 24f86a78-f602-a22a-bc9a-cd37a14c37b2
<
* Connection #0 to host webapp.istioinaction.io left intact
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]
🏷️ 트레이스의 태그 커스터마이징하기
1. webapp 배포 YAML에서 커스텀 트레이싱 태그 확인
1
cat ch8/webapp-deployment-zipkin-tag.yaml
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: webapp
name: webapp
spec:
replicas: 1
selector:
matchLabels:
app: webapp
template:
metadata:
annotations:
proxy.istio.io/config: |
tracing:
sampling: 100
customTags: # 커스텀 태그의 키
custom_tag:
literal:
value: "Test Tag" # 커스텀 태그의 값
zipkin:
address: zipkin.istio-system:9411
labels:
app: webapp
spec:
containers:
- env:
- name: KUBERNETES_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: CATALOG_SERVICE_HOST
value: catalog.istioinaction
- name: CATALOG_SERVICE_PORT
value: "80"
- name: FORUM_SERVICE_HOST
value: forum.istioinaction
- name: FORUM_SERVICE_PORT
value: "80"
image: istioinaction/webapp:latest
imagePullPolicy: IfNotPresent
name: webapp
ports:
- containerPort: 8080
name: http
protocol: TCP
securityContext:
privileged: false
2. webapp 배포에 커스텀 태그 설정 적용
1
2
3
4
kubectl apply -n istioinaction -f ch8/webapp-deployment-zipkin-tag.yaml
# 결과
deployment.apps/webapp configured
3. 커스텀 태그 반영 확인을 위한 트래픽 생성
1
for in in {1..10}; do curl -s http://webapp.istioinaction.io:30000/api/catalog ; sleep 0.5; done
✅ 출력
1
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}][{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}][{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}][{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}][{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}][{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}][{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}][{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}][{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}][{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]
🧩 백엔드 분산 트레이싱 엔진 커스터마이징하기
1. webapp의 현재 트레이싱 부트스트랩 설정 확인
1
2
3
docker exec -it myk8s-control-plane bash
root@myk8s-control-plane:/# istioctl pc bootstrap -n istioinaction deploy/webapp -o json | jq .bootstrap.tracing
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
{
"http": {
"name": "envoy.tracers.zipkin",
"typedConfig": {
"@type": "type.googleapis.com/envoy.config.trace.v3.ZipkinConfig",
"collectorCluster": "zipkin",
"collectorEndpoint": "/api/v2/spans",
"traceId128bit": true,
"sharedSpanContext": false,
"collectorEndpointVersion": "HTTP_JSON"
}
}
}
2. 사용자 정의 트레이싱 부트스트랩 설정 ConfigMap 작성 확인
1
2
3
4
root@myk8s-control-plane:/# exit
exit
cat ch8/istio-custom-bootstrap.yaml
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
apiVersion: v1
kind: ConfigMap
metadata:
name: istio-custom-zipkin
data:
custom_bootstrap.json: |
{
"tracing": {
"http": {
"name": "envoy.tracers.zipkin",
"typedConfig": {
"@type": "type.googleapis.com/envoy.config.trace.v3.ZipkinConfig",
"collectorCluster": "zipkin",
"collectorEndpoint": "/zipkin/api/v1/spans",
"traceId128bit": "true",
"collectorEndpointVersion": "HTTP_JSON"
}
}
}
}
3. 사용자 정의 부트스트랩 ConfigMap 생성
1
2
3
4
kubectl apply -n istioinaction -f ch8/istio-custom-bootstrap.yaml
# 결과
configmap/istio-custom-zipkin created
4. webapp에 사용자 정의 부트스트랩 설정 확인
1
cat ch8/webapp-deployment-custom-boot.yaml
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: webapp
name: webapp
spec:
replicas: 1
selector:
matchLabels:
app: webapp
template:
metadata:
annotations:
sidecar.istio.io/bootstrapOverride: "istio-custom-zipkin"
proxy.istio.io/config: |
tracing:
sampling: 10
zipkin:
address: zipkin.istio-system:9411
labels:
app: webapp
spec:
containers:
- env:
- name: KUBERNETES_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: CATALOG_SERVICE_HOST
value: catalog.istioinaction
- name: CATALOG_SERVICE_PORT
value: "80"
- name: FORUM_SERVICE_HOST
value: forum.istioinaction
- name: FORUM_SERVICE_PORT
value: "80"
image: istioinaction/webapp:latest
imagePullPolicy: IfNotPresent
name: webapp
ports:
- containerPort: 8080
name: http
protocol: TCP
securityContext:
privileged: false
5. 사용자 정의 부트스트랩이 적용된 webapp 재배포
1
2
3
kubectl apply -n istioinaction -f ch8/webapp-deployment-custom-boot.yaml
deployment.apps/webapp configured
6. webapp에 적용된 사용자 정의 부트스트랩 설정 확인
1
2
3
docker exec -it myk8s-control-plane bash
root@myk8s-control-plane:/# istioctl pc bootstrap -n istioinaction deploy/webapp -o json | jq .bootstrap.tracing
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
{
"http": {
"name": "envoy.tracers.zipkin",
"typedConfig": {
"@type": "type.googleapis.com/envoy.config.trace.v3.ZipkinConfig",
"collectorCluster": "zipkin",
"collectorEndpoint": "/zipkin/api/v1/spans",
"traceId128bit": true,
"collectorEndpointVersion": "HTTP_JSON"
}
}
}
올바르지 않은 수집 엔드포인트(/zipkin/api/v1/spans)가 설정되어 트레이스 데이터가 정상적으로 수집되지 않음
7. 기본 부트스트랩 설정으로 webapp 설정 원복
1
2
3
4
5
6
kubectl apply -n istioinaction -f services/webapp/kubernetes/webapp.yaml
# 결과
serviceaccount/webapp unchanged
service/webapp unchanged
deployment.apps/webapp configured
📈 키알리를 이용한 시각화
1. Helm 저장소 추가 및 갱신
1
2
helm repo add kiali https://kiali.org/helm-charts
helm repo update
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
"kiali" has been added to your repositories
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "metrics-server" chart repository
...Successfully got an update from the "kedacore" chart repository
...Successfully got an update from the "hashicorp" chart repository
...Successfully got an update from the "eks" chart repository
...Successfully got an update from the "kiali" chart repository
...Successfully got an update from the "argo" chart repository
...Successfully got an update from the "grafana-charts" chart repository
...Successfully got an update from the "prometheus-community" chart repository
...Successfully got an update from the "geek-cookbook" chart repository
...Successfully got an update from the "flagger" chart repository
...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈
2. kiali-operator 설치
1
helm install --namespace kiali-operator --create-namespace --version 1.63.2 kiali-operator kiali/kiali-operator
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
NAME: kiali-operator
LAST DEPLOYED: Sat May 3 23:31:11 2025
NAMESPACE: kiali-operator
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Welcome to Kiali! For more details on Kiali, see: https://kiali.io
The Kiali Operator [v1.63.2] has been installed in namespace [kiali-operator]. It will be ready soon.
You have elected not to install a Kiali CR. You must first install a Kiali CR before you can access Kiali. The operator is watching all namespaces, so you can create the Kiali CR anywhere.
If you ever want to uninstall the Kiali Operator, remember to delete the Kiali CR first before uninstalling the operator to give the operator a chance to uninstall and remove all the Kiali Server resources.
(Helm: Chart=[kiali-operator], Release=[kiali-operator], Version=[1.63.2])
3. kiali-operator 파드 상태 확인
1
kubectl get pod -n kiali-operator
✅ 출력
1
2
NAME READY STATUS RESTARTS AGE
kiali-operator-584858fb7-ktnd9 0/1 ContainerCreating 0 42s
4. 키알리 인스턴스 정의 파일 확인
1
cat ch8/kiali.yaml
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
apiVersion: kiali.io/v1alpha1
kind: Kiali
metadata:
namespace: istio-system
name: kiali
spec:
istio_namespace: "istio-system"
istio_component_namespaces:
prometheus: prometheus
auth:
strategy: anonymous
deployment:
accessible_namespaces:
- '**'
external_services:
prometheus:
cache_duration: 10
cache_enabled: true
cache_expiration: 300
url: "http://prom-kube-prometheus-stack-prometheus.prometheus:9090"
tracing:
enabled: true
in_cluster_url: "http://tracing.istio-system:16685/jaeger"
use_grpc: true
5. 키알리 인스턴스 설치
1
2
3
4
kubectl apply -f ch8/kiali.yaml
# 결과
kiali.kiali.io/kiali created
6. 키알리 배포 및 서비스 리소스 상태 확인
1
kubectl get deploy,svc -n istio-system kiali
✅ 출력
1
2
3
4
5
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/kiali 0/1 1 0 29s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kiali ClusterIP 10.200.1.37 <none> 20001/TCP,9090/TCP 28s
7. Kiali 서비스 NodePort로 변경 (포트 30003)
1
2
3
4
kubectl patch svc -n istio-system kiali -p '{"spec": {"type": "NodePort", "ports": [{"port": 20001, "targetPort": 20001, "nodePort": 30003}]}}'
# 결과
service/kiali patched
8. Catalog API 반복 호출로 트래픽 생성
1
while true; do curl -s http://webapp.istioinaction.io:30000/api/catalog ; date "+%Y-%m-%d %H:%M:%S" ; sleep 1; echo; done
✅ 출력
1
2
3
4
5
6
7
8
9
10
11
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]2025-05-03 23:37:14
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]2025-05-03 23:37:15
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]2025-05-03 23:37:16
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]2025-05-03 23:37:17
[{"id":1,"color":"amber","department":"Eyewear","name":"Elinor Glasses","price":"282.00"},{"id":2,"color":"cyan","department":"Clothing","name":"Atlas Shirt","price":"127.00"},{"id":3,"color":"teal","department":"Clothing","name":"Small Metal Shoes","price":"232.00"},{"id":4,"color":"red","department":"Watches","name":"Red Dragon Watch","price":"232.00"}]2025-05-03 23:37:18
...
This post is licensed under CC BY 4.0 by the author.