Cilium 8주차 정리

Posted Sep 7, 2025

By GAGA

106 min read

🔧 실습 환경 구성

1. Vagrantfile 다운로드 및 가상머신 구성

curl -O https://raw.githubusercontent.com/gasida/vagrant-lab/refs/heads/main/cilium-study/8w/Vagrantfile
vagrant up

2. VM 부팅 타임아웃 오류 해결

(1) vagrant up 실행 중 하기 오류 발생

  
....
Timed out while waiting for the machine to boot. This means that
Vagrant was unable to communicate with the guest machine within
the configured ("config.vm.boot_timeout" value) time period.

If you look above, you should be able to see the error(s) that
Vagrant had when attempting to connect to the machine. These errors
are usually good hints as to what may be wrong.

If you're using a custom box, make sure that networking is properly
working and you're able to connect to the machine. It is a common
problem that networking isn't setup properly in these boxes.
Verify that authentication configurations are also setup properly,
as well.

If the box appears to be booting properly, you may want to increase
the timeout ("config.vm.boot_timeout") value.

(2) Vagrantfile 수정

  
# Variables
K8SV = '1.33.4-1.1' # Kubernetes Version : apt list -a kubelet , ex) 1.32.5-1.1
CONTAINERDV = '1.7.27-1' # Containerd Version : apt list -a containerd.io , ex) 1.6.33-1
CILIUMV = '1.18.1' # Cilium CNI Version : https://github.com/cilium/cilium/tags
N = 2 # max number of worker nodes
# Base Image  https://portal.cloud.hashicorp.com/vagrant/discover/bento/ubuntu-24.04
BOX_IMAGE = "bento/ubuntu-24.04"
BOX_VERSION = "202508.03.0"

Vagrant.configure("2") do |config|
  # 전역 설정
  config.vm.boot_timeout = 600  # 10분으로 증가
  config.vm.graceful_halt_timeout = 60
  
  #-ControlPlane Node
  config.vm.define "k8s-ctr" do |subconfig|
    subconfig.vm.box = BOX_IMAGE
    subconfig.vm.box_version = BOX_VERSION
    
    subconfig.vm.provider "virtualbox" do |vb|
      vb.customize ["modifyvm", :id, "--groups", "/Cilium-Lab"]
      vb.customize ["modifyvm", :id, "--nicpromisc2", "allow-all"]
      vb.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
      vb.customize ["modifyvm", :id, "--natdnsproxy1", "on"]
      vb.name = "k8s-ctr"
      vb.cpus = 4
      vb.memory = 3072  # 3GB로 증가
      vb.linked_clone = true
    end
    
    subconfig.vm.host_name = "k8s-ctr"
    subconfig.vm.network "private_network", ip: "192.168.10.100"
    subconfig.vm.network "forwarded_port", guest: 22, host: 60000, auto_correct: true, id: "ssh"
    subconfig.vm.synced_folder "./", "/vagrant", disabled: true
    
    subconfig.vm.provision "shell", path: "https://raw.githubusercontent.com/gasida/vagrant-lab/refs/heads/main/cilium-study/8w/init_cfg.sh", args: [ K8SV, CONTAINERDV ]
    subconfig.vm.provision "shell", path: "https://raw.githubusercontent.com/gasida/vagrant-lab/refs/heads/main/cilium-study/8w/k8s-ctr.sh", args: [ N, CILIUMV, K8SV ]
  end

  #-Worker Nodes Subnet1
  (1..N).each do |i|
    config.vm.define "k8s-w#{i}" do |subconfig|
      subconfig.vm.box = BOX_IMAGE
      subconfig.vm.box_version = BOX_VERSION
      
      subconfig.vm.provider "virtualbox" do |vb|
        vb.customize ["modifyvm", :id, "--groups", "/Cilium-Lab"]
        vb.customize ["modifyvm", :id, "--nicpromisc2", "allow-all"]
        vb.customize ["modifyvm", :id, "--natdnshostresolver1", "on"]
        vb.customize ["modifyvm", :id, "--natdnsproxy1", "on"]
        vb.name = "k8s-w#{i}"
        vb.cpus = 4
        vb.memory = 2560  # 2.5GB로 증가
        vb.linked_clone = true
      end
      
      subconfig.vm.host_name = "k8s-w#{i}"
      subconfig.vm.network "private_network", ip: "192.168.10.10#{i}"
      subconfig.vm.network "forwarded_port", guest: 22, host: "6000#{i}", auto_correct: true, id: "ssh"
      subconfig.vm.synced_folder "./", "/vagrant", disabled: true
      
      subconfig.vm.provision "shell", path: "https://raw.githubusercontent.com/gasida/vagrant-lab/refs/heads/main/cilium-study/8w/init_cfg.sh", args: [ K8SV, CONTAINERDV ]
      subconfig.vm.provision "shell", path: "https://raw.githubusercontent.com/gasida/vagrant-lab/refs/heads/main/cilium-study/8w/k8s-w.sh"
    end
  end
end

config.vm.boot_timeout = 600 (10분)으로 증가
config.vm.graceful_halt_timeout = 60 추가
컨트롤플레인 노드(k8s-ctr) → 메모리 3GB로 설정
워커 노드(k8s-w1, k8s-w2) → 메모리 2.5GB로 설정

(3) vagrant up 재시도

  
Bringing machine 'k8s-ctr' up with 'virtualbox' provider...
Bringing machine 'k8s-w1' up with 'virtualbox' provider...
Bringing machine 'k8s-w2' up with 'virtualbox' provider...
==> k8s-ctr: Cloning VM...
==> k8s-ctr: Matching MAC address for NAT networking...
==> k8s-ctr: Checking if box 'bento/ubuntu-24.04' version '202508.03.0' is up to date...
==> k8s-ctr: Setting the name of the VM: k8s-ctr
==> k8s-ctr: Clearing any previously set network interfaces...
==> k8s-ctr: Preparing network interfaces based on configuration...
    k8s-ctr: Adapter 1: nat
    k8s-ctr: Adapter 2: hostonly
==> k8s-ctr: Forwarding ports...
    k8s-ctr: 22 (guest) => 60000 (host) (adapter 1)
==> k8s-ctr: Running 'pre-boot' VM customizations...
==> k8s-ctr: Booting VM...
==> k8s-ctr: Waiting for machine to boot. This may take a few minutes...
    k8s-ctr: SSH address: 127.0.0.1:60000
    k8s-ctr: SSH username: vagrant
    k8s-ctr: SSH auth method: private key
    k8s-ctr: 
    k8s-ctr: Vagrant insecure key detected. Vagrant will automatically replace
    k8s-ctr: this with a newly generated keypair for better security.
    k8s-ctr: 
    k8s-ctr: Inserting generated public key within guest...
    k8s-ctr: Removing insecure key from the guest if it's present...
    k8s-ctr: Key inserted! Disconnecting and reconnecting using new SSH key...
==> k8s-ctr: Machine booted and ready!
==> k8s-ctr: Checking for guest additions in VM...
    k8s-ctr: The guest additions on this VM do not match the installed version of
    k8s-ctr: VirtualBox! In most cases this is fine, but in rare cases it can
    k8s-ctr: prevent things such as shared folders from working properly. If you see
    k8s-ctr: shared folder errors, please make sure the guest additions within the
    k8s-ctr: virtual machine match the version of VirtualBox you have installed on
    k8s-ctr: your host and reload your VM.
    k8s-ctr: 
    k8s-ctr: Guest Additions Version: 7.1.12
    k8s-ctr: VirtualBox Version: 7.2
==> k8s-ctr: Setting hostname...
==> k8s-ctr: Configuring and enabling network interfaces...
==> k8s-ctr: Running provisioner: shell...
    k8s-ctr: Running: /tmp/vagrant-shell20250902-480239-erzmme.sh
    k8s-ctr: >>>> Initial Config Start <<<<
    k8s-ctr: [TASK 1] Setting Profile & Bashrc
    k8s-ctr: [TASK 2] Disable AppArmor
    k8s-ctr: [TASK 3] Disable and turn off SWAP
    k8s-ctr: [TASK 4] Install Packages
    k8s-ctr: [TASK 5] Install Kubernetes components (kubeadm, kubelet and kubectl)
    k8s-ctr: [TASK 6] Install Packages & Helm
    k8s-ctr: [TASK 7] Install pwru
    k8s-ctr: >>>> Initial Config End <<<<
==> k8s-ctr: Running provisioner: shell...
    k8s-ctr: Running: /tmp/vagrant-shell20250902-480239-dnsuae.sh
    k8s-ctr: >>>> K8S Controlplane config Start <<<<
    k8s-ctr: [TASK 1] Initial Kubernetes
    k8s-ctr: [TASK 2] Setting kube config file
    k8s-ctr: [TASK 3] Source the completion
    k8s-ctr: [TASK 4] Alias kubectl to k
    k8s-ctr: [TASK 5] Install Kubectx & Kubens
    k8s-ctr: [TASK 6] Install Kubeps & Setting PS1
    k8s-ctr: [TASK 7] Install Cilium CNI
    k8s-ctr: [TASK 8] Install Cilium / Hubble CLI
    k8s-ctr: cilium
    k8s-ctr: hubble
    k8s-ctr: [TASK 9] Remove node taint
    k8s-ctr: node/k8s-ctr untainted
    k8s-ctr: [TASK 10] local DNS with hosts file
    k8s-ctr: [TASK 11] Dynamically provisioning persistent local storage with Kubernetes
    k8s-ctr: [TASK 12] Install Prometheus & Grafana
    k8s-ctr: [TASK 13] Install Metrics-server
    k8s-ctr: [TASK 14] Install k9s
    k8s-ctr: >>>> K8S Controlplane Config End <<<<
==> k8s-w1: Cloning VM...
==> k8s-w1: Matching MAC address for NAT networking...
==> k8s-w1: Checking if box 'bento/ubuntu-24.04' version '202508.03.0' is up to date...
==> k8s-w1: Setting the name of the VM: k8s-w1
==> k8s-w1: Clearing any previously set network interfaces...
==> k8s-w1: Preparing network interfaces based on configuration...
    k8s-w1: Adapter 1: nat
    k8s-w1: Adapter 2: hostonly
==> k8s-w1: Forwarding ports...
    k8s-w1: 22 (guest) => 60001 (host) (adapter 1)
==> k8s-w1: Running 'pre-boot' VM customizations...
==> k8s-w1: Booting VM...
==> k8s-w1: Waiting for machine to boot. This may take a few minutes...
    k8s-w1: SSH address: 127.0.0.1:60001
    k8s-w1: SSH username: vagrant
    k8s-w1: SSH auth method: private key
    k8s-w1: 
    k8s-w1: Vagrant insecure key detected. Vagrant will automatically replace
    k8s-w1: this with a newly generated keypair for better security.
    k8s-w1: 
    k8s-w1: Inserting generated public key within guest...
    k8s-w1: Removing insecure key from the guest if it's present...
    k8s-w1: Key inserted! Disconnecting and reconnecting using new SSH key...
==> k8s-w1: Machine booted and ready!
==> k8s-w1: Checking for guest additions in VM...
    k8s-w1: The guest additions on this VM do not match the installed version of
    k8s-w1: VirtualBox! In most cases this is fine, but in rare cases it can
    k8s-w1: prevent things such as shared folders from working properly. If you see
    k8s-w1: shared folder errors, please make sure the guest additions within the
    k8s-w1: virtual machine match the version of VirtualBox you have installed on
    k8s-w1: your host and reload your VM.
    k8s-w1: 
    k8s-w1: Guest Additions Version: 7.1.12
    k8s-w1: VirtualBox Version: 7.2
==> k8s-w1: Setting hostname...
==> k8s-w1: Configuring and enabling network interfaces...
==> k8s-w1: Running provisioner: shell...
    k8s-w1: Running: /tmp/vagrant-shell20250902-480239-takpig.sh
    k8s-w1: >>>> Initial Config Start <<<<
    k8s-w1: [TASK 1] Setting Profile & Bashrc
    k8s-w1: [TASK 2] Disable AppArmor
    k8s-w1: [TASK 3] Disable and turn off SWAP
    k8s-w1: [TASK 4] Install Packages
    k8s-w1: [TASK 5] Install Kubernetes components (kubeadm, kubelet and kubectl)
    k8s-w1: [TASK 6] Install Packages & Helm
    k8s-w1: [TASK 7] Install pwru
    k8s-w1: >>>> Initial Config End <<<<
==> k8s-w1: Running provisioner: shell...
    k8s-w1: Running: /tmp/vagrant-shell20250902-480239-9ldmc2.sh
    k8s-w1: >>>> K8S Node config Start <<<<
    k8s-w1: [TASK 1] K8S Controlplane Join
    k8s-w1: >>>> K8S Node config End <<<<
==> k8s-w2: Cloning VM...
==> k8s-w2: Matching MAC address for NAT networking...
==> k8s-w2: Checking if box 'bento/ubuntu-24.04' version '202508.03.0' is up to date...
==> k8s-w2: Setting the name of the VM: k8s-w2
==> k8s-w2: Clearing any previously set network interfaces...
==> k8s-w2: Preparing network interfaces based on configuration...
    k8s-w2: Adapter 1: nat
    k8s-w2: Adapter 2: hostonly
==> k8s-w2: Forwarding ports...
    k8s-w2: 22 (guest) => 60002 (host) (adapter 1)
==> k8s-w2: Running 'pre-boot' VM customizations...
==> k8s-w2: Booting VM...
==> k8s-w2: Waiting for machine to boot. This may take a few minutes...
    k8s-w2: SSH address: 127.0.0.1:60002
    k8s-w2: SSH username: vagrant
    k8s-w2: SSH auth method: private key
    k8s-w2: 
    k8s-w2: Vagrant insecure key detected. Vagrant will automatically replace
    k8s-w2: this with a newly generated keypair for better security.
    k8s-w2: 
    k8s-w2: Inserting generated public key within guest...
    k8s-w2: Removing insecure key from the guest if it's present...
    k8s-w2: Key inserted! Disconnecting and reconnecting using new SSH key...
==> k8s-w2: Machine booted and ready!
==> k8s-w2: Checking for guest additions in VM...
    k8s-w2: The guest additions on this VM do not match the installed version of
    k8s-w2: VirtualBox! In most cases this is fine, but in rare cases it can
    k8s-w2: prevent things such as shared folders from working properly. If you see
    k8s-w2: shared folder errors, please make sure the guest additions within the
    k8s-w2: virtual machine match the version of VirtualBox you have installed on
    k8s-w2: your host and reload your VM.
    k8s-w2: 
    k8s-w2: Guest Additions Version: 7.1.12
    k8s-w2: VirtualBox Version: 7.2
==> k8s-w2: Setting hostname...
==> k8s-w2: Configuring and enabling network interfaces...
==> k8s-w2: Running provisioner: shell...
    k8s-w2: Running: /tmp/vagrant-shell20250902-480239-t7hfv1.sh
    k8s-w2: >>>> Initial Config Start <<<<
    k8s-w2: [TASK 1] Setting Profile & Bashrc
    k8s-w2: [TASK 2] Disable AppArmor
    k8s-w2: [TASK 3] Disable and turn off SWAP
    k8s-w2: [TASK 4] Install Packages
    k8s-w2: [TASK 5] Install Kubernetes components (kubeadm, kubelet and kubectl)
    k8s-w2: [TASK 6] Install Packages & Helm
    k8s-w2: [TASK 7] Install pwru
    k8s-w2: >>>> Initial Config End <<<<
==> k8s-w2: Running provisioner: shell...
    k8s-w2: Running: /tmp/vagrant-shell20250902-480239-oir40.sh
    k8s-w2: >>>> K8S Node config Start <<<<
    k8s-w2: [TASK 1] K8S Controlplane Join
    k8s-w2: >>>> K8S Node config End <<<<

3. 모니터링 및 관리 도구 접속 확인

(1) 프로메테우스: http://192.168.10.100:30001

(2) 그라파나: http://192.168.10.100:30002

(3) 허블 UI: http://192.168.10.100:30003/

📑 Identity

https://docs.cilium.io/en/stable/gettingstarted/terminology/#identity
모든 엔드포인트에 ID가 할당. ID는 Labels 과 클러스터 내에 유일한 ID로 구성
엔드포인트에는 Security Relevant Labels에 일치하는 ID가 할당
엔드포인트들이 동일한 Security Relevant Labels 사용 시 동일한 ID를 공유

1. CiliumEndpoint 리소스로 엔드포인트 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl get ciliumendpoints.cilium.io -n kube-system

✅ 출력

NAME                              SECURITY IDENTITY   ENDPOINT STATE   IPV4           IPV6
coredns-674b8bbfcf-7xcrj          324                 ready            172.20.0.224   
coredns-674b8bbfcf-vxhtv          324                 ready            172.20.0.155   
hubble-relay-fdd49b976-g22hx      54079               ready            172.20.0.181   
hubble-ui-655f947f96-xtwg4        25043               ready            172.20.0.215   
metrics-server-5dd7b49d79-dw9x6   54044               ready            172.20.0.218   

각 파드 단위 엔드포인트와 할당된 SECURITY IDENTITY, 상태, IP 정보 확인 가능
예시에서 coredns 파드 2개가 ID(324) 를 공유 → 동일한 보안 정책을 적용받음을 의미

2. CiliumIdentity 리소스 조회

(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl get ciliumidentities.cilium.io 

✅ 출력

NAME    NAMESPACE            AGE
 local-path-storage   107m
 cilium-monitoring    107m
 kube-system          107m
   kube-system          107m
 cilium-monitoring    107m
 kube-system          107m
 kube-system          107m

kubectl get ciliumidentities.cilium.io 54044 -o yaml | yq

✅ 출력

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl get ciliumidentities.cilium.io 60202 -o yaml | yq
{
  "apiVersion": "cilium.io/v2",
  "kind": "CiliumIdentity",
  "metadata": {
    "creationTimestamp": "2025-09-06T05:47:17Z",
    "generation": 1,
    "labels": {
      "io.kubernetes.pod.namespace": "kube-system"
    },
    "name": "54044",
    "resourceVersion": "842",
    "uid": "f8d8931f-158c-453d-9945-589f2d9eab15"
  },
  "security-labels": {
    "k8s:app.kubernetes.io/instance": "metrics-server",
    "k8s:app.kubernetes.io/name": "metrics-server",
    "k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name": "kube-system",
    "k8s:io.cilium.k8s.policy.cluster": "default",
    "k8s:io.cilium.k8s.policy.serviceaccount": "metrics-server",
    "k8s:io.kubernetes.pod.namespace": "kube-system"
  }
}

54044 ID는 metrics-server 파드에 해당, Pod 라벨 및 자동 부착된 Cilium 라벨들이 표시됨

3. Cilium Agent에서 Identity 라벨 상세 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it -n kube-system ds/cilium -- cilium identity list

✅ 출력

  
ID      LABELS
1       reserved:host
        reserved:kube-apiserver
2       reserved:world
3       reserved:unmanaged
4       reserved:health
5       reserved:init
6       reserved:remote-node
7       reserved:kube-apiserver
        reserved:remote-node
8       reserved:ingress
9       reserved:world-ipv4
10      reserved:world-ipv6
324     k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=kube-system
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=coredns
        k8s:io.kubernetes.pod.namespace=kube-system
        k8s:k8s-app=kube-dns
12931   k8s:app=local-path-provisioner
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=local-path-storage
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=local-path-provisioner-service-account
        k8s:io.kubernetes.pod.namespace=local-path-storage
20866   k8s:app=grafana
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=cilium-monitoring
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=default
        k8s:io.kubernetes.pod.namespace=cilium-monitoring
25043   k8s:app.kubernetes.io/name=hubble-ui
        k8s:app.kubernetes.io/part-of=cilium
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=kube-system
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=hubble-ui
        k8s:io.kubernetes.pod.namespace=kube-system
        k8s:k8s-app=hubble-ui
42541   k8s:app=prometheus
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=cilium-monitoring
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=prometheus-k8s
        k8s:io.kubernetes.pod.namespace=cilium-monitoring
54044   k8s:app.kubernetes.io/instance=metrics-server
        k8s:app.kubernetes.io/name=metrics-server
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=kube-system
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=metrics-server
        k8s:io.kubernetes.pod.namespace=kube-system
54079   k8s:app.kubernetes.io/name=hubble-relay
        k8s:app.kubernetes.io/part-of=cilium
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=kube-system
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=hubble-relay
        k8s:io.kubernetes.pod.namespace=kube-system
        k8s:k8s-app=hubble-relay

Cilium Agent 내부에서 Identity와 라벨 매핑 정보 확인 가능
324 ID에 coredns 관련 라벨이 붙어 있는 것을 확인할 수 있음

4. 라벨 변경 후 Identity 재할당 확인

(1) coredns 파드 초기 상태 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl get pod -n kube-system -l k8s-app=kube-dns --show-labels

✅ 출력

  
NAME                       READY   STATUS    RESTARTS   AGE    LABELS
coredns-674b8bbfcf-7xcrj   1/1     Running   0          113m   k8s-app=kube-dns,pod-template-hash=674b8bbfcf
coredns-674b8bbfcf-vxhtv   1/1     Running   0          113m   k8s-app=kube-dns,pod-template-hash=674b8bbfcf

k8s-app=kube-dns, pod-template-hash=674b8bbfcf

(2) k9s 진입

(3) 라벨 추가(app: testing)

(4) 재기동

(5) 새로운 라벨 반영 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl get pod -n kube-system -l k8s-app=kube-dns --show-labels

✅ 출력

  
NAME                      READY   STATUS    RESTARTS   AGE   LABELS
coredns-99ff8c6c4-6zg5v   1/1     Running   0          33s   app=testing,k8s-app=kube-dns,pod-template-hash=99ff8c6c4
coredns-99ff8c6c4-fpx2g   1/1     Running   0          33s   app=testing,k8s-app=kube-dns,pod-template-hash=99ff8c6c4

대략 30초 정도 이후, LABEL(app=testing)이 반영되었다.

(6) 새로운 ID 할당 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it -n kube-system ds/cilium -- cilium identity list

✅ 출력

  
ID      LABELS
1       reserved:host
        reserved:kube-apiserver
2       reserved:world
3       reserved:unmanaged
4       reserved:health
5       reserved:init
6       reserved:remote-node
7       reserved:kube-apiserver
        reserved:remote-node
8       reserved:ingress
9       reserved:world-ipv4
10      reserved:world-ipv6
324     k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=kube-system
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=coredns
        k8s:io.kubernetes.pod.namespace=kube-system
        k8s:k8s-app=kube-dns
12931   k8s:app=local-path-provisioner
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=local-path-storage
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=local-path-provisioner-service-account
        k8s:io.kubernetes.pod.namespace=local-path-storage
18194   k8s:app=testing
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=kube-system
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=coredns
        k8s:io.kubernetes.pod.namespace=kube-system
        k8s:k8s-app=kube-dns
20866   k8s:app=grafana
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=cilium-monitoring
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=default
        k8s:io.kubernetes.pod.namespace=cilium-monitoring
25043   k8s:app.kubernetes.io/name=hubble-ui
        k8s:app.kubernetes.io/part-of=cilium
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=kube-system
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=hubble-ui
        k8s:io.kubernetes.pod.namespace=kube-system
        k8s:k8s-app=hubble-ui
42541   k8s:app=prometheus
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=cilium-monitoring
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=prometheus-k8s
        k8s:io.kubernetes.pod.namespace=cilium-monitoring
54044   k8s:app.kubernetes.io/instance=metrics-server
        k8s:app.kubernetes.io/name=metrics-server
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=kube-system
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=metrics-server
        k8s:io.kubernetes.pod.namespace=kube-system
54079   k8s:app.kubernetes.io/name=hubble-relay
        k8s:app.kubernetes.io/part-of=cilium
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=kube-system
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=hubble-relay
        k8s:io.kubernetes.pod.namespace=kube-system
        k8s:k8s-app=hubble-relay

  
18194   k8s:app=testing
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=kube-system
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=coredns
        k8s:io.kubernetes.pod.namespace=kube-system
        k8s:k8s-app=kube-dns

기존 324 ID 대신 18194 ID가 새롭게 생성되어 적용됨
라벨 변경 시 ID가 변경되며 보안 정책도 자동으로 갱신됨

5. Special Identities 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it -n kube-system ds/cilium -- cilium identity list

✅ 출력

ID      LABELS
     reserved:host
        reserved:kube-apiserver
     reserved:world
     reserved:unmanaged
     reserved:health
     reserved:init
     reserved:remote-node
     reserved:kube-apiserver
        reserved:remote-node
     reserved:ingress
     reserved:world-ipv4
    reserved:world-ipv6
...

Cilium은 모든 엔드포인트(Pod 포함)에 고유한 Security Identity(ID) 를 부여함
그러나 클러스터 외부 엔드포인트나 Cilium이 직접 관리하지 않는 대상과의 통신을 허용하기 위해 특수 예약 ID(Reserved Identity)를 제공함
이러한 예약된 ID에는 reserved 접두사가 붙음

🚀 [Lab1] DNS 기반 보안 정책

https://docs.cilium.io/en/stable/security/dns/

1. 데모 애플리케이션 배포

(1) 배포 파일 생성

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# cat << EOF > dns-sw-app.yaml
apiVersion: v1
kind: Pod
metadata:
  name: mediabot
  labels:
    org: empire
    class: mediabot
    app: mediabot
spec:
  containers:
  - name: mediabot
    image: quay.io/cilium/json-mock:v1.3.8@sha256:5aad04835eda9025fe4561ad31be77fd55309af8158ca8663a72f6abb78c2603
EOF

L7 Envoy Proxy가 아닌 Cilium Agent 내장 DNS Proxy 로 동작하는 DNS 정책 제어 실습
json-mock 컨테이너 배포 후 api.github.com에 외부 통신 테스트
공식 예제에는 app: mediabot 라벨이 없어 직접 추가 후 배포

(2) 파드 생성

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl apply -f dns-sw-app.yaml

# 결과
pod/mediabot created

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl wait pod/mediabot --for=condition=Ready

# 결과
pod/mediabot condition met

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it -n kube-system ds/cilium -- cilium identity list
ID      LABELS
1       reserved:host
        reserved:kube-apiserver
2       reserved:world
3       reserved:unmanaged
4       reserved:health
5       reserved:init
6       reserved:remote-node
7       reserved:kube-apiserver
        reserved:remote-node
8       reserved:ingress
9       reserved:world-ipv4
10      reserved:world-ipv6
324     k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=kube-system
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=coredns
        k8s:io.kubernetes.pod.namespace=kube-system
        k8s:k8s-app=kube-dns
12931   k8s:app=local-path-provisioner
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=local-path-storage
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=local-path-provisioner-service-account
        k8s:io.kubernetes.pod.namespace=local-path-storage
18194   k8s:app=testing
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=kube-system
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=coredns
        k8s:io.kubernetes.pod.namespace=kube-system
        k8s:k8s-app=kube-dns
20866   k8s:app=grafana
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=cilium-monitoring
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=default
        k8s:io.kubernetes.pod.namespace=cilium-monitoring
25043   k8s:app.kubernetes.io/name=hubble-ui
        k8s:app.kubernetes.io/part-of=cilium
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=kube-system
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=hubble-ui
        k8s:io.kubernetes.pod.namespace=kube-system
        k8s:k8s-app=hubble-ui
42541   k8s:app=prometheus
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=cilium-monitoring
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=prometheus-k8s
        k8s:io.kubernetes.pod.namespace=cilium-monitoring
53288   k8s:app=mediabot
        k8s:class=mediabot
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=default
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=default
        k8s:io.kubernetes.pod.namespace=default
        k8s:org=empire
54044   k8s:app.kubernetes.io/instance=metrics-server
        k8s:app.kubernetes.io/name=metrics-server
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=kube-system
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=metrics-server
        k8s:io.kubernetes.pod.namespace=kube-system
54079   k8s:app.kubernetes.io/name=hubble-relay
        k8s:app.kubernetes.io/part-of=cilium
        k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=kube-system
        k8s:io.cilium.k8s.policy.cluster=default
        k8s:io.cilium.k8s.policy.serviceaccount=hubble-relay
        k8s:io.kubernetes.pod.namespace=kube-system
        k8s:k8s-app=hubble-relay

소스 Identity에 app=mediabot 라벨 정상 반영 확인

(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl get pods
NAME       READY   STATUS    RESTARTS   AGE
mediabot   1/1     Running   0          79s

mediabot 파드 정상 기동 확인

2. 외부 통신 동작 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec mediabot -- curl -I -s https://api.github.com | head -1

# 결과
HTTP/2 200

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec mediabot -- curl -I -s --max-time 5 https://support.github.com | head -1

# 결과
HTTP/2 302

🔎 DNS Egress 정책 적용 1:`mediabot` 파드가 `api.github.com`에만 액세스하도록 허용

1. CiliumNetworkPolicy 생성

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# cat << EOF | kubectl apply -f -
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "fqdn"
spec:
  endpointSelector:
    matchLabels:
      org: empire
      class: mediabot
  egress:
  - toFQDNs:
    - matchName: "api.github.com"
  - toEndpoints:
    - matchLabels:
        "k8s:io.kubernetes.pod.namespace": kube-system
        "k8s:k8s-app": kube-dns
    toPorts:
    - ports:
      - port: "53"
        protocol: ANY
      rules:
        dns:
        - matchPattern: "*"
EOF

# 결과
ciliumnetworkpolicy.cilium.io/fqdn created

mediabot 파드가 api.github.com 도메인만 접근 허용되도록 CiliumNetworkPolicy 작성 및 적용
DNS 질의 허용을 위해 kube-dns:53 (UDP/TCP) 포트 규칙 추가

2. 정책 생성 여부 확인

(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl get cnp
NAME   AGE   VALID
fqdn   10s   True

3. Policy Selector 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it -n kube-system ds/cilium -- cilium policy selectors 

✅ 출력

  
SELECTOR                                                                                                                                                                      LABELS         USERS   IDENTITIES
&LabelSelector{MatchLabels:map[string]string{any.class: mediabot,any.org: empire,k8s.io.kubernetes.pod.namespace: default,},MatchExpressions:[]LabelSelectorRequirement{},}   default/fqdn   1       53288 

org=empire, class=mediabot, app=mediabot 라벨 기반으로 정책 매칭됨
공식 예제에는 app 라벨이 없어 default로만 표시되어 혼동이 발생 → 직접 라벨 추가

4. Cilium DNS 관련 설정 조회

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# cilium config view | grep -i dns

✅ 출력

  
dnsproxy-enable-transparent-mode                  true
dnsproxy-socket-linger-timeout                    10
hubble-metrics                                    dns drop tcp flow port-distribution icmp httpV2:exemplars=true;labelsContext=source_ip,source_namespace,source_workload,destination_ip,destination_namespace,destination_workload,traffic_direction
tofqdns-dns-reject-response-code                  refused
tofqdns-enable-dns-compression                    true
tofqdns-endpoint-max-ip-per-hostname              1000
tofqdns-idle-connection-grace-period              0s
tofqdns-max-deferred-connection-deletes           10000
tofqdns-preallocate-identities                    true
tofqdns-proxy-response-max-delay                  100ms

CiliumNetworkPolicy 로 DNS 기반 정책을 적용하면, Cilium의 내장 DNS Proxy 가 활성화되어 요청을 가로채고 정책을 적용
DNS Proxy는 CoreDNS 앞단에서 동작하면서 도메인 질의/응답 흐름을 제어함
tofqdns-* 관련 옵션들은 L3 계층에서 DNS 기반 통제 정책을 관리하는 항목

5. `api.github.com` 허용 도메인 접근 테스트

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec mediabot -- curl -I -s https://api.github.com | head -1

# 결과
HTTP/2 200

6. 허용되지 않은 도메인 차단 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec mediabot -- curl -I -s --max-time 5 https://support.github.com | head -1

# 결과
command terminated with exit code 28

exit code 28 (연결 실패) → 정책에 의해 차단됨
Cilium 네트워크 정책은 허용 정책이 있으면 나머지는 모두 차단하는 방식

7. Hubble Relay 포트포워딩

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# cilium hubble port-forward&

# 결과
[1] 8266
(⎈|HomeLab:N/A) root@k8s-ctr:~# ℹ️  Hubble Relay is available at 127.0.0.1:4245

8. Hubble Observe 통한 DNS Proxy 동작 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# hubble observe --pod mediabot

✅ 출력

  
Sep  6 07:52:52.043: default/mediabot:52753 (ID:53288) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  6 07:52:52.043: default/mediabot:52753 (ID:53288) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 07:52:52.043: default/mediabot:52753 (ID:53288) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 07:52:52.043: default/mediabot:52753 (ID:53288) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-network FORWARDED (UDP)
Sep  6 07:52:52.046: default/mediabot:50557 (ID:53288) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  6 07:52:52.046: default/mediabot:50557 (ID:53288) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 07:52:52.047: default/mediabot:50557 (ID:53288) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 07:52:52.053: default/mediabot:50557 (ID:53288) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-network FORWARDED (UDP)
Sep  6 07:56:00.580: default/mediabot:45901 (ID:53288) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  6 07:56:00.580: default/mediabot:45901 (ID:53288) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 07:56:00.580: default/mediabot:45901 (ID:53288) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 07:56:00.580: default/mediabot:45901 (ID:53288) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-network FORWARDED (UDP)
Sep  6 07:56:00.583: default/mediabot:46967 (ID:53288) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  6 07:56:00.584: default/mediabot:46967 (ID:53288) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 07:56:00.584: default/mediabot:46967 (ID:53288) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 07:56:00.584: default/mediabot:46967 (ID:53288) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-network FORWARDED (UDP)
Sep  6 07:56:00.599: default/mediabot:53534 (ID:53288) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  6 07:56:00.599: default/mediabot:53534 (ID:53288) <> kube-system/coredns-99ff8c6c4-fpx2g (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 07:56:00.599: default/mediabot:53534 (ID:53288) <> kube-system/coredns-99ff8c6c4-fpx2g (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 07:56:00.606: default/mediabot:53534 (ID:53288) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-proxy FORWARDED (UDP)
Sep  6 07:56:00.606: default/mediabot:53534 (ID:53288) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) dns-response proxy FORWARDED (DNS Answer  TTL: 4294967295 (Proxy support.github.com. AAAA))
Sep  6 07:56:00.606: kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) <> default/mediabot (ID:53288) pre-xlate-rev TRACED (UDP)
Sep  6 07:56:00.606: kube-system/kube-dns:53 (world) <> default/mediabot (ID:53288) post-xlate-rev TRANSLATED (UDP)
Sep  6 07:56:00.606: default/mediabot:53534 (ID:53288) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) dns-response proxy FORWARDED (DNS Answer "185.199.109.133,185.199.108.133,185.199.110.133,185.199.111.133" TTL: 30 (Proxy support.github.com. A))
Sep  6 07:56:00.607: kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) <> default/mediabot (ID:53288) pre-xlate-rev TRACED (UDP)
Sep  6 07:56:00.607: kube-system/kube-dns:53 (world) <> default/mediabot (ID:53288) post-xlate-rev TRANSLATED (UDP)
Sep  6 07:56:00.607: default/mediabot:51188 (ID:53288) <> support.github.com:443 (world) policy-verdict:none EGRESS DENIED (TCP Flags: SYN)
Sep  6 07:56:00.607: default/mediabot:51188 (ID:53288) <> support.github.com:443 (world) Policy denied DROPPED (TCP Flags: SYN)
Sep  6 07:56:01.625: default/mediabot:51188 (ID:53288) <> support.github.com:443 (world) policy-verdict:none EGRESS DENIED (TCP Flags: SYN)
Sep  6 07:56:01.625: default/mediabot:51188 (ID:53288) <> support.github.com:443 (world) Policy denied DROPPED (TCP Flags: SYN)
Sep  6 07:56:02.649: default/mediabot:51188 (ID:53288) <> support.github.com:443 (world) policy-verdict:none EGRESS DENIED (TCP Flags: SYN)
Sep  6 07:56:02.649: default/mediabot:51188 (ID:53288) <> support.github.com:443 (world) Policy denied DROPPED (TCP Flags: SYN)
Sep  6 07:56:04.095: default/mediabot:37546 (ID:53288) <> support.github.com:443 (world) policy-verdict:none EGRESS DENIED (TCP Flags: SYN)
Sep  6 07:56:04.095: default/mediabot:37546 (ID:53288) <> support.github.com:443 (world) Policy denied DROPPED (TCP Flags: SYN)
Sep  6 07:56:04.957: default/mediabot:59306 (ID:53288) <> support.github.com:443 (world) policy-verdict:none EGRESS DENIED (TCP Flags: SYN)
Sep  6 07:56:04.957: default/mediabot:59306 (ID:53288) <> support.github.com:443 (world) Policy denied DROPPED (TCP Flags: SYN)
EVENTS LOST: HUBBLE_RING_BUFFER CPU(0) 1

9. CoreDNS 로그 활성화 및 확인

log 추가

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl logs -n kube-system -l k8s-app=kube-dns -f

# 결과
maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined
.:53
[INFO] plugin/reload: Running configuration SHA512 = 1b226df79860026c6a52e67daa10d7f0d57ec5b023288ec00c5e05f93523c894564e15b91770d3a07ae1cfbe861d15b37d4a0027e69c546ab112970993a3b03b
CoreDNS-1.12.0
linux/amd64, go1.23.3, 51e11f1
maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined
.:53
[INFO] plugin/reload: Running configuration SHA512 = 1b226df79860026c6a52e67daa10d7f0d57ec5b023288ec00c5e05f93523c894564e15b91770d3a07ae1cfbe861d15b37d4a0027e69c546ab112970993a3b03b
CoreDNS-1.12.0
linux/amd64, go1.23.3, 51e11f1
[INFO] Reloading
[INFO] plugin/reload: Running configuration SHA512 = b997d646f2866cdb764039218df1d7492383070c88b4c8bb8a04cb6fcc0183bf42530d185d9bd227f85a77a57fa184d6184af081e37fc852d6819bf5b3d8a5c4
[INFO] Reloading complete
[INFO] 127.0.0.1:55259 - 23681 "HINFO IN 5995726665743064631.5070209189804637255. udp 57 false 512" NXDOMAIN qr,rd,ra 132 0.010613518s
[INFO] Reloading
[INFO] plugin/reload: Running configuration SHA512 = b997d646f2866cdb764039218df1d7492383070c88b4c8bb8a04cb6fcc0183bf42530d185d9bd227f85a77a57fa184d6184af081e37fc852d6819bf5b3d8a5c4
[INFO] Reloading complete
[INFO] 127.0.0.1:60628 - 1978 "HINFO IN 1275766829604113746.7917362220424931272. udp 57 false 512" NXDOMAIN qr,rd,ra 132 0.01771642s

10. 반복 질의 시도 및 로그 분석

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec mediabot -- curl -I -s https://api.github.com | head -1

✅ 출력

HTTP/2 200

  
[INFO] 172.20.2.46:43944 - 34281 "AAAA IN api.github.com.default.svc.cluster.local. udp 58 false 512" NXDOMAIN qr,aa,rd 151 0.000119368s
[INFO] 172.20.2.46:43944 - 38379 "A IN api.github.com.default.svc.cluster.local. udp 58 false 512" NXDOMAIN qr,aa,rd 151 0.000230762s
[INFO] 172.20.2.46:46453 - 58433 "AAAA IN api.github.com.svc.cluster.local. udp 50 false 512" NXDOMAIN qr,aa,rd 143 0.000309497s
[INFO] 172.20.2.46:46453 - 43587 "A IN api.github.com.svc.cluster.local. udp 50 false 512" NXDOMAIN qr,aa,rd 143 0.000172214s
[INFO] 172.20.2.46:40722 - 24412 "A IN api.github.com.cluster.local. udp 46 false 512" NXDOMAIN qr,aa,rd 139 0.000130767s
[INFO] 172.20.2.46:40722 - 20317 "AAAA IN api.github.com.cluster.local. udp 46 false 512" NXDOMAIN qr,aa,rd 139 0.000159982s
[INFO] 172.20.2.46:40697 - 25020 "A IN api.github.com.davolink. udp 41 false 512" NXDOMAIN qr,rd,ra 116 0.006048631s
[INFO] 172.20.2.46:40697 - 25275 "AAAA IN api.github.com.davolink. udp 41 false 512" NXDOMAIN qr,rd,ra 116 0.006064105s
[INFO] 172.20.2.46:58031 - 6124 "A IN api.github.com. udp 32 false 512" NOERROR qr,rd,ra 410 0.005004666s
[INFO] 172.20.2.46:58031 - 41710 "AAAA IN api.github.com. udp 32 false 512" NOERROR qr,rd,ra 129 0.004969755s

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec mediabot -- curl -I -s https://api.github.com | head -1

✅ 출력

HTTP/2 200

  
[INFO] 172.20.2.46:58536 - 65204 "A IN api.github.com.default.svc.cluster.local. udp 58 false 512" NXDOMAIN qr,aa,rd 151 0.00040403s
[INFO] 172.20.2.46:58536 - 31113 "AAAA IN api.github.com.default.svc.cluster.local. udp 58 false 512" NXDOMAIN qr,aa,rd 151 0.000817802s
[INFO] 172.20.2.46:33003 - 18168 "AAAA IN api.github.com.svc.cluster.local. udp 50 false 512" NXDOMAIN qr,aa,rd 143 0.000283518s
[INFO] 172.20.2.46:33003 - 61178 "A IN api.github.com.svc.cluster.local. udp 50 false 512" NXDOMAIN qr,aa,rd 143 0.000897778s
[INFO] 172.20.2.46:42726 - 52191 "A IN api.github.com.cluster.local. udp 46 false 512" NXDOMAIN qr,aa,rd 139 0.000383001s
[INFO] 172.20.2.46:42726 - 53716 "AAAA IN api.github.com.cluster.local. udp 46 false 512" NXDOMAIN qr,aa,rd 139 0.000780851s
[INFO] 172.20.2.46:60419 - 11887 "A IN api.github.com.davolink. udp 41 false 512" NXDOMAIN qr,rd,ra 116 0.006338245s
[INFO] 172.20.2.46:60419 - 9578 "AAAA IN api.github.com.davolink. udp 41 false 512" NXDOMAIN qr,rd,ra 116 0.205647425s
[INFO] 172.20.2.46:56354 - 52407 "AAAA IN api.github.com. udp 32 false 512" NOERROR qr,rd,ra 116 0.004314325s
[INFO] 172.20.2.46:56354 - 13993 "A IN api.github.com. udp 32 false 512" NOERROR qr,rd,ra 62 0.006500433s

api.github.com 질의가 계속 CoreDNS에 기록됨

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# hubble observe --pod mediabot

✅ 출력

  
Sep  6 08:08:33.289: default/mediabot:58536 (ID:53288) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  6 08:08:33.290: default/mediabot:58536 (ID:53288) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 08:08:33.290: default/mediabot:58536 (ID:53288) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 08:08:33.290: default/mediabot:58536 (ID:53288) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-network FORWARDED (UDP)
Sep  6 08:08:35.374: default/mediabot:38441 (ID:53288) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  6 08:08:35.374: default/mediabot:38441 (ID:53288) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 08:08:35.374: default/mediabot:38441 (ID:53288) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 08:08:35.374: default/mediabot:38441 (ID:53288) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-network FORWARDED (UDP)
Sep  6 08:08:35.376: default/mediabot:38802 (ID:53288) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  6 08:08:35.376: default/mediabot:38802 (ID:53288) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 08:08:35.376: default/mediabot:38802 (ID:53288) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 08:08:35.376: default/mediabot:38802 (ID:53288) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-network FORWARDED (UDP)
Sep  6 08:08:35.377: default/mediabot:33620 (ID:53288) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  6 08:08:35.377: default/mediabot:33620 (ID:53288) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 08:08:35.378: default/mediabot:33620 (ID:53288) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 08:08:35.378: default/mediabot:33620 (ID:53288) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-network FORWARDED (UDP)
Sep  6 08:08:35.380: default/mediabot:59503 (ID:53288) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  6 08:08:35.380: default/mediabot:59503 (ID:53288) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 08:08:35.380: default/mediabot:59503 (ID:53288) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  6 08:08:35.388: default/mediabot:59503 (ID:53288) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-network FORWARDED (UDP)
Sep  6 08:08:35.390: default/mediabot:50527 (ID:53288) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) dns-response proxy FORWARDED (DNS Answer  TTL: 4294967295 (Proxy api.github.com. AAAA))
Sep  6 08:08:35.390: default/mediabot:50527 (ID:53288) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) dns-response proxy FORWARDED (DNS Answer "20.200.245.245" TTL: 15 (Proxy api.github.com. A))
Sep  6 08:08:35.390: kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) <> default/mediabot (ID:53288) pre-xlate-rev TRACED (UDP)
Sep  6 08:08:35.390: kube-system/kube-dns:53 (world) <> default/mediabot (ID:53288) post-xlate-rev TRANSLATED (UDP)
Sep  6 08:08:35.390: kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) <> default/mediabot (ID:53288) pre-xlate-rev TRACED (UDP)
Sep  6 08:08:35.390: kube-system/kube-dns:53 (world) <> default/mediabot (ID:53288) post-xlate-rev TRANSLATED (UDP)
Sep  6 08:08:35.390: default/mediabot:47458 (ID:53288) -> api.github.com:443 (ID:16777217) policy-verdict:L3-Only EGRESS ALLOWED (TCP Flags: SYN)
Sep  6 08:08:35.390: default/mediabot:47458 (ID:53288) -> api.github.com:443 (ID:16777217) to-network FORWARDED (TCP Flags: SYN)
Sep  6 08:08:35.398: default/mediabot:47458 (ID:53288) <- api.github.com:443 (ID:16777217) to-endpoint FORWARDED (TCP Flags: SYN, ACK)
Sep  6 08:08:35.398: default/mediabot:47458 (ID:53288) -> api.github.com:443 (ID:16777217) to-network FORWARDED (TCP Flags: ACK)
Sep  6 08:08:35.399: api.github.com:443 (ID:16777217) <> default/mediabot (ID:53288) pre-xlate-rev TRACED (TCP)
Sep  6 08:08:35.401: default/mediabot:47458 (ID:53288) -> api.github.com:443 (ID:16777217) to-network FORWARDED (TCP Flags: ACK, PSH)
Sep  6 08:08:35.409: default/mediabot:47458 (ID:53288) <- api.github.com:443 (ID:16777217) to-endpoint FORWARDED (TCP Flags: ACK, PSH)
Sep  6 08:08:35.455: api.github.com:443 (ID:16777217) <> default/mediabot (ID:53288) pre-xlate-rev TRACED (TCP)
Sep  6 08:08:35.455: api.github.com:443 (ID:16777217) <> default/mediabot (ID:53288) pre-xlate-rev TRACED (TCP)
Sep  6 08:08:35.464: default/mediabot:47458 (ID:53288) -> api.github.com:443 (ID:16777217) to-network FORWARDED (TCP Flags: ACK, FIN)
Sep  6 08:08:35.474: default/mediabot:47458 (ID:53288) <- api.github.com:443 (ID:16777217) to-endpoint FORWARDED (TCP Flags: ACK, FIN)
Sep  6 08:08:35.474: default/mediabot:47458 (ID:53288) -> api.github.com:443 (ID:16777217) to-network FORWARDED (TCP Flags: RST)
Sep  6 08:08:35.474: default/mediabot:47458 (ID:53288) -> api.github.com:443 (ID:16777217) to-network FORWARDED (TCP Flags: RST)
Sep  6 08:08:35.474: default/mediabot:47458 (ID:53288) <- api.github.com:443 (ID:16777217) to-endpoint FORWARDED (TCP Flags: ACK, RST)

캐싱이 있음에도 불구하고 CoreDNS 로그에 계속 보이는 현상 관찰됨

11. DNS Proxy 설정값 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# cilium config view | grep -i dns

✅ 출력

  
dnsproxy-enable-transparent-mode                  true
dnsproxy-socket-linger-timeout                    10
hubble-metrics                                    dns drop tcp flow port-distribution icmp httpV2:exemplars=true;labelsContext=source_ip,source_namespace,source_workload,destination_ip,destination_namespace,destination_workload,traffic_direction
tofqdns-dns-reject-response-code                  refused
tofqdns-enable-dns-compression                    true
tofqdns-endpoint-max-ip-per-hostname              1000
tofqdns-idle-connection-grace-period              0s
tofqdns-max-deferred-connection-deletes           10000
tofqdns-preallocate-identities                    true
tofqdns-proxy-response-max-delay                  100ms

12. Cilium Agent 캐시 조회

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# export CILIUMPOD0=$(kubectl get -l k8s-app=cilium pods -n kube-system --field-selector spec.nodeName=k8s-ctr -o jsonpath='{.items[0].metadata.name}')
export CILIUMPOD1=$(kubectl get -l k8s-app=cilium pods -n kube-system --field-selector spec.nodeName=k8s-w1  -o jsonpath='{.items[0].metadata.name}')
export CILIUMPOD2=$(kubectl get -l k8s-app=cilium pods -n kube-system --field-selector spec.nodeName=k8s-w2  -o jsonpath='{.items[0].metadata.name}')
echo $CILIUMPOD0 $CILIUMPOD1 $CILIUMPOD2

# 결과
cilium-q2wkn cilium-8wkbz cilium-wcpr2

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# alias c0="kubectl exec -it $CILIUMPOD0 -n kube-system -c cilium-agent -- cilium"
alias c1="kubectl exec -it $CILIUMPOD1 -n kube-system -c cilium-agent -- cilium"
alias c2="kubectl exec -it $CILIUMPOD2 -n kube-system -c cilium-agent -- cilium"

노드별 Cilium Pod를 변수(CILIUMPOD0, CILIUMPOD1, CILIUMPOD2)로 설정 후 alias 등록

(⎈|HomeLab:N/A) root@k8s-ctr:~# c1 fqdn cache list
c2 fqdn cache list

✅ 출력

Endpoint   Source   FQDN   TTL   ExpirationTime   IPs   
Endpoint   Source       FQDN                  TTL   ExpirationTime             IPs               
     connection   support.github.com.   0     2025-09-06T08:17:43.159Z   185.199.110.133   
     connection   support.github.com.   0     2025-09-06T08:17:43.159Z   185.199.111.133   
     connection   support.github.com.   0     2025-09-06T08:17:43.159Z   185.199.109.133   
     connection   support.github.com.   0     2025-09-06T08:17:43.159Z   185.199.108.133   
     connection   api.github.com.       0     2025-09-06T08:17:43.159Z   20.200.245.245  

캐시는 존재하지만, CoreDNS 로그에도 질의가 여전히 기록되는 불일치 현상 발생

13. FQDNPolicySelector 확인

(⎈|HomeLab:N/A) root@k8s-ctr:~# c1 fqdn names
c2 fqdn names

✅ 출력

  
{
  "DNSPollNames": null,
  "FQDNPolicySelectors": []
}

{
  "DNSPollNames": null,
  "FQDNPolicySelectors": [
    {
      "regexString": "^api[.]github[.]com[.]$",
      "selectorString": "MatchName: api.github.com, MatchPattern: "
    }
  ]
}

정책에 따라 api.github.com 만 허용되고 나머지는 차단됨
https://github.com/cilium/cilium/blob/main/pkg/fqdn/dnsproxy/proxy.go

🌐 DNS Egress 정책 적용 2: 모든 GitHub 하위 도메인에 액세스

1. FQDN 캐시 및 정책 초기화

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl delete cnp fqdn
c1 fqdn cache clean -f
c2 fqdn cache clean -f

# 결과
ciliumnetworkpolicy.cilium.io "fqdn" deleted
FQDN proxy cache cleared
FQDN proxy cache cleared

기존 fqdn 정책과 캐시가 남아있으면 새 정책 적용에 영향을 줄 수 있어 초기화 수행

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# c2 fqdn cache list

# 결과
Endpoint   Source   FQDN   TTL   ExpirationTime   IPs 

**2. GitHub 하위 도메인 허용 정책 생성 (`*.github.com`)**

(1) dns-pattern.yaml

  
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "fqdn"
spec:
  endpointSelector:
    matchLabels:
      org: empire
      class: mediabot
  egress:
  - toFQDNs:
    - matchName: "*.github.com"
  - toEndpoints:
    - matchLabels:
        "k8s:io.kubernetes.pod.namespace": kube-system
        "k8s:k8s-app": kube-dns
    toPorts:
    - ports:
      - port: "53"
        protocol: ANY
      rules:
        dns:
        - matchPattern: "*"

(2) 적용

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/1.18.1/examples/kubernetes-dns/dns-pattern.yaml

# 결과
ciliumnetworkpolicy.cilium.io/fqdn created

3. 정책 적용 결과 확인

(⎈|HomeLab:N/A) root@k8s-ctr:~# c1 fqdn names
c2 fqdn names
c1 fqdn cache list
c2 fqdn cache list

✅ 출력

  
{
  "DNSPollNames": null,
  "FQDNPolicySelectors": []
}

{
  "DNSPollNames": null,
  "FQDNPolicySelectors": [
    {
      "regexString": "^[-a-zA-Z0-9_]*[.]github[.]com[.]$",
      "selectorString": "MatchName: , MatchPattern: *.github.com"
    }
  ]
}

Endpoint   Source   FQDN   TTL   ExpirationTime   IPs   
Endpoint   Source   FQDN   TTL   ExpirationTime   IPs  

*.github.com 패턴이 정규식 형태로 변환되어 적용됨
캐시 리스트 초기 상태는 비어 있음

4. GitHub 하위 도메인 접근 테스트

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec mediabot -- curl -I -s https://support.github.com | head -1

# 결과
HTTP/2 302 

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec mediabot -- curl -I -s https://gist.github.com | head -1

# 결과
HTTP/2 302

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec mediabot -- curl -I -s --max-time 5 https://github.com | head -1

# 결과
HTTP/2 200 

support.github.com 요청 → HTTP/2 302 (허용)
gist.github.com 요청 → HTTP/2 302 (허용)
github.com 요청 → HTTP/2 200 (차단 예상이었으나 실제로는 허용되는 동작)
- 공식 문서 설명에 따르면 *.github.com 에 github.com 자체는 포함되지 않아야 함 → 예상과 다른 결과

5. 비허용 도메인 접근 테스트

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec mediabot -- curl -I -s --max-time 5 https://cilium.io| head -1
command terminated with exit code 28

🏷️ DNS Egress 정책 적용 3 : DNS, Port 조합 적용

1. 기존 정책 및 캐시 초기화

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl delete cnp fqdn
c1 fqdn cache clean -f
c2 fqdn cache clean -f

# 결과
ciliumnetworkpolicy.cilium.io "fqdn" deleted
FQDN proxy cache cleared
FQDN proxy cache cleared

2. DNS, Port 조합 정책 생성

  
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "fqdn"
spec:
  endpointSelector:
    matchLabels:
      org: empire
      class: mediabot
  egress:
  - toFQDNs:
    - matchPattern: "*.github.com"
    toPorts:
    - ports:
      - port: "443"
        protocol: TCP
  - toEndpoints:
    - matchLabels:
        "k8s:io.kubernetes.pod.namespace": kube-system
        "k8s:k8s-app": kube-dns
    toPorts:
    - ports:
      - port: "53"
        protocol: ANY
      rules:
        dns:
        - matchPattern: "*"

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/1.18.1/examples/kubernetes-dns/dns-port.yaml

# 결과
ciliumnetworkpolicy.cilium.io/fqdn created

.github.com 대상으로 TCP 443(HTTPS) 만 허용하는 정책 생성
DNS 질의를 위해 CoreDNS(53번 포트)로의 접근은 별도로 허용

3. 정책 적용 결과 확인

(⎈|HomeLab:N/A) root@k8s-ctr:~# c1 fqdn names
c2 fqdn names
c1 fqdn cache list
c2 fqdn cache list

✅ 출력

  
{
  "DNSPollNames": null,
  "FQDNPolicySelectors": []
}

{
  "DNSPollNames": null,
  "FQDNPolicySelectors": [
    {
      "regexString": "^[-a-zA-Z0-9_]*[.]github[.]com[.]$",
      "selectorString": "MatchName: , MatchPattern: *.github.com"
    }
  ]
}

Endpoint   Source   FQDN   TTL   ExpirationTime   IPs   
Endpoint   Source   FQDN   TTL   ExpirationTime   IPs   

4. 도메인 접근 테스트

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec mediabot -- curl -I -s https://support.github.com | head -1

# 결과
HTTP/2 302 

https://support.github.com 요청 → HTTP/2 302 (정상적으로 허용됨)

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec mediabot -- curl -I -s --max-time 5 http://support.github.com | head -1

# 결과
command terminated with exit code 28

http://support.github.com 요청 → exit code 28 (정책에 따라 차단됨)

5. 실습 리소스 정리

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# c1 fqdn cache clean -f
c2 fqdn cache clean -f

# 결과
FQDN proxy cache cleared
FQDN proxy cache cleared

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl delete -f https://raw.githubusercontent.com/cilium/cilium/1.18.1/examples/kubernetes-dns/dns-sw-app.yaml
kubectl delete cnp fqdn

# 결과
pod "mediabot" deleted
ciliumnetworkpolicy.cilium.io "fqdn" deleted

📦 샘플 애플리케이션 배포 및 통신 문제 확인

1. 샘플 애플리케이션 배포

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# cat << EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: webpod
spec:
  replicas: 2
  selector:
    matchLabels:
      app: webpod
  template:
    metadata:
      labels:
        app: webpod
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: app
                operator: In
                values:
                - sample-app
            topologyKey: "kubernetes.io/hostname"
      containers:
      - name: webpod
        image: traefik/whoami
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: webpod
  labels:
    app: webpod
spec:
  selector:
    app: webpod
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
  type: ClusterIP
EOF

# 결과
deployment.apps/webpod created
service/webpod created

2. k8s-ctr 노드에 curl-pod 파드 배포

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: curl-pod
  labels:
    app: curl
spec:
  nodeName: k8s-ctr
  containers:
  - name: curl
    image: nicolaka/netshoot
    command: ["tail"]
    args: ["-f", "/dev/null"]
  terminationGracePeriodSeconds: 0
EOF

# 결과
pod/curl-pod created

🔁 [Lab2] WireGuard 설정 및 실습 : 터널 모드는 두 번 캡슐화됨

https://docs.cilium.io/en/stable/security/network/encryption-wireguard/

1. 커널 WireGuard 지원 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# uname -ar

# 결과
Linux k8s-ctr 6.8.0-64-generic #67-Ubuntu SMP PREEMPT_DYNAMIC Sun Jun 15 20:23:31 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# grep -E 'CONFIG_WIREGUARD=m' /boot/config-$(uname -r)

# 결과
CONFIG_WIREGUARD=m

CONFIG_WIREGUARD=m → Linux 5.6 이상에서 WireGuard 모듈 사용 가능 확인

2. WireGuard 활성화 전 네트워크 상태 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# ip -c addr
ip -c route
ip rule show

✅ 출력

  
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:6d:e2:c4 brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    inet 10.0.2.15/24 metric 100 brd 10.0.2.255 scope global dynamic eth0
       valid_lft 79621sec preferred_lft 79621sec
    inet6 fd17:625c:f037:2:a00:27ff:fe6d:e2c4/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 86122sec preferred_lft 14122sec
    inet6 fe80::a00:27ff:fe6d:e2c4/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:1b:7e:53 brd ff:ff:ff:ff:ff:ff
    altname enp0s8
    inet 192.168.10.100/24 brd 192.168.10.255 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fe1b:7e53/64 scope link 
       valid_lft forever preferred_lft forever
4: cilium_net@cilium_host: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether aa:06:64:c5:fe:45 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::a806:64ff:fec5:fe45/64 scope link 
       valid_lft forever preferred_lft forever
5: cilium_host@cilium_net: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 12:77:a3:e7:d3:5d brd ff:ff:ff:ff:ff:ff
    inet 172.20.0.188/32 scope global cilium_host
       valid_lft forever preferred_lft forever
    inet6 fe80::1077:a3ff:fee7:d35d/64 scope link 
       valid_lft forever preferred_lft forever
7: lxc6e1bde6d9a82@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 02:a2:a1:c5:16:d2 brd ff:ff:ff:ff:ff:ff link-netns cni-66c371ca-2139-d948-994d-bbc0212eb89c
    inet6 fe80::a2:a1ff:fec5:16d2/64 scope link 
       valid_lft forever preferred_lft forever
9: lxc26e84e529505@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether f6:10:c8:bd:bd:13 brd ff:ff:ff:ff:ff:ff link-netns cni-42947c9d-0424-f65e-dfe9-be44ec108d3a
    inet6 fe80::f410:c8ff:febd:bd13/64 scope link 
       valid_lft forever preferred_lft forever
11: lxcf62e2cdd153a@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether ca:b9:69:22:d2:e7 brd ff:ff:ff:ff:ff:ff link-netns cni-3c60e571-202c-2e3d-6edc-03db2dc9cfb3
    inet6 fe80::c8b9:69ff:fe22:d2e7/64 scope link 
       valid_lft forever preferred_lft forever
13: lxc053b4aa84d05@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether f2:ca:88:ea:7a:9b brd ff:ff:ff:ff:ff:ff link-netns cni-7ce2feec-3b4f-bdbd-c538-9f507b25666a
    inet6 fe80::f0ca:88ff:feea:7a9b/64 scope link 
       valid_lft forever preferred_lft forever
15: lxce8925468e90e@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e6:5b:f3:b7:a2:10 brd ff:ff:ff:ff:ff:ff link-netns cni-937843ba-934d-8818-e990-b02ef54878d0
    inet6 fe80::e45b:f3ff:feb7:a210/64 scope link 
       valid_lft forever preferred_lft forever
17: lxcf22a71024ba4@if16: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 6e:56:09:c5:86:aa brd ff:ff:ff:ff:ff:ff link-netns cni-d76de399-05bd-1845-85fb-96c7c1a305ad
    inet6 fe80::6c56:9ff:fec5:86aa/64 scope link 
       valid_lft forever preferred_lft forever
23: lxc49a7fe1ea885@if22: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 8a:b5:82:36:0f:c8 brd ff:ff:ff:ff:ff:ff link-netns cni-315ff049-97c6-94ee-022a-e1b3e4e7c385
    inet6 fe80::88b5:82ff:fe36:fc8/64 scope link 
       valid_lft forever preferred_lft forever
       
default via 10.0.2.2 dev eth0 proto dhcp src 10.0.2.15 metric 100 
1.214.68.2 via 10.0.2.2 dev eth0 proto dhcp src 10.0.2.15 metric 100 
10.0.2.0/24 dev eth0 proto kernel scope link src 10.0.2.15 metric 100 
10.0.2.2 dev eth0 proto dhcp scope link src 10.0.2.15 metric 100 
61.41.153.2 via 10.0.2.2 dev eth0 proto dhcp src 10.0.2.15 metric 100 
172.20.0.95 dev lxcf22a71024ba4 proto kernel scope link 
172.20.0.112 dev lxc053b4aa84d05 proto kernel scope link 
172.20.0.119 dev lxc26e84e529505 proto kernel scope link 
172.20.0.181 dev lxc6e1bde6d9a82 proto kernel scope link 
172.20.0.195 dev lxc49a7fe1ea885 proto kernel scope link 
172.20.0.215 dev lxcf62e2cdd153a proto kernel scope link 
172.20.0.218 dev lxce8925468e90e proto kernel scope link 
172.20.1.0/24 via 192.168.10.101 dev eth1 proto kernel 
172.20.2.0/24 via 192.168.10.102 dev eth1 proto kernel 
192.168.10.0/24 dev eth1 proto kernel scope link src 192.168.10.100 

9:	from all fwmark 0x200/0xf00 lookup 2004
10:	from all fwmark 0xa00/0xf00 lookup 2005
100:	from all lookup local
32766:	from all lookup main
32767:	from all lookup default

현재 cilium_wg0 인터페이스 없음 → WireGuard 암호화 적용 전 기본 네트워크 상태 확인

3. Cilium WireGuard 암호화 활성화

(1) Helm upgrade로 Cilium에 WireGuard 암호화 옵션 적용

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# helm upgrade cilium cilium/cilium --version 1.18.1 --namespace kube-system --reuse-values \
  --set encryption.enabled=true --set encryption.type=wireguard
  
# 결과 
I0907 09:15:56.120915   11365 warnings.go:110] "Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice"
I0907 09:15:56.121975   11365 warnings.go:110] "Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice"
I0907 09:15:56.126031   11365 warnings.go:110] "Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice"
Release "cilium" has been upgraded. Happy Helming!
NAME: cilium
LAST DEPLOYED: Sun Sep  7 09:15:54 2025
NAMESPACE: kube-system
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
You have successfully installed Cilium with Hubble Relay and Hubble UI.

Your release version is 1.18.1.

For any further help, visit https://docs.cilium.io/en/v1.18/gettinghelp

(2) 적용 후 DaemonSet 재시작

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl -n kube-system rollout restart ds/cilium

# 결과
daemonset.apps/cilium restarted

(3) 적용 결과

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# cilium config view | grep -i wireguard

✅ 출력

enable-wireguard                                  true
wireguard-persistent-keepalive                    0s

4. 암호화 상태 및 키 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it -n kube-system ds/cilium -- cilium encrypt status

✅ 출력

Encryption: Wireguard                 
Interface: cilium_wg0
	Public key: 2Qoj+Ao91VcauXdQUBjrex80MmYqypiIzIeOJe6UWUM=
	Number of peers: 2

WireGuard 암호화 활성화 및 Public Key 표시

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it -n kube-system ds/cilium -- cilium status | grep Encryption

✅ 출력

  
Encryption:              Wireguard   [NodeEncryption: Disabled, cilium_wg0 (Pubkey: 2Qoj+Ao91VcauXdQUBjrex80MmYqypiIzIeOJe6UWUM=, Port: 51871, Peers: 2)]

cilium_wg0 인터페이스와 Port(51871) 확인
각 노드 간 Public Key 교환으로 Peer 인증 수행

5. WireGuard 인터페이스 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# ip -d -c addr show cilium_wg0

✅ 출력

24: cilium_wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default 
    link/none  promiscuity 0  allmulti 0 minmtu 0 maxmtu 2147483552 
    wireguard numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 tso_max_size 65536 tso_max_segs 65535 gro_max_size 65536 

(⎈|HomeLab:N/A) root@k8s-ctr:~# wg show

✅ 출력

  
interface: cilium_wg0
  public key: Y5uWPr1Tmpyth78YEWFei83nNIN5Sl22IhUUbgdjBBU=
  private key: (hidden)
  listening port: 51871
  fwmark: 0xe00

peer: 2Qoj+Ao91VcauXdQUBjrex80MmYqypiIzIeOJe6UWUM=
  endpoint: 192.168.10.101:51871
  allowed ips: 172.20.1.113/32, 192.168.10.101/32, 172.20.1.249/32, 172.20.1.211/32, 172.20.1.0/24, 172.20.1.204/32
  latest handshake: 4 minutes, 54 seconds ago
  transfer: 524 B received, 468 B sent

peer: zJr4UGy+Ngkw7YirUtPbrrxzGbvwEnZwnCVLrtXXVmA=
  endpoint: 192.168.10.102:51871
  allowed ips: 192.168.10.102/32, 172.20.2.81/32, 172.20.2.0/24, 172.20.2.149/32, 172.20.2.84/32, 172.20.2.79/32
  latest handshake: 4 minutes, 54 seconds ago
  transfer: 316 B received, 324 B sent

Public/Private Key, Peer 정보, Handshake 및 전송량 확인 가능

6. WireGuard 상세 정보 확인

(⎈|HomeLab:N/A) root@k8s-ctr:~# wg show all public-key
wg show all private-key
wg show all preshared-keys
wg show all endpoints
wg show all transfer

✅ 출력

  
cilium_wg0	Y5uWPr1Tmpyth78YEWFei83nNIN5Sl22IhUUbgdjBBU=
cilium_wg0	MJUHH5QDUjmK30yCaL7hT2ZvxdOewuaDF3Uuue1HS14=
cilium_wg0	2Qoj+Ao91VcauXdQUBjrex80MmYqypiIzIeOJe6UWUM=	(none)
cilium_wg0	zJr4UGy+Ngkw7YirUtPbrrxzGbvwEnZwnCVLrtXXVmA=	(none)
cilium_wg0	2Qoj+Ao91VcauXdQUBjrex80MmYqypiIzIeOJe6UWUM=	192.168.10.101:51871
zJr4UGy+Ngkw7YirUtPbrrxzGbvwEnZwnCVLrtXXVmA=	192.168.10.102:51871
cilium_wg0	2Qoj+Ao91VcauXdQUBjrex80MmYqypiIzIeOJe6UWUM=	524	468
cilium_wg0	zJr4UGy+Ngkw7YirUtPbrrxzGbvwEnZwnCVLrtXXVmA=	316	324

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl get cn -o yaml | grep annotations -A1

✅ 출력

  
    annotations:
      network.cilium.io/wg-pub-key: Y5uWPr1Tmpyth78YEWFei83nNIN5Sl22IhUUbgdjBBU=
--
    annotations:
      network.cilium.io/wg-pub-key: 2Qoj+Ao91VcauXdQUBjrex80MmYqypiIzIeOJe6UWUM=
--
    annotations:
      network.cilium.io/wg-pub-key: zJr4UGy+Ngkw7YirUtPbrrxzGbvwEnZwnCVLrtXXVmA=

각 Cilium Node의 Public Key는 Kubernetes Node 리소스 애노테이션에도 기록됨

7. Pod 간 통신 확인 (암호화 경로 테스트)

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# tcpdump -i cilium_wg0 -n

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it curl-pod -- curl webpod

✅ 출력

Hostname: webpod-697b545f57-zkb6k
IP: 127.0.0.1
IP: ::1
IP: 172.20.1.249
IP: fe80::102f:89ff:fe3c:e632
RemoteAddr: 172.20.0.195:35450
GET / HTTP/1.1
Host: webpod
User-Agent: curl/8.14.1
Accept: */*

  
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on cilium_wg0, link-type RAW (Raw IP), snapshot length 262144 bytes
29:32.237736 IP 172.20.0.195.53279 > 172.20.2.84.53: 20628+ [1au] A? webpod.default.svc.cluster.local. (73)
29:32.237954 IP 172.20.0.195.53279 > 172.20.2.84.53: 50422+ [1au] AAAA? webpod.default.svc.cluster.local. (73)
29:32.240059 IP 172.20.2.84.53 > 172.20.0.195.53279: 50422*- 0/1/1 (166)
29:32.240095 IP 172.20.2.84.53 > 172.20.0.195.53279: 20628*- 1/0/1 A 10.96.200.211 (121)
29:32.240747 IP 172.20.0.195.35450 > 172.20.1.249.80: Flags [S], seq 100649939, win 64860, options [mss 1380,sackOK,TS val 2403113329 ecr 0,nop,wscale 7], length 0
29:32.242105 IP 172.20.1.249.80 > 172.20.0.195.35450: Flags [S.], seq 116405546, ack 100649940, win 64296, options [mss 1380,sackOK,TS val 2757771911 ecr 2403113329,nop,wscale 7], length 0
29:32.242198 IP 172.20.0.195.35450 > 172.20.1.249.80: Flags [.], ack 1, win 507, options [nop,nop,TS val 2403113331 ecr 2757771911], length 0
29:32.242317 IP 172.20.0.195.35450 > 172.20.1.249.80: Flags [P.], seq 1:71, ack 1, win 507, options [nop,nop,TS val 2403113331 ecr 2757771911], length 70: HTTP: GET / HTTP/1.1
29:32.243266 IP 172.20.1.249.80 > 172.20.0.195.35450: Flags [.], ack 71, win 502, options [nop,nop,TS val 2757771912 ecr 2403113331], length 0
29:32.243853 IP 172.20.1.249.80 > 172.20.0.195.35450: Flags [P.], seq 1:323, ack 71, win 502, options [nop,nop,TS val 2757771913 ecr 2403113331], length 322: HTTP: HTTP/1.1 200 OK
29:32.243890 IP 172.20.0.195.35450 > 172.20.1.249.80: Flags [.], ack 323, win 505, options [nop,nop,TS val 2403113332 ecr 2757771913], length 0
29:32.244064 IP 172.20.0.195.35450 > 172.20.1.249.80: Flags [F.], seq 71, ack 323, win 505, options [nop,nop,TS val 2403113332 ecr 2757771913], length 0
29:32.245104 IP 172.20.1.249.80 > 172.20.0.195.35450: Flags [F.], seq 323, ack 72, win 502, options [nop,nop,TS val 2757771914 ecr 2403113332], length 0
29:32.245398 IP 172.20.0.195.35450 > 172.20.1.249.80: Flags [.], ack 324, win 505, options [nop,nop,TS val 2403113334 ecr 2757771914], length 0

tcpdump -i cilium_wg0 결과, Pod 통신이 WireGuard 인터페이스를 통해 캡슐화되어 흐름 추적됨

8. 물리 인터페이스 트래픽 캡처

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# tcpdump -eni any udp port 51871

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it curl-pod -- curl webpod

✅ 출력

Hostname: webpod-697b545f57-mmxjm
IP: 127.0.0.1
IP: ::1
IP: 172.20.2.149
IP: fe80::a4c5:4fff:fe8a:9a2e
RemoteAddr: 172.20.0.195:44090
GET / HTTP/1.1
Host: webpod
User-Agent: curl/8.14.1
Accept: */*

  
tcpdump: data link type LINUX_SLL2
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
33:22.875937 eth1  Out ifindex 3 08:00:27:1b:7e:53 ethertype IPv4 (0x0800), length 196: 192.168.10.100.51871 > 192.168.10.101.51871: UDP, length 148
33:22.876608 eth1  In  ifindex 3 08:00:27:9c:b4:fd ethertype IPv4 (0x0800), length 140: 192.168.10.101.51871 > 192.168.10.100.51871: UDP, length 92
33:22.876845 eth1  Out ifindex 3 08:00:27:1b:7e:53 ethertype IPv4 (0x0800), length 192: 192.168.10.100.51871 > 192.168.10.101.51871: UDP, length 144
33:22.876863 eth1  Out ifindex 3 08:00:27:1b:7e:53 ethertype IPv4 (0x0800), length 192: 192.168.10.100.51871 > 192.168.10.101.51871: UDP, length 144
33:22.877669 eth1  In  ifindex 3 08:00:27:9c:b4:fd ethertype IPv4 (0x0800), length 288: 192.168.10.101.51871 > 192.168.10.100.51871: UDP, length 240
33:22.877669 eth1  In  ifindex 3 08:00:27:9c:b4:fd ethertype IPv4 (0x0800), length 240: 192.168.10.101.51871 > 192.168.10.100.51871: UDP, length 192
33:22.878559 eth1  Out ifindex 3 08:00:27:1b:7e:53 ethertype IPv4 (0x0800), length 196: 192.168.10.100.51871 > 192.168.10.102.51871: UDP, length 148
33:22.879186 eth1  In  ifindex 3 08:00:27:2d:d6:34 ethertype IPv4 (0x0800), length 140: 192.168.10.102.51871 > 192.168.10.100.51871: UDP, length 92
33:22.879368 eth1  Out ifindex 3 08:00:27:1b:7e:53 ethertype IPv4 (0x0800), length 144: 192.168.10.100.51871 > 192.168.10.102.51871: UDP, length 96
33:22.879864 eth1  In  ifindex 3 08:00:27:2d:d6:34 ethertype IPv4 (0x0800), length 144: 192.168.10.102.51871 > 192.168.10.100.51871: UDP, length 96
33:22.880074 eth1  Out ifindex 3 08:00:27:1b:7e:53 ethertype IPv4 (0x0800), length 144: 192.168.10.100.51871 > 192.168.10.102.51871: UDP, length 96
33:22.880085 eth1  Out ifindex 3 08:00:27:1b:7e:53 ethertype IPv4 (0x0800), length 208: 192.168.10.100.51871 > 192.168.10.102.51871: UDP, length 160
33:22.880988 eth1  In  ifindex 3 08:00:27:2d:d6:34 ethertype IPv4 (0x0800), length 144: 192.168.10.102.51871 > 192.168.10.100.51871: UDP, length 96
33:22.880988 eth1  In  ifindex 3 08:00:27:2d:d6:34 ethertype IPv4 (0x0800), length 464: 192.168.10.102.51871 > 192.168.10.100.51871: UDP, length 416
33:22.881816 eth1  Out ifindex 3 08:00:27:1b:7e:53 ethertype IPv4 (0x0800), length 144: 192.168.10.100.51871 > 192.168.10.102.51871: UDP, length 96
33:22.881827 eth1  Out ifindex 3 08:00:27:1b:7e:53 ethertype IPv4 (0x0800), length 144: 192.168.10.100.51871 > 192.168.10.102.51871: UDP, length 96
33:22.882277 eth1  In  ifindex 3 08:00:27:2d:d6:34 ethertype IPv4 (0x0800), length 144: 192.168.10.102.51871 > 192.168.10.100.51871: UDP, length 96
33:22.882436 eth1  Out ifindex 3 08:00:27:1b:7e:53 ethertype IPv4 (0x0800), length 144: 192.168.10.100.51871 > 192.168.10.102.51871: UDP, length 96
33:32.960639 eth1  In  ifindex 3 08:00:27:2d:d6:34 ethertype IPv4 (0x0800), length 80: 192.168.10.102.51871 > 192.168.10.100.51871: UDP, length 32
33:33.018232 eth1  Out ifindex 3 08:00:27:1b:7e:53 ethertype IPv4 (0x0800), length 80: 192.168.10.100.51871 > 192.168.10.101.51871: UDP, length 32

실제 노드 NIC(eth1)에서 UDP 51871 포트로 암호화된 패킷 교환 확인
192.168.10.100.51871 > 192.168.10.101.51871

9. 패킷 저장 및 복호화 불가 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# tcpdump -eni any udp port 51871 -w /tmp/wg.pcap

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it curl-pod -- curl webpod
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it curl-pod -- curl webpod

  
vagrant scp k8s-ctr:/tmp/wg.pcap .

# 결과
Warning: Permanently added '[127.0.0.1]:60000' (ED25519) to the list of known hosts.
wg.pcap                                                                                                                                             100% 6312     3.9MB/s   00:00    

termshark -r wg.pcap

✅ 출력

패킷 내용은 암호화되어 중간에서 탈취하더라도 복호화 불가

10. Hubble 로깅 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# hubble observe --pod curl-pod

✅ 출력

  
Sep  7 00:43:46.008: default/curl-pod:48687 (ID:49833) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  7 00:43:46.008: default/curl-pod:48687 (ID:49833) <> kube-system/coredns-99ff8c6c4-fpx2g (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 00:43:46.009: default/curl-pod:48687 (ID:49833) <> kube-system/coredns-99ff8c6c4-fpx2g (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 00:43:46.009: default/curl-pod:48687 (ID:49833) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-network FORWARDED (UDP)
Sep  7 00:43:46.010: default/curl-pod:42504 (ID:49833) -> default/webpod-697b545f57-mmxjm:80 (ID:48381) to-endpoint FORWARDED (TCP Flags: SYN)
Sep  7 00:43:46.010: default/curl-pod:42504 (ID:49833) <- default/webpod-697b545f57-mmxjm:80 (ID:48381) to-network FORWARDED (TCP Flags: SYN, ACK)
Sep  7 00:43:46.011: default/curl-pod:42504 (ID:49833) -> default/webpod-697b545f57-mmxjm:80 (ID:48381) to-endpoint FORWARDED (TCP Flags: ACK)
Sep  7 00:43:46.011: default/curl-pod:42504 (ID:49833) -> default/webpod-697b545f57-mmxjm:80 (ID:48381) to-endpoint FORWARDED (TCP Flags: ACK, PSH)
Sep  7 00:43:46.011: default/curl-pod:42504 (ID:49833) <> default/webpod-697b545f57-mmxjm (ID:48381) pre-xlate-rev TRACED (TCP)
Sep  7 00:43:46.011: default/curl-pod:42504 (ID:49833) <> default/webpod-697b545f57-mmxjm (ID:48381) pre-xlate-rev TRACED (TCP)
Sep  7 00:43:46.011: default/curl-pod:42504 (ID:49833) <> default/webpod-697b545f57-mmxjm (ID:48381) pre-xlate-rev TRACED (TCP)
Sep  7 00:43:46.012: default/curl-pod:42504 (ID:49833) <> default/webpod-697b545f57-mmxjm (ID:48381) pre-xlate-rev TRACED (TCP)
Sep  7 00:43:46.012: default/curl-pod:42504 (ID:49833) <> default/webpod-697b545f57-mmxjm (ID:48381) pre-xlate-rev TRACED (TCP)
Sep  7 00:43:46.012: default/curl-pod:42504 (ID:49833) <- default/webpod-697b545f57-mmxjm:80 (ID:48381) to-network FORWARDED (TCP Flags: ACK, PSH)
Sep  7 00:43:46.012: default/curl-pod:42504 (ID:49833) -> default/webpod-697b545f57-mmxjm:80 (ID:48381) to-endpoint FORWARDED (TCP Flags: ACK, FIN)
Sep  7 00:43:46.013: default/curl-pod:42504 (ID:49833) <- default/webpod-697b545f57-mmxjm:80 (ID:48381) to-network FORWARDED (TCP Flags: ACK, FIN)
Sep  7 00:43:46.013: default/curl-pod:42504 (ID:49833) <- default/webpod-697b545f57-mmxjm:80 (ID:48381) to-endpoint FORWARDED (TCP Flags: ACK, FIN)
Sep  7 00:43:46.013: default/curl-pod:42504 (ID:49833) -> default/webpod-697b545f57-mmxjm:80 (ID:48381) to-network FORWARDED (TCP Flags: ACK)
Sep  7 00:43:46.013: default/curl-pod:42504 (ID:49833) -> default/webpod-697b545f57-mmxjm:80 (ID:48381) to-endpoint FORWARDED (TCP Flags: ACK)
Sep  7 00:43:47.399: default/curl-pod (ID:49833) <> kube-system/kube-dns:53 (world) pre-xlate-fwd TRACED (UDP)
Sep  7 00:43:47.399: default/curl-pod (ID:49833) <> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) post-xlate-fwd TRANSLATED (UDP)
Sep  7 00:43:47.399: default/curl-pod:42521 (ID:49833) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-network FORWARDED (UDP)
Sep  7 00:43:47.401: default/curl-pod:42521 (ID:49833) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  7 00:43:47.402: default/curl-pod:42521 (ID:49833) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 00:43:47.402: default/curl-pod:42521 (ID:49833) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 00:43:47.402: default/curl-pod:42521 (ID:49833) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-network FORWARDED (UDP)
Sep  7 00:43:47.402: default/curl-pod:42521 (ID:49833) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  7 00:43:47.402: kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) <> default/curl-pod (ID:49833) pre-xlate-rev TRACED (UDP)
Sep  7 00:43:47.402: kube-system/kube-dns:53 (world) <> default/curl-pod (ID:49833) post-xlate-rev TRANSLATED (UDP)
Sep  7 00:43:47.402: kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) <> default/curl-pod (ID:49833) pre-xlate-rev TRACED (UDP)
Sep  7 00:43:47.402: kube-system/kube-dns:53 (world) <> default/curl-pod (ID:49833) post-xlate-rev TRANSLATED (UDP)
Sep  7 00:43:47.402: default/curl-pod (ID:49833) <> default/webpod:80 (world) pre-xlate-fwd TRACED (TCP)
Sep  7 00:43:47.402: default/curl-pod (ID:49833) <> default/webpod-697b545f57-zkb6k:80 (ID:48381) post-xlate-fwd TRANSLATED (TCP)
Sep  7 00:43:47.402: default/curl-pod:38224 (ID:49833) -> default/webpod-697b545f57-zkb6k:80 (ID:48381) to-network FORWARDED (TCP Flags: SYN)
Sep  7 00:43:47.403: default/curl-pod:38224 (ID:49833) -> default/webpod-697b545f57-zkb6k:80 (ID:48381) to-endpoint FORWARDED (TCP Flags: SYN)
Sep  7 00:43:47.403: default/curl-pod:38224 (ID:49833) <- default/webpod-697b545f57-zkb6k:80 (ID:48381) to-network FORWARDED (TCP Flags: SYN, ACK)
Sep  7 00:43:47.403: default/curl-pod:38224 (ID:49833) <- default/webpod-697b545f57-zkb6k:80 (ID:48381) to-endpoint FORWARDED (TCP Flags: SYN, ACK)
Sep  7 00:43:47.403: default/curl-pod:38224 (ID:49833) -> default/webpod-697b545f57-zkb6k:80 (ID:48381) to-network FORWARDED (TCP Flags: ACK)
Sep  7 00:43:47.403: default/curl-pod:38224 (ID:49833) -> default/webpod-697b545f57-zkb6k:80 (ID:48381) to-network FORWARDED (TCP Flags: ACK, PSH)
Sep  7 00:43:47.404: default/curl-pod:38224 (ID:49833) -> default/webpod-697b545f57-zkb6k:80 (ID:48381) to-endpoint FORWARDED (TCP Flags: ACK)
Sep  7 00:43:47.404: default/curl-pod:38224 (ID:49833) -> default/webpod-697b545f57-zkb6k:80 (ID:48381) to-endpoint FORWARDED (TCP Flags: ACK, PSH)
Sep  7 00:43:47.404: default/curl-pod:38224 (ID:49833) <> default/webpod-697b545f57-zkb6k (ID:48381) pre-xlate-rev TRACED (TCP)
Sep  7 00:43:47.404: default/curl-pod:38224 (ID:49833) <> default/webpod-697b545f57-zkb6k (ID:48381) pre-xlate-rev TRACED (TCP)
Sep  7 00:43:47.404: default/curl-pod:38224 (ID:49833) <> default/webpod-697b545f57-zkb6k (ID:48381) pre-xlate-rev TRACED (TCP)
Sep  7 00:43:47.405: default/curl-pod:38224 (ID:49833) <> default/webpod-697b545f57-zkb6k (ID:48381) pre-xlate-rev TRACED (TCP)
Sep  7 00:43:47.405: default/curl-pod:38224 (ID:49833) <> default/webpod-697b545f57-zkb6k (ID:48381) pre-xlate-rev TRACED (TCP)
Sep  7 00:43:47.405: default/curl-pod:38224 (ID:49833) <- default/webpod-697b545f57-zkb6k:80 (ID:48381) to-network FORWARDED (TCP Flags: ACK, PSH)
Sep  7 00:43:47.405: default/curl-pod:38224 (ID:49833) <- default/webpod-697b545f57-zkb6k:80 (ID:48381) to-endpoint FORWARDED (TCP Flags: ACK, PSH)
Sep  7 00:43:47.405: default/curl-pod:38224 (ID:49833) -> default/webpod-697b545f57-zkb6k:80 (ID:48381) to-network FORWARDED (TCP Flags: ACK, FIN)
Sep  7 00:43:47.406: default/curl-pod:38224 (ID:49833) -> default/webpod-697b545f57-zkb6k:80 (ID:48381) to-endpoint FORWARDED (TCP Flags: ACK, FIN)
Sep  7 00:43:47.406: default/curl-pod:38224 (ID:49833) <- default/webpod-697b545f57-zkb6k:80 (ID:48381) to-network FORWARDED (TCP Flags: ACK, FIN)
Sep  7 00:43:47.406: default/curl-pod:38224 (ID:49833) <- default/webpod-697b545f57-zkb6k:80 (ID:48381) to-endpoint FORWARDED (TCP Flags: ACK, FIN)
Sep  7 00:43:47.406: default/curl-pod:38224 (ID:49833) -> default/webpod-697b545f57-zkb6k:80 (ID:48381) to-network FORWARDED (TCP Flags: ACK)
Sep  7 00:43:47.407: default/curl-pod:38224 (ID:49833) -> default/webpod-697b545f57-zkb6k:80 (ID:48381) to-endpoint FORWARDED (TCP Flags: ACK)

Pod 간 요청/응답이 암호화된 WireGuard 터널을 경유하여 전달되는 과정 확인

11. 설정 원복

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# helm upgrade cilium cilium/cilium --version 1.18.1 --namespace kube-system --reuse-values \
  --set encryption.enabled=false
  
# 결과  
I0907 09:47:15.951908   13591 warnings.go:110] "Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice"
I0907 09:47:15.956586   13591 warnings.go:110] "Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice"
I0907 09:47:15.965752   13591 warnings.go:110] "Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice"
Release "cilium" has been upgraded. Happy Helming!
NAME: cilium
LAST DEPLOYED: Sun Sep  7 09:47:11 2025
NAMESPACE: kube-system
STATUS: deployed
REVISION: 3
TEST SUITE: None
NOTES:
You have successfully installed Cilium with Hubble Relay and Hubble UI.

Your release version is 1.18.1.

For any further help, visit https://docs.cilium.io/en/v1.18/gettinghelp

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl -n kube-system rollout restart ds/cilium

# 결과
daemonset.apps/cilium restarted

⚙️ Inspecting TLS Encrypted Connections with Cilium

https://docs.cilium.io/en/stable/security/tls-visibility/
대부분의 내부 사용자는 HTTPS(암호화된) 트래픽으로 외부와 통신
일반 방화벽은 암호화된 HTTPS 내용을 직접 볼 수 없어 위험 사이트 접근 차단 불가
보안이 강화된 기업 환경에서는, 외부 통신 시 Cilium이 중간에서 TLS 트래픽을 가로채 모니터링 가능

1. `cilium-secrets` 네임스페이스 존재 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl get all,secret,cm -n cilium-secrets

✅ 출력

NAME                         DATA   AGE
configmap/kube-root-ca.crt   1      19h

2. TLS 설정 파일 생성

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# cat << EOF > tls-config.yaml
tls:
  readSecretsOnlyFromSecretsNamespace: true
  secretsNamespace:
    name: cilium-secrets # This setting is optional, as it is the default
  secretSync:
    enabled: true
EOF

TLS 인증서/비밀을 Cilium이 감시·동기화할 수 있도록 허용

3. Cilium 재배포 (TLS 설정 반영)

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# helm upgrade cilium cilium/cilium --version 1.18.1 --namespace kube-system --reuse-values \
-f tls-config.yaml

# 결과
I0907 09:58:41.136472   14540 warnings.go:110] "Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice"
I0907 09:58:41.138440   14540 warnings.go:110] "Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice"
I0907 09:58:41.142939   14540 warnings.go:110] "Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice"
Release "cilium" has been upgraded. Happy Helming!
NAME: cilium
LAST DEPLOYED: Sun Sep  7 09:58:36 2025
NAMESPACE: kube-system
STATUS: deployed
REVISION: 4
TEST SUITE: None
NOTES:
You have successfully installed Cilium with Hubble Relay and Hubble UI.

Your release version is 1.18.1.

For any further help, visit https://docs.cilium.io/en/v1.18/gettinghelp

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl -n kube-system rollout restart deploy/cilium-operator
kubectl -n kube-system rollout restart ds/cilium

# 결과
deployment.apps/cilium-operator restarted
daemonset.apps/cilium restarted

4. TLS Secret 동기화 설정 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# cilium config view | grep -i secret

✅ 출력

  
enable-ingress-secrets-sync                       true
enable-policy-secrets-sync                        true
ingress-secrets-namespace                         cilium-secrets
policy-secrets-namespace                          cilium-secrets
policy-secrets-only-from-secrets-namespace        true

🛡️ [Lab3] Cilium TLS Interception Demo 실습

https://docs.cilium.io/en/stable/security/tls-visibility/#deploy-the-demo-application

1. Demo 애플리케이션 배포 (mediabot)

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# cat << EOF > dns-sw-app.yaml
apiVersion: v1
kind: Pod
metadata:
  name: mediabot
  labels:
    org: empire
    class: mediabot
    app: mediabot
spec:
  containers:
  - name: mediabot
    image: quay.io/cilium/json-mock:v1.3.8@sha256:5aad04835eda9025fe4561ad31be77fd55309af8158ca8663a72f6abb78c2603
EOF

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl apply -f dns-sw-app.yaml
kubectl wait pod/mediabot --for=condition=Ready

# 결과
pod/mediabot created
pod/mediabot condition met

mediabot 파드를 배포하여 Star Wars API(HTTPS 기반) 접근 테스트 준비
HTTPS 트래픽은 기본적으로 암호화되어 네트워크 계층(Cilium)에서 HTTP 세부 정보 확인 불가

(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl get pods

✅ 출력

NAME                      READY   STATUS    RESTARTS   AGE
curl-pod                  1/1     Running   0          16h
mediabot                  1/1     Running   0          40s
webpod-697b545f57-mmxjm   1/1     Running   0          16h
webpod-697b545f57-zkb6k   1/1     Running   0          16h

2. 내부 CA 키 생성

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# openssl genrsa -des3 -out myCA.key 2048

Enter PEM pass phrase: qwe123
Verifying - Enter PEM pass phrase: qwe123

openssl genrsa 명령으로 CA 개인 키(myCA.key) 생성

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# ls *.key
myCA.key

3. CA 인증서 생성

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# openssl req -x509 -new -nodes -key myCA.key -sha256 -days 1825 -out myCA.crt

Enter pass phrase for myCA.key: qwe123
Country Name (2 letter code) [AU]:KR
State or Province Name (full name) [Some-State]:Seoul
Locality Name (eg, city) []:Seoul
Organization Name (eg, company) [Internet Widgits Pty Ltd]:cloudneta
Organizational Unit Name (eg, section) []:IT
Common Name (e.g. server FQDN or YOUR name) []:cloudneta.net
Email Address []:

생성한 myCA.key를 기반으로 자체 서명 CA 인증서(myCA.crt) 발급

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# openssl x509 -in myCA.crt -noout -text

✅ 출력

  
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            70:04:f6:77:46:af:71:1a:96:5a:81:a8:49:fb:76:ff:a7:e8:e6:d1
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = KR, ST = Seoul, L = Seoul, O = cloudneta, OU = IT, CN = cloudneta.net
        Validity
            Not Before: Sep  7 02:01:32 2025 GMT
            Not After : Sep  6 02:01:32 2030 GMT
        Subject: C = KR, ST = Seoul, L = Seoul, O = cloudneta, OU = IT, CN = cloudneta.net
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:a0:41:c3:47:6f:78:16:5a:db:67:f6:f2:49:07:
                    51:18:d9:77:1c:77:cd:6d:84:95:7f:9d:c1:ce:f6:
                    9b:01:52:0d:af:7d:8b:e2:d7:2c:41:e4:18:ea:4f:
                    fd:70:52:cf:1e:a1:f4:b1:6b:35:29:40:73:0d:7b:
                    ba:f9:03:9d:7c:9b:bb:bd:d8:dc:5a:d5:b5:5e:a8:
                    02:40:95:86:28:6f:17:dd:85:90:bc:ee:a5:15:9c:
                    d1:2b:d8:9a:ea:8e:92:93:3a:d9:b1:14:0f:4b:e6:
                    04:87:c3:33:ca:2d:8a:e8:a4:6b:08:34:5e:99:5e:
                    e1:56:90:3f:0d:69:fe:95:25:7c:e3:ac:ee:71:8e:
                    9d:3c:80:75:e4:97:9e:1c:cc:77:66:0a:a8:53:63:
                    da:7d:bd:9f:5c:81:b5:18:54:13:8a:bb:40:25:cc:
                    d0:28:79:09:cb:4f:8c:72:3c:1d:dd:48:8d:8f:10:
                    14:49:e3:c4:d4:fb:9c:a8:3a:07:b0:09:2a:44:18:
                    84:68:f7:b8:7c:23:95:51:82:7a:7e:20:83:15:f2:
                    f9:03:8a:98:6c:14:13:24:ca:33:8e:08:86:4f:9b:
                    62:52:5f:62:ea:b5:42:51:e7:84:3e:c5:48:5e:1f:
                    50:be:35:7b:61:68:57:4c:1c:27:2c:f8:05:a3:a7:
                    12:b3
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Key Identifier: 
                24:41:F1:08:5B:13:03:5E:B3:FB:A9:20:D6:39:F1:2B:C6:46:6D:78
            X509v3 Authority Key Identifier: 
                24:41:F1:08:5B:13:03:5E:B3:FB:A9:20:D6:39:F1:2B:C6:46:6D:78
            X509v3 Basic Constraints: critical
                CA:TRUE
    Signature Algorithm: sha256WithRSAEncryption
    Signature Value:
        82:c8:3b:28:4b:a1:3f:1d:c8:d8:68:81:a2:8c:de:26:f3:bd:
        56:fe:2a:83:a4:b1:c8:59:13:0b:e8:77:02:02:6f:fb:3b:9e:
        00:e3:7b:c7:8a:02:b7:ca:74:9a:dc:ac:10:c6:af:f7:40:b2:
        9a:1f:f6:2d:91:6a:fa:3e:ee:48:8e:32:bb:0e:c0:bb:4a:00:
        aa:7c:87:bf:9a:50:e9:93:fd:37:8c:c4:6d:a1:dc:26:9e:09:
        8e:e8:81:85:13:2c:fa:ac:95:cb:47:c3:7d:4f:19:8c:ec:1d:
        f7:2d:ce:bc:ee:de:ac:9c:66:bb:e4:e6:79:68:1a:9c:48:aa:
        5f:da:8b:8f:27:8e:87:d5:d2:96:9e:97:2a:7c:63:bf:04:19:
        ed:06:a3:26:18:75:f2:cc:63:14:6c:44:af:89:fb:f2:9e:85:
        5f:dc:6a:09:7e:9d:4c:89:01:9f:a7:f0:c5:85:f2:90:f0:67:
        f1:ef:5a:34:98:3a:55:a8:54:41:f6:8d:fb:22:99:74:a1:3f:
        32:79:4f:0e:57:0e:d1:f7:3b:f3:3b:09:cd:39:ab:bf:ac:78:
        8f:98:5d:85:5e:7c:b2:92:11:35:6e:52:48:3d:b0:6c:4b:9c:
        de:35:3e:05:45:97:f5:7e:12:26:fc:ea:03:c9:08:cb:ce:3c:
        3c:14:b4:78

CA:TRUE 속성 확인 → 루트 CA 권한 보유

4. 대상 서비스용 개인 키 및 CSR 생성

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# openssl genrsa -out internal-httpbin.key 2048
(⎈|HomeLab:N/A) root@k8s-ctr:~# ls internal-httpbin.key

# 결과
internal-httpbin.key

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# openssl req -new -key internal-httpbin.key -out internal-httpbin.csr

You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]: KR
State or Province Name (full name) [Some-State]:Seoul
Locality Name (eg, city) []:Seoul
Organization Name (eg, company) [Internet Widgits Pty Ltd]:cloudneta
Organizational Unit Name (eg, section) []:IT
Common Name (e.g. server FQDN or YOUR name) []:httpbin.org
Email Address []:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# ls internal-httpbin.csr

# 결과
internal-httpbin.csr

5. CA로 서명된 인증서 발급

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# openssl x509 -req -days 360 -in internal-httpbin.csr -CA myCA.crt -CAkey myCA.key -CAcreateserial -out internal-httpbin.crt -sha256
Certificate request self-signature ok
subject=C = KR, ST = Seoul, L = Seoul, O = cloudneta, OU = IT, CN = httpbin.org
Enter pass phrase for myCA.key: qwe123

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# ls internal-httpbin.crt

# 결과
internal-httpbin.crt

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# openssl x509 -in internal-httpbin.crt -noout -text

✅ 출력

  
Certificate:
    Data:
        Version: 1 (0x0)
        Serial Number:
            18:d7:be:ea:13:75:4b:40:c5:b1:6e:fe:16:16:ea:55:0d:ae:00:75
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = KR, ST = Seoul, L = Seoul, O = cloudneta, OU = IT, CN = cloudneta.net
        Validity
            Not Before: Sep  7 02:09:59 2025 GMT
            Not After : Sep  2 02:09:59 2026 GMT
        Subject: C = KR, ST = Seoul, L = Seoul, O = cloudneta, OU = IT, CN = httpbin.org
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:d1:4c:93:6e:74:c7:0f:70:cd:fd:36:8d:66:b9:
                    19:81:17:d4:b1:8e:60:6a:17:f2:43:eb:74:22:3a:
                    45:43:35:bf:40:84:44:ba:75:72:fb:6b:69:39:84:
                    b4:38:9e:6f:09:57:e4:2f:a0:e6:46:10:6b:74:2e:
                    bf:df:23:ee:0b:b6:51:62:a6:01:26:60:b1:4a:d0:
                    76:fe:a5:80:e6:cd:0b:60:b8:9d:d5:a8:45:63:7a:
                    78:4c:fc:eb:d1:85:76:e7:76:91:2c:1d:46:a4:41:
                    c9:54:5d:10:bf:23:68:f3:02:bc:a3:ff:4b:92:90:
                    e9:07:12:9c:17:0e:3a:cb:ac:20:01:a6:e4:8b:47:
                    6c:7e:fa:0a:30:76:c2:10:c7:3d:fe:ad:78:1f:f8:
                    e2:10:4e:74:f9:b7:90:78:8b:c7:e2:03:41:f2:27:
                    ed:b7:bc:a1:f9:db:0a:a8:71:41:5a:99:4e:51:e6:
                    d2:d5:9c:af:a6:cb:4b:be:e8:eb:89:2d:93:d4:c5:
                    fc:cc:5c:f3:7e:ee:71:56:b2:7d:36:fe:df:7a:2f:
                    39:b4:9d:24:34:c7:f8:cf:2a:a7:c5:2e:15:06:06:
                    f2:2f:17:d8:f3:da:9b:7f:16:96:ed:9c:14:ad:0f:
                    27:00:78:0f:fb:2a:5a:bd:20:2f:75:91:25:ce:95:
                    fa:d7
                Exponent: 65537 (0x10001)
    Signature Algorithm: sha256WithRSAEncryption
    Signature Value:
        03:52:66:d6:68:d6:02:33:23:8a:4b:f4:02:37:3f:fd:19:06:
        7c:1c:fd:50:f0:f1:3e:26:31:ee:00:a7:a9:db:52:ef:37:af:
        27:aa:b6:09:a3:9b:51:63:6d:bd:29:c4:48:74:2b:79:c0:6e:
        2b:75:80:fe:ee:4f:e3:cb:14:86:9d:4d:92:d6:08:a1:14:73:
        a6:90:67:ff:c0:b3:6a:8d:35:e9:14:66:af:a5:1a:00:50:cf:
        6e:e2:ed:e7:65:6a:e7:4d:49:9d:ba:16:eb:3e:86:1c:d8:ac:
        a2:5e:fb:7b:46:4e:f7:20:a4:66:ed:e0:fd:93:de:40:64:59:
        62:4e:be:78:75:d8:27:fd:72:a8:e6:ae:f0:e2:1a:8f:0c:9b:
        f6:2b:b4:d5:9a:59:4a:4e:e3:a9:c0:9f:c4:79:2d:cd:4a:8e:
        de:f2:2b:fc:c6:6f:f9:bc:fe:85:9f:6a:d8:2a:04:04:b3:30:
        ff:4b:54:12:56:cc:df:fc:94:63:c5:b5:37:f9:64:db:24:c9:
        c5:20:2c:9b:5f:70:b9:63:b7:cd:79:ec:64:79:17:71:e9:dd:
        81:54:6e:73:8a:18:d5:43:83:43:5a:66:5a:97:02:03:d2:ae:
        78:26:f2:93:b6:a7:5a:ac:3a:a4:ae:64:83:cf:53:03:46:79:
        19:bd:7e:b2

발급된 인증서 Subject 필드에서 CN이 httpbin.org 로 설정된 것 확인

6. Kubernetes Secret 생성 (Terminating TLS)

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl create secret tls httpbin-tls-data -n kube-system --cert=internal-httpbin.crt --key=internal-httpbin.key

# 결과
secret/httpbin-tls-data created

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl get secret -n kube-system  httpbin-tls-data

✅ 출력

NAME               TYPE                DATA   AGE
httpbin-tls-data   kubernetes.io/tls   2      27s

Secret 정상 생성 확인 (kubernetes.io/tls 타입, 데이터 2개)

7. 클라이언트 파드에 내부 CA 추가

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it mediabot -- ls -l /usr/local/share/ca-certificates/

# 결과
total 0

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl cp myCA.crt default/mediabot:/usr/local/share/ca-certificates/myCA.crt
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it mediabot -- ls -l /usr/local/share/ca-certificates/

# 결과
total 4
-rw-r--r-- 1 root root 1334 Sep  7 02:14 myCA.crt

mediabot 파드 내 /usr/local/share/ca-certificates/ 디렉터리에 myCA.crt 복사

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it mediabot -- ls -l /etc/ssl/certs/ca-certificates.crt

# 결과
-rw-r--r-- 1 root root 213777 Jan  9  2024 /etc/ssl/certs/ca-certificates.crt

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec mediabot -- update-ca-certificates

# 결과
Updating certificates in /etc/ssl/certs...
rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.

내부 CA를 시스템 신뢰 CA 목록에 반영

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it mediabot -- ls -l /etc/ssl/certs/ca-certificates.crt
-rw-r--r-- 1 root root 215111 Sep  7 02:16 /etc/ssl/certs/ca-certificates.crt

/etc/ssl/certs/ca-certificates.crt 크기 증가 → 내부 CA가 추가됨 확인

8. Cilium에 신뢰할 수 있는 CA 목록 제공

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl cp default/mediabot:/etc/ssl/certs/ca-certificates.crt ca-certificates.crt

# 결과
tar: Removing leading `/' from member names

mediabot 파드에서 /etc/ssl/certs/ca-certificates.crt 추출

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl create secret generic tls-orig-data -n kube-system --from-file=ca.crt=./ca-certificates.crt

# 결과
secret/tls-orig-data created

이를 기반으로 tls-orig-data Secret 생성 (kube-system 네임스페이스)
Cilium이 외부 TLS 연결 시 사용할 신뢰 CA 목록 제공

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl get secret -n kube-system tls-orig-data
NAME            TYPE     DATA   AGE
tls-orig-data   Opaque   1      31s

📑 Apply DNS and TLS-aware Egress Policy

1. `CiliumNetworkPolicy(l7-visibility-tls)` 작성

  
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
  name: "l7-visibility-tls"
spec:
  description: L7 policy with TLS
  endpointSelector:
    matchLabels:
      org: empire
      class: mediabot
  egress:
  - toFQDNs:
    - matchName: "httpbin.org"
    toPorts:
    - ports:
      - port: "443"
        protocol: "TCP"
      terminatingTLS:
        secret:
          namespace: "kube-system"
          name: "httpbin-tls-data"
      originatingTLS:
        secret:
          namespace: "kube-system"
          name: "tls-orig-data"
      rules:
        http:
        - {}
  - toPorts:
    - ports:
      - port: "53"
        protocol: ANY
      rules:
        dns:
          - matchPattern: "*"

2. 정책 적용 전 트래픽 상태 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# hubble observe --pod mediabot -f

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it mediabot -- curl -sL 'https://httpbin.org/anything'

✅ 출력

  
{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Host": "httpbin.org", 
    "User-Agent": "curl/7.88.1", 
    "X-Amzn-Trace-Id": "Root=1-68bced16-291e405a1fbfd499591dde37"
  }, 
  "json": null, 
  "method": "GET", 
  "origin": "182.230.60.93", 
  "url": "https://httpbin.org/anything"
}

  
Sep  7 02:25:07.616: default/mediabot (ID:23265) <> kube-system/kube-dns:53 (world) pre-xlate-fwd TRACED (UDP)
Sep  7 02:25:07.616: default/mediabot (ID:23265) <> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) post-xlate-fwd TRANSLATED (UDP)
Sep  7 02:25:07.616: default/mediabot:47008 (ID:23265) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.616: default/mediabot:47008 (ID:23265) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.616: default/mediabot:47008 (ID:23265) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  7 02:25:07.616: default/mediabot:47008 (ID:23265) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  7 02:25:07.616: kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.616: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:25:07.617: kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.617: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:25:07.617: default/mediabot (ID:23265) <> kube-system/kube-dns:53 (world) pre-xlate-fwd TRACED (UDP)
Sep  7 02:25:07.617: default/mediabot (ID:23265) <> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) post-xlate-fwd TRANSLATED (UDP)
Sep  7 02:25:07.617: default/mediabot:33128 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-network FORWARDED (UDP)
Sep  7 02:25:07.617: default/mediabot:33128 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  7 02:25:07.617: default/mediabot:33128 (ID:23265) <> kube-system/coredns-99ff8c6c4-fpx2g (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.618: default/mediabot:33128 (ID:23265) <> kube-system/coredns-99ff8c6c4-fpx2g (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.618: default/mediabot:33128 (ID:23265) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  7 02:25:07.618: kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.618: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:25:07.618: kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.618: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:25:07.618: default/mediabot (ID:23265) <> kube-system/kube-dns:53 (world) pre-xlate-fwd TRACED (UDP)
Sep  7 02:25:07.618: default/mediabot (ID:23265) <> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) post-xlate-fwd TRANSLATED (UDP)
Sep  7 02:25:07.618: default/mediabot:33128 (ID:23265) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-network FORWARDED (UDP)
Sep  7 02:25:07.618: default/mediabot:60232 (ID:23265) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.618: default/mediabot:60232 (ID:23265) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.618: default/mediabot:60232 (ID:23265) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  7 02:25:07.618: default/mediabot:60232 (ID:23265) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  7 02:25:07.618: kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.618: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:25:07.618: kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.618: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:25:07.618: default/mediabot (ID:23265) <> kube-system/kube-dns:53 (world) pre-xlate-fwd TRACED (UDP)
Sep  7 02:25:07.618: default/mediabot (ID:23265) <> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) post-xlate-fwd TRANSLATED (UDP)
Sep  7 02:25:07.618: default/mediabot:44667 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-network FORWARDED (UDP)
Sep  7 02:25:07.619: default/mediabot:44667 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  7 02:25:07.619: default/mediabot:44667 (ID:23265) <> kube-system/coredns-99ff8c6c4-fpx2g (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.619: default/mediabot:44667 (ID:23265) <> kube-system/coredns-99ff8c6c4-fpx2g (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.628: default/mediabot:44667 (ID:23265) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  7 02:25:07.628: default/mediabot:44667 (ID:23265) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-network FORWARDED (UDP)
Sep  7 02:25:07.628: kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.628: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:25:07.699: kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.699: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:25:07.700: default/mediabot (ID:23265) <> kube-system/kube-dns:53 (world) pre-xlate-fwd TRACED (UDP)
Sep  7 02:25:07.700: default/mediabot (ID:23265) <> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) post-xlate-fwd TRANSLATED (UDP)
Sep  7 02:25:07.700: default/mediabot:60434 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-network FORWARDED (UDP)
Sep  7 02:25:07.700: default/mediabot:60434 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  7 02:25:07.700: default/mediabot:60434 (ID:23265) <> kube-system/coredns-99ff8c6c4-fpx2g (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.700: default/mediabot:60434 (ID:23265) <> kube-system/coredns-99ff8c6c4-fpx2g (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.705: default/mediabot:60434 (ID:23265) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  7 02:25:07.705: kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.705: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:25:07.705: default/mediabot:60434 (ID:23265) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-network FORWARDED (UDP)
Sep  7 02:25:07.740: kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:25:07.740: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:25:07.740: default/mediabot:34984 (ID:23265) -> 52.71.132.100:443 (world) to-network FORWARDED (TCP Flags: SYN)
Sep  7 02:25:07.947: default/mediabot:34984 (ID:23265) <- 52.71.132.100:443 (world) to-endpoint FORWARDED (TCP Flags: SYN, ACK)
Sep  7 02:25:07.948: default/mediabot:34984 (ID:23265) -> 52.71.132.100:443 (world) to-network FORWARDED (TCP Flags: ACK)
Sep  7 02:25:07.948: 52.71.132.100:443 (world) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (TCP)
Sep  7 02:25:07.949: default/mediabot:34984 (ID:23265) -> 52.71.132.100:443 (world) to-network FORWARDED (TCP Flags: ACK, PSH)
Sep  7 02:25:08.163: default/mediabot:34984 (ID:23265) <- 52.71.132.100:443 (world) to-endpoint FORWARDED (TCP Flags: ACK, PSH)
Sep  7 02:25:08.372: 52.71.132.100:443 (world) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (TCP)
Sep  7 02:25:08.372: 52.71.132.100:443 (world) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (TCP)
Sep  7 02:25:09.096: default/mediabot:34984 (ID:23265) -> 52.71.132.100:443 (world) to-network FORWARDED (TCP Flags: ACK, RST)

mediabot → https://httpbin.org/anything 요청 시 정상 200 응답
Hubble 로그에는 world IP 수준까지만 기록되고, L7 세부 정보는 확인 불가

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it mediabot -- curl -sL 'https://httpbin.org/headers' -v

✅ 출력

  
*   Trying 44.195.242.49:443...
* Connected to httpbin.org (44.195.242.49) port 443 (#0)
* ALPN: offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=httpbin.org
*  start date: Jul 20 00:00:00 2025 GMT
*  expire date: Aug 17 23:59:59 2026 GMT
*  subjectAltName: host "httpbin.org" matched cert's "httpbin.org"
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M03
*  SSL certificate verify ok.
* using HTTP/2
* h2h3 [:method: GET]
* h2h3 [:path: /headers]
* h2h3 [:scheme: https]
* h2h3 [:authority: httpbin.org]
* h2h3 [user-agent: curl/7.88.1]
* h2h3 [accept: */*]
* Using Stream ID: 1 (easy handle 0x5cef43e9ac80)
> GET /headers HTTP/2
> Host: httpbin.org
> user-agent: curl/7.88.1
> accept: */*
> 
< HTTP/2 200 
< date: Sun, 07 Sep 2025 02:27:09 GMT
< content-type: application/json
< content-length: 173
< server: gunicorn/19.9.0
< access-control-allow-origin: *
< access-control-allow-credentials: true
< 
{
  "headers": {
    "Accept": "*/*", 
    "Host": "httpbin.org", 
    "User-Agent": "curl/7.88.1", 
    "X-Amzn-Trace-Id": "Root=1-68bced7d-0275a60e6c1206872e96159a"
  }
}
* Connection #0 to host httpbin.org left intact

curl -v 옵션으로 확인 시 Amazon에서 발급한 인증서가 표시됨

3. 정책 적용 (TLS 인식 Egress Policy)

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl create -f https://raw.githubusercontent.com/cilium/cilium/1.18.1/examples/kubernetes-tls-inspection/l7-visibility-tls.yaml

# 결과
ciliumnetworkpolicy.cilium.io/l7-visibility-tls created

4. 정책 적용 후 L7 가시성 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# hubble observe --pod mediabot -f

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it mediabot -- curl -sL 'https://httpbin.org/anything'

✅ 출력

  
{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Host": "httpbin.org", 
    "User-Agent": "curl/7.88.1", 
    "X-Amzn-Trace-Id": "Root=1-68bcee86-162e56176873065d43ba22f7", 
    "X-Envoy-Expected-Rq-Timeout-Ms": "3600000", 
    "X-Envoy-Internal": "true"
  }, 
  "json": null, 
  "method": "GET", 
  "origin": "182.230.60.93", 
  "url": "https://httpbin.org/anything"
}

  
Sep  7 02:31:15.767: default/mediabot (ID:23265) <> kube-system/kube-dns:53 (world) pre-xlate-fwd TRACED (UDP)
Sep  7 02:31:15.767: default/mediabot (ID:23265) <> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) post-xlate-fwd TRANSLATED (UDP)
Sep  7 02:31:15.767: default/mediabot:36388 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) policy-verdict:L4-Only EGRESS ALLOWED (UDP)
Sep  7 02:31:15.767: default/mediabot:36388 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-proxy FORWARDED (UDP)
Sep  7 02:31:15.767: default/mediabot:36388 (ID:23265) <> 192.168.10.101 (host) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.767: default/mediabot:36388 (ID:23265) <> 192.168.10.101 (host) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.768: default/mediabot:36388 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) dns-request proxy FORWARDED (DNS Query httpbin.org.default.svc.cluster.local. AAAA)
Sep  7 02:31:15.768: default/mediabot:36388 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) dns-request proxy FORWARDED (DNS Query httpbin.org.default.svc.cluster.local. A)
Sep  7 02:31:15.768: default/mediabot:36388 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  7 02:31:15.769: default/mediabot:36388 (ID:23265) <> kube-system/coredns-99ff8c6c4-fpx2g (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.769: default/mediabot:36388 (ID:23265) <> kube-system/coredns-99ff8c6c4-fpx2g (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.769: default/mediabot:36388 (ID:23265) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-network FORWARDED (UDP)
Sep  7 02:31:15.769: default/mediabot:36388 (ID:23265) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-proxy FORWARDED (UDP)
Sep  7 02:31:15.769: default/mediabot:36388 (ID:23265) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) dns-response proxy FORWARDED (DNS Answer RCode: Non-Existent Domain TTL: 4294967295 (Proxy httpbin.org.default.svc.cluster.local. AAAA))
Sep  7 02:31:15.769: kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.769: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:31:15.770: default/mediabot:36388 (ID:23265) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) dns-response proxy FORWARDED (DNS Answer RCode: Non-Existent Domain TTL: 4294967295 (Proxy httpbin.org.default.svc.cluster.local. A))
Sep  7 02:31:15.771: kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.771: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:31:15.771: default/mediabot (ID:23265) <> kube-system/kube-dns:53 (world) pre-xlate-fwd TRACED (UDP)
Sep  7 02:31:15.771: default/mediabot (ID:23265) <> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) post-xlate-fwd TRANSLATED (UDP)
Sep  7 02:31:15.771: default/mediabot:60675 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) policy-verdict:L4-Only EGRESS ALLOWED (UDP)
Sep  7 02:31:15.771: default/mediabot:60675 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-proxy FORWARDED (UDP)
Sep  7 02:31:15.771: default/mediabot:60675 (ID:23265) <> 192.168.10.101 (host) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.771: default/mediabot:60675 (ID:23265) <> 192.168.10.101 (host) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.771: default/mediabot:60675 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) dns-request proxy FORWARDED (DNS Query httpbin.org.svc.cluster.local. AAAA)
Sep  7 02:31:15.771: default/mediabot:60675 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) dns-request proxy FORWARDED (DNS Query httpbin.org.svc.cluster.local. A)
Sep  7 02:31:15.772: default/mediabot:60675 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  7 02:31:15.772: default/mediabot:60675 (ID:23265) <> kube-system/coredns-99ff8c6c4-fpx2g (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.772: default/mediabot:60675 (ID:23265) <> kube-system/coredns-99ff8c6c4-fpx2g (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.772: default/mediabot:60675 (ID:23265) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-network FORWARDED (UDP)
Sep  7 02:31:15.773: default/mediabot:60675 (ID:23265) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-proxy FORWARDED (UDP)
Sep  7 02:31:15.773: default/mediabot:60675 (ID:23265) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) dns-response proxy FORWARDED (DNS Answer RCode: Non-Existent Domain TTL: 4294967295 (Proxy httpbin.org.svc.cluster.local. A))
Sep  7 02:31:15.773: default/mediabot:60675 (ID:23265) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) dns-response proxy FORWARDED (DNS Answer RCode: Non-Existent Domain TTL: 4294967295 (Proxy httpbin.org.svc.cluster.local. AAAA))
Sep  7 02:31:15.773: kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.773: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:31:15.773: kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.773: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:31:15.774: default/mediabot:55813 (ID:23265) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) dns-request proxy FORWARDED (DNS Query httpbin.org.cluster.local. AAAA)
Sep  7 02:31:15.774: default/mediabot:55813 (ID:23265) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) dns-request proxy FORWARDED (DNS Query httpbin.org.cluster.local. A)
Sep  7 02:31:15.774: default/mediabot (ID:23265) <> kube-system/kube-dns:53 (world) pre-xlate-fwd TRACED (UDP)
Sep  7 02:31:15.774: default/mediabot (ID:23265) <> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) post-xlate-fwd TRANSLATED (UDP)
Sep  7 02:31:15.774: default/mediabot:55813 (ID:23265) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) policy-verdict:L4-Only EGRESS ALLOWED (UDP)
Sep  7 02:31:15.774: default/mediabot:55813 (ID:23265) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-proxy FORWARDED (UDP)
Sep  7 02:31:15.774: default/mediabot:55813 (ID:23265) <> 192.168.10.101 (host) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.774: default/mediabot:55813 (ID:23265) <> 192.168.10.101 (host) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.774: default/mediabot:55813 (ID:23265) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  7 02:31:15.774: default/mediabot:55813 (ID:23265) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.774: default/mediabot:55813 (ID:23265) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) dns-response proxy FORWARDED (DNS Answer RCode: Non-Existent Domain TTL: 4294967295 (Proxy httpbin.org.cluster.local. AAAA))
Sep  7 02:31:15.774: default/mediabot:55813 (ID:23265) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) dns-response proxy FORWARDED (DNS Answer RCode: Non-Existent Domain TTL: 4294967295 (Proxy httpbin.org.cluster.local. A))
Sep  7 02:31:15.774: kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.775: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:31:15.775: default/mediabot:55813 (ID:23265) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.775: default/mediabot:55813 (ID:23265) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-proxy FORWARDED (UDP)
Sep  7 02:31:15.775: kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.775: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:31:15.775: default/mediabot (ID:23265) <> kube-system/kube-dns:53 (world) pre-xlate-fwd TRACED (UDP)
Sep  7 02:31:15.775: default/mediabot (ID:23265) <> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) post-xlate-fwd TRANSLATED (UDP)
Sep  7 02:31:15.775: default/mediabot:36959 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) policy-verdict:L4-Only EGRESS ALLOWED (UDP)
Sep  7 02:31:15.775: default/mediabot:36959 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-proxy FORWARDED (UDP)
Sep  7 02:31:15.775: default/mediabot:36959 (ID:23265) <> 192.168.10.101 (host) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.775: default/mediabot:36959 (ID:23265) <> 192.168.10.101 (host) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.775: default/mediabot:36959 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) dns-request proxy FORWARDED (DNS Query httpbin.org.davolink. A)
Sep  7 02:31:15.775: default/mediabot:36959 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) dns-request proxy FORWARDED (DNS Query httpbin.org.davolink. AAAA)
Sep  7 02:31:15.775: default/mediabot:36959 (ID:23265) -> kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  7 02:31:15.775: default/mediabot:36959 (ID:23265) <> kube-system/coredns-99ff8c6c4-fpx2g (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.776: default/mediabot:36959 (ID:23265) <> kube-system/coredns-99ff8c6c4-fpx2g (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.785: default/mediabot:36959 (ID:23265) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-network FORWARDED (UDP)
Sep  7 02:31:15.786: default/mediabot:36959 (ID:23265) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) to-proxy FORWARDED (UDP)
Sep  7 02:31:15.786: default/mediabot:36959 (ID:23265) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) dns-response proxy FORWARDED (DNS Answer RCode: Non-Existent Domain TTL: 4294967295 (Proxy httpbin.org.davolink. A))
Sep  7 02:31:15.787: default/mediabot:36959 (ID:23265) <- kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) dns-response proxy FORWARDED (DNS Answer RCode: Non-Existent Domain TTL: 4294967295 (Proxy httpbin.org.davolink. AAAA))
Sep  7 02:31:15.787: kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.787: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:31:15.787: kube-system/coredns-99ff8c6c4-fpx2g:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.787: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:31:15.787: default/mediabot (ID:23265) <> kube-system/kube-dns:53 (world) pre-xlate-fwd TRACED (UDP)
Sep  7 02:31:15.787: default/mediabot (ID:23265) <> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) post-xlate-fwd TRANSLATED (UDP)
Sep  7 02:31:15.787: default/mediabot:58129 (ID:23265) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) policy-verdict:L4-Only EGRESS ALLOWED (UDP)
Sep  7 02:31:15.787: default/mediabot:58129 (ID:23265) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-proxy FORWARDED (UDP)
Sep  7 02:31:15.787: default/mediabot:58129 (ID:23265) <> 192.168.10.101 (host) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.787: default/mediabot:58129 (ID:23265) <> 192.168.10.101 (host) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.788: default/mediabot:58129 (ID:23265) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) dns-request proxy FORWARDED (DNS Query httpbin.org. AAAA)
Sep  7 02:31:15.788: default/mediabot:58129 (ID:23265) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-endpoint FORWARDED (UDP)
Sep  7 02:31:15.788: default/mediabot:58129 (ID:23265) -> kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) dns-request proxy FORWARDED (DNS Query httpbin.org. A)
Sep  7 02:31:15.788: default/mediabot:58129 (ID:23265) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.789: default/mediabot:58129 (ID:23265) <> kube-system/coredns-99ff8c6c4-6zg5v (ID:18194) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.794: default/mediabot:58129 (ID:23265) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) to-proxy FORWARDED (UDP)
Sep  7 02:31:15.795: default/mediabot:58129 (ID:23265) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) dns-response proxy FORWARDED (DNS Answer  TTL: 4294967295 (Proxy httpbin.org. AAAA))
Sep  7 02:31:15.795: kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.795: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:31:15.797: default/mediabot:58129 (ID:23265) <- kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) dns-response proxy FORWARDED (DNS Answer "44.195.242.49,54.227.118.219,34.238.12.187,52.71.132.100,54.166.11.78,13.222.46.84" TTL: 30 (Proxy httpbin.org. A))
Sep  7 02:31:15.798: kube-system/coredns-99ff8c6c4-6zg5v:53 (ID:18194) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (UDP)
Sep  7 02:31:15.798: kube-system/kube-dns:53 (world) <> default/mediabot (ID:23265) post-xlate-rev TRANSLATED (UDP)
Sep  7 02:31:15.798: default/mediabot:34464 (ID:23265) -> httpbin.org:443 (ID:16777217) policy-verdict:L3-L4 EGRESS ALLOWED (TCP Flags: SYN)
Sep  7 02:31:15.798: default/mediabot:34464 (ID:23265) -> httpbin.org:443 (ID:16777217) to-proxy FORWARDED (TCP Flags: SYN)
Sep  7 02:31:15.798: default/mediabot:34464 (ID:23265) <- httpbin.org:443 (ID:16777217) to-endpoint FORWARDED (TCP Flags: SYN, ACK)
Sep  7 02:31:15.798: default/mediabot:34464 (ID:23265) -> httpbin.org:443 (ID:16777217) to-proxy FORWARDED (TCP Flags: ACK)
Sep  7 02:31:15.799: httpbin.org:443 (ID:16777217) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (TCP)
Sep  7 02:31:15.799: default/mediabot:34464 (ID:23265) <> 192.168.10.101 (host) pre-xlate-rev TRACED (TCP)
Sep  7 02:31:15.802: default/mediabot:34464 (ID:23265) -> httpbin.org:443 (ID:16777217) to-proxy FORWARDED (TCP Flags: ACK, PSH)
Sep  7 02:31:15.810: default/mediabot:34464 (ID:23265) <- httpbin.org:443 (ID:16777217) to-endpoint FORWARDED (TCP Flags: ACK, PSH)
Sep  7 02:31:15.842: httpbin.org:443 (ID:16777217) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (TCP)
Sep  7 02:31:15.842: httpbin.org:443 (ID:16777217) <> default/mediabot (ID:23265) pre-xlate-rev TRACED (TCP)
Sep  7 02:31:15.843: default/mediabot:34464 (ID:23265) -> httpbin.org:443 (ID:16777217) http-request FORWARDED (HTTP/1.1 GET https://httpbin.org/anything)
Sep  7 02:31:16.651: default/mediabot:34464 (ID:23265) <- httpbin.org:443 (ID:16777217) http-response FORWARDED (HTTP/1.1 200 808ms (GET https://httpbin.org/anything))
Sep  7 02:31:16.652: default/mediabot:34464 (ID:23265) <- httpbin.org:443 (ID:16777217) to-endpoint FORWARDED (TCP Flags: ACK, FIN)
Sep  7 02:31:16.653: default/mediabot:34464 (ID:23265) -> httpbin.org:443 (ID:16777217) to-proxy FORWARDED (TCP Flags: ACK, RST)

mediabot → https://httpbin.org/anything 요청 시 여전히 정상 응답 (200 OK)
Hubble 로그에서 HTTP L7 레벨 세부정보 확인 가능
- ex. (HTTP/1.1 GET https://httpbin.org/anything)
원래는 외부 IP만 기록됐지만, 이제는 L7 요청/응답까지 추적 가능

5. 서버 인증서 확인 (변경 확인)

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -it mediabot -- curl -sL 'https://httpbin.org/headers' -v

✅ 출력

  
*   Trying 52.71.132.100:443...
* Connected to httpbin.org (52.71.132.100) port 443 (#0)
* ALPN: offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN: server did not agree on a protocol. Uses default.
* Server certificate:
*  subject: C=KR; ST=Seoul; L=Seoul; O=cloudneta; OU=IT; CN=httpbin.org
*  start date: Sep  7 02:09:59 2025 GMT
*  expire date: Sep  2 02:09:59 2026 GMT
*  common name: httpbin.org (matched)
*  issuer: C=KR; ST=Seoul; L=Seoul; O=cloudneta; OU=IT; CN=cloudneta.net
*  SSL certificate verify ok.
* using HTTP/1.x
> GET /headers HTTP/1.1
> Host: httpbin.org
> User-Agent: curl/7.88.1
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/1.1 200 OK
< date: Sun, 07 Sep 2025 02:35:31 GMT
< content-type: application/json
< content-length: 256
< server: envoy
< access-control-allow-origin: *
< access-control-allow-credentials: true
< x-envoy-upstream-service-time: 811
< 
{
  "headers": {
    "Accept": "*/*", 
    "Host": "httpbin.org", 
    "User-Agent": "curl/7.88.1", 
    "X-Amzn-Trace-Id": "Root=1-68bcef73-793ab92f5b0e49233cf3b64b", 
    "X-Envoy-Expected-Rq-Timeout-Ms": "3600000", 
    "X-Envoy-Internal": "true"
  }
}
* Connection #0 to host httpbin.org left intact

기존: Amazon RSA 2048 M03 발급 인증서 표시
정책 적용 후: cloudneta.net 내부 CA가 발급한 httpbin.org 인증서로 표시

6. 실습 리소스 삭제

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl delete -f https://raw.githubusercontent.com/cilium/cilium/1.18.1/examples/kubernetes-dns/dns-sw-app.yaml
kubectl delete cnp l7-visibility-tls
kubectl delete secret -n kube-system tls-orig-data
kubectl delete secret -n kube-system httpbin-tls-data

# 결과
pod "mediabot" deleted
ciliumnetworkpolicy.cilium.io "l7-visibility-tls" deleted
secret "tls-orig-data" deleted
secret "httpbin-tls-data" deleted

🕵️ Tetragon

Tetragon은 리눅스 커널 레벨에서 eBPF를 활용해 프로세스 실행, 시스템 콜, 네트워크 및 파일 I/O 이벤트를 탐지·모니터링·통제 가능
Kubernetes 환경을 인식하여 네임스페이스, 파드, 워크로드 단위 보안 이벤트 추적 및 정책 적용 가능
eBPF 레벨에서 정책과 필터링을 직접 적용하는 런타임 보안 및 관찰 도구
https://tetragon.io/docs/getting-started/

1. Tetragon 설치

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# helm repo add cilium https://helm.cilium.io
helm repo update
helm install tetragon cilium/tetragon -n kube-system

# 결과
Error: repository name (cilium) already exists, please specify a different name
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "metrics-server" chart repository
...Successfully got an update from the "cilium" chart repository
Update Complete. ⎈Happy Helming!⎈
NAME: tetragon
LAST DEPLOYED: Sun Sep  7 11:44:41 2025
NAMESPACE: kube-system
STATUS: deployed
REVISION: 1
TEST SUITE: None

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl rollout status -n kube-system ds/tetragon -w

# 결과
daemon set "tetragon" successfully rolled out

2. Tetragon Pod 및 구성 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# k -n kube-system get deploy tetragon-operator -owide
k -n kube-system get cm tetragon-operator-config tetragon-config

✅ 출력

NAME                READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS          IMAGES                                    SELECTOR
tetragon-operator   1/1     1            1           59s   tetragon-operator   quay.io/cilium/tetragon-operator:v1.5.0   app.kubernetes.io/instance=tetragon,app.kubernetes.io/name=tetragon-operator

NAME                       DATA   AGE
tetragon-operator-config   9      59s
tetragon-config            33     59s

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# k -n kube-system get ds tetragon -owide
k -n kube-system get svc,ep tetragon
k -n kube-system get svc,ep tetragon-operator-metrics

✅ 출력

  
NAME       DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE    CONTAINERS               IMAGES                                                                      SELECTOR
tetragon   3         3         3       3            3           <none>          108s   export-stdout,tetragon   quay.io/cilium/hubble-export-stdout:v1.1.0,quay.io/cilium/tetragon:v1.5.0   app.kubernetes.io/instance=tetragon,app.kubernetes.io/name=tetragon

Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
NAME               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/tetragon   ClusterIP   10.96.203.139   <none>        2112/TCP   108s

NAME                 ENDPOINTS                                                     AGE
endpoints/tetragon   192.168.10.100:2112,192.168.10.101:2112,192.168.10.102:2112   108s

Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice
NAME                                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/tetragon-operator-metrics   ClusterIP   10.96.239.199   <none>        2113/TCP   108s

NAME                                  ENDPOINTS         AGE
endpoints/tetragon-operator-metrics   172.20.1.5:2113   108s

  
`(⎈|HomeLab:N/A) root@k8s-ctr:~# k -n kube-system get pod -l app.kubernetes.io/part-of=tetragon -owide
k -n kube-system get pod -l app.kubernetes.io/name=tetragon
kc -n kube-system describe pod -l app.kubernetes.io/name=tetragon

✅ 출력

  
NAME                                READY   STATUS    RESTARTS   AGE     IP               NODE      NOMINATED NODE   READINESS GATES
tetragon-8h56q                      2/2     Running   0          2m42s   192.168.10.102   k8s-w2    <none>           <none>
tetragon-fpq52                      2/2     Running   0          2m42s   192.168.10.100   k8s-ctr   <none>           <none>
tetragon-operator-58c6ddf88-5bfv6   1/1     Running   0          2m42s   172.20.1.5       k8s-w1    <none>           <none>
tetragon-xls2q                      2/2     Running   0          2m42s   192.168.10.101   k8s-w1    <none>           <none>

NAME             READY   STATUS    RESTARTS   AGE
tetragon-8h56q   2/2     Running   0          2m42s
tetragon-fpq52   2/2     Running   0          2m42s
tetragon-xls2q   2/2     Running   0          2m42s

Name:             tetragon-8h56q
Namespace:        kube-system
Priority:         0
Service Account:  tetragon
Node:             k8s-w2/192.168.10.102
Start Time:       Sun, 07 Sep 2025 11:44:42 +0900
Labels:           app.kubernetes.io/component=agent
                  app.kubernetes.io/instance=tetragon
                  app.kubernetes.io/managed-by=Helm
                  app.kubernetes.io/name=tetragon
                  app.kubernetes.io/part-of=tetragon
                  app.kubernetes.io/version=1.5.0
                  controller-revision-hash=9d9c7b5dd
                  helm.sh/chart=tetragon-1.5.0
                  pod-template-generation=1
Annotations:      checksum/configmap: 4b7a67f7d1778591e8c448707a378338bbd1353a88d4ea0e707be33d31db59e8
Status:           Running
IP:               192.168.10.102
IPs:
  IP:           192.168.10.102
Controlled By:  DaemonSet/tetragon
Containers:
  export-stdout:
    Container ID:  containerd://13c801156a13e828502827f4f36f01fc1ac0bbb0f35165424b98466a1de6ed81
    Image:         quay.io/cilium/hubble-export-stdout:v1.1.0
    Image ID:      quay.io/cilium/hubble-export-stdout@sha256:e1549e5852e32223e6db4e44ef883ed86c7659c260986b35c9a015e33d874437
    Port:          <none>
    Host Port:     <none>
    Command:
      hubble-export-stdout
    Args:
      /var/run/cilium/tetragon/tetragon.log
    State:          Running
      Started:      Sun, 07 Sep 2025 11:44:48 +0900
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/cilium/tetragon from export-logs (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-r4jvt (ro)
  tetragon:
    Container ID:  containerd://6906b3eb223092e25af31fb3ecd30ad8f11408337d754117d9726d9f6df4dfe5
    Image:         quay.io/cilium/tetragon:v1.5.0
    Image ID:      quay.io/cilium/tetragon@sha256:3f9ab530fa832c9c80244aa6776ba794abbd9c313ce6c3177a96b3991b01469c
    Port:          <none>
    Host Port:     <none>
    Args:
      --config-dir=/etc/tetragon/tetragon.conf.d/
    State:          Running
      Started:      Sun, 07 Sep 2025 11:45:02 +0900
    Ready:          True
    Restart Count:  0
    Liveness:       grpc <pod>:6789 liveness delay=0s timeout=60s period=10s #success=1 #failure=3
    Environment:
      NODE_NAME:   (v1:spec.nodeName)
    Mounts:
      /etc/tetragon/tetragon.conf.d/ from tetragon-config (ro)
      /procRoot from host-proc (rw)
      /sys/fs/bpf from bpf-maps (rw)
      /var/run/cilium from cilium-run (rw)
      /var/run/cilium/tetragon from export-logs (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-r4jvt (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True 
  Initialized                 True 
  Ready                       True 
  ContainersReady             True 
  PodScheduled                True 
Volumes:
  cilium-run:
    Type:          HostPath (bare host directory volume)
    Path:          /var/run/cilium
    HostPathType:  DirectoryOrCreate
  export-logs:
    Type:          HostPath (bare host directory volume)
    Path:          /var/run/cilium/tetragon
    HostPathType:  DirectoryOrCreate
  tetragon-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      tetragon-config
    Optional:  false
  bpf-maps:
    Type:          HostPath (bare host directory volume)
    Path:          /sys/fs/bpf
    HostPathType:  DirectoryOrCreate
  host-proc:
    Type:          HostPath (bare host directory volume)
    Path:          /proc
    HostPathType:  Directory
  kube-api-access-r4jvt:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    Optional:                false
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 op=Exists
                             node.kubernetes.io/disk-pressure:NoSchedule op=Exists
                             node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                             node.kubernetes.io/network-unavailable:NoSchedule op=Exists
                             node.kubernetes.io/not-ready:NoExecute op=Exists
                             node.kubernetes.io/pid-pressure:NoSchedule op=Exists
                             node.kubernetes.io/unreachable:NoExecute op=Exists
                             node.kubernetes.io/unschedulable:NoSchedule op=Exists
Events:
  Type    Reason     Age    From               Message
  ----    ------     ----   ----               -------
  Normal  Scheduled  2m42s  default-scheduler  Successfully assigned kube-system/tetragon-8h56q to k8s-w2
  Normal  Pulling    2m42s  kubelet            Pulling image "quay.io/cilium/hubble-export-stdout:v1.1.0"
  Normal  Pulled     2m36s  kubelet            Successfully pulled image "quay.io/cilium/hubble-export-stdout:v1.1.0" in 5.717s (5.717s including waiting). Image size: 1524888 bytes.
  Normal  Created    2m36s  kubelet            Created container: export-stdout
  Normal  Started    2m36s  kubelet            Started container export-stdout
  Normal  Pulling    2m36s  kubelet            Pulling image "quay.io/cilium/tetragon:v1.5.0"
  Normal  Pulled     2m23s  kubelet            Successfully pulled image "quay.io/cilium/tetragon:v1.5.0" in 13.387s (13.387s including waiting). Image size: 77257092 bytes.
  Normal  Created    2m23s  kubelet            Created container: tetragon
  Normal  Started    2m22s  kubelet            Started container tetragon

Name:             tetragon-fpq52
Namespace:        kube-system
Priority:         0
Service Account:  tetragon
Node:             k8s-ctr/192.168.10.100
Start Time:       Sun, 07 Sep 2025 11:44:42 +0900
Labels:           app.kubernetes.io/component=agent
                  app.kubernetes.io/instance=tetragon
                  app.kubernetes.io/managed-by=Helm
                  app.kubernetes.io/name=tetragon
                  app.kubernetes.io/part-of=tetragon
                  app.kubernetes.io/version=1.5.0
                  controller-revision-hash=9d9c7b5dd
                  helm.sh/chart=tetragon-1.5.0
                  pod-template-generation=1
Annotations:      checksum/configmap: 4b7a67f7d1778591e8c448707a378338bbd1353a88d4ea0e707be33d31db59e8
Status:           Running
IP:               192.168.10.100
IPs:
  IP:           192.168.10.100
Controlled By:  DaemonSet/tetragon
Containers:
  export-stdout:
    Container ID:  containerd://423fb266b39671cd0603498e93205f670c09ad0955731bc084f7dd1e0c9a1bed
    Image:         quay.io/cilium/hubble-export-stdout:v1.1.0
    Image ID:      quay.io/cilium/hubble-export-stdout@sha256:e1549e5852e32223e6db4e44ef883ed86c7659c260986b35c9a015e33d874437
    Port:          <none>
    Host Port:     <none>
    Command:
      hubble-export-stdout
    Args:
      /var/run/cilium/tetragon/tetragon.log
    State:          Running
      Started:      Sun, 07 Sep 2025 11:44:47 +0900
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/cilium/tetragon from export-logs (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-k2tqd (ro)
  tetragon:
    Container ID:  containerd://f61dc4c727a6b39b7cd46841345503f7dbdb98f08b4e1ca79cffa3715b4f945f
    Image:         quay.io/cilium/tetragon:v1.5.0
    Image ID:      quay.io/cilium/tetragon@sha256:3f9ab530fa832c9c80244aa6776ba794abbd9c313ce6c3177a96b3991b01469c
    Port:          <none>
    Host Port:     <none>
    Args:
      --config-dir=/etc/tetragon/tetragon.conf.d/
    State:          Running
      Started:      Sun, 07 Sep 2025 11:45:01 +0900
    Ready:          True
    Restart Count:  0
    Liveness:       grpc <pod>:6789 liveness delay=0s timeout=60s period=10s #success=1 #failure=3
    Environment:
      NODE_NAME:   (v1:spec.nodeName)
    Mounts:
      /etc/tetragon/tetragon.conf.d/ from tetragon-config (ro)
      /procRoot from host-proc (rw)
      /sys/fs/bpf from bpf-maps (rw)
      /var/run/cilium from cilium-run (rw)
      /var/run/cilium/tetragon from export-logs (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-k2tqd (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True 
  Initialized                 True 
  Ready                       True 
  ContainersReady             True 
  PodScheduled                True 
Volumes:
  cilium-run:
    Type:          HostPath (bare host directory volume)
    Path:          /var/run/cilium
    HostPathType:  DirectoryOrCreate
  export-logs:
    Type:          HostPath (bare host directory volume)
    Path:          /var/run/cilium/tetragon
    HostPathType:  DirectoryOrCreate
  tetragon-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      tetragon-config
    Optional:  false
  bpf-maps:
    Type:          HostPath (bare host directory volume)
    Path:          /sys/fs/bpf
    HostPathType:  DirectoryOrCreate
  host-proc:
    Type:          HostPath (bare host directory volume)
    Path:          /proc
    HostPathType:  Directory
  kube-api-access-k2tqd:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    Optional:                false
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 op=Exists
                             node.kubernetes.io/disk-pressure:NoSchedule op=Exists
                             node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                             node.kubernetes.io/network-unavailable:NoSchedule op=Exists
                             node.kubernetes.io/not-ready:NoExecute op=Exists
                             node.kubernetes.io/pid-pressure:NoSchedule op=Exists
                             node.kubernetes.io/unreachable:NoExecute op=Exists
                             node.kubernetes.io/unschedulable:NoSchedule op=Exists
Events:
  Type    Reason     Age    From               Message
  ----    ------     ----   ----               -------
  Normal  Scheduled  2m42s  default-scheduler  Successfully assigned kube-system/tetragon-fpq52 to k8s-ctr
  Normal  Pulling    2m42s  kubelet            Pulling image "quay.io/cilium/hubble-export-stdout:v1.1.0"
  Normal  Pulled     2m37s  kubelet            Successfully pulled image "quay.io/cilium/hubble-export-stdout:v1.1.0" in 5.123s (5.123s including waiting). Image size: 1524888 bytes.
  Normal  Created    2m37s  kubelet            Created container: export-stdout
  Normal  Started    2m37s  kubelet            Started container export-stdout
  Normal  Pulling    2m37s  kubelet            Pulling image "quay.io/cilium/tetragon:v1.5.0"
  Normal  Pulled     2m23s  kubelet            Successfully pulled image "quay.io/cilium/tetragon:v1.5.0" in 13.679s (13.679s including waiting). Image size: 77257092 bytes.
  Normal  Created    2m23s  kubelet            Created container: tetragon
  Normal  Started    2m23s  kubelet            Started container tetragon

Name:             tetragon-xls2q
Namespace:        kube-system
Priority:         0
Service Account:  tetragon
Node:             k8s-w1/192.168.10.101
Start Time:       Sun, 07 Sep 2025 11:44:42 +0900
Labels:           app.kubernetes.io/component=agent
                  app.kubernetes.io/instance=tetragon
                  app.kubernetes.io/managed-by=Helm
                  app.kubernetes.io/name=tetragon
                  app.kubernetes.io/part-of=tetragon
                  app.kubernetes.io/version=1.5.0
                  controller-revision-hash=9d9c7b5dd
                  helm.sh/chart=tetragon-1.5.0
                  pod-template-generation=1
Annotations:      checksum/configmap: 4b7a67f7d1778591e8c448707a378338bbd1353a88d4ea0e707be33d31db59e8
Status:           Running
IP:               192.168.10.101
IPs:
  IP:           192.168.10.101
Controlled By:  DaemonSet/tetragon
Containers:
  export-stdout:
    Container ID:  containerd://cfcc56cdf2a8b3b549322adc6563696a55fe54b9cadda67bdaa27fdfe6203ee1
    Image:         quay.io/cilium/hubble-export-stdout:v1.1.0
    Image ID:      quay.io/cilium/hubble-export-stdout@sha256:e1549e5852e32223e6db4e44ef883ed86c7659c260986b35c9a015e33d874437
    Port:          <none>
    Host Port:     <none>
    Command:
      hubble-export-stdout
    Args:
      /var/run/cilium/tetragon/tetragon.log
    State:          Running
      Started:      Sun, 07 Sep 2025 11:44:47 +0900
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/cilium/tetragon from export-logs (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-ndq5m (ro)
  tetragon:
    Container ID:  containerd://c71f95d322d8d2b5315230b03adef4f3314e7a371d924c721aa90af1bee9d71e
    Image:         quay.io/cilium/tetragon:v1.5.0
    Image ID:      quay.io/cilium/tetragon@sha256:3f9ab530fa832c9c80244aa6776ba794abbd9c313ce6c3177a96b3991b01469c
    Port:          <none>
    Host Port:     <none>
    Args:
      --config-dir=/etc/tetragon/tetragon.conf.d/
    State:          Running
      Started:      Sun, 07 Sep 2025 11:45:10 +0900
    Ready:          True
    Restart Count:  0
    Liveness:       grpc <pod>:6789 liveness delay=0s timeout=60s period=10s #success=1 #failure=3
    Environment:
      NODE_NAME:   (v1:spec.nodeName)
    Mounts:
      /etc/tetragon/tetragon.conf.d/ from tetragon-config (ro)
      /procRoot from host-proc (rw)
      /sys/fs/bpf from bpf-maps (rw)
      /var/run/cilium from cilium-run (rw)
      /var/run/cilium/tetragon from export-logs (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-ndq5m (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True 
  Initialized                 True 
  Ready                       True 
  ContainersReady             True 
  PodScheduled                True 
Volumes:
  cilium-run:
    Type:          HostPath (bare host directory volume)
    Path:          /var/run/cilium
    HostPathType:  DirectoryOrCreate
  export-logs:
    Type:          HostPath (bare host directory volume)
    Path:          /var/run/cilium/tetragon
    HostPathType:  DirectoryOrCreate
  tetragon-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      tetragon-config
    Optional:  false
  bpf-maps:
    Type:          HostPath (bare host directory volume)
    Path:          /sys/fs/bpf
    HostPathType:  DirectoryOrCreate
  host-proc:
    Type:          HostPath (bare host directory volume)
    Path:          /proc
    HostPathType:  Directory
  kube-api-access-ndq5m:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    Optional:                false
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 op=Exists
                             node.kubernetes.io/disk-pressure:NoSchedule op=Exists
                             node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                             node.kubernetes.io/network-unavailable:NoSchedule op=Exists
                             node.kubernetes.io/not-ready:NoExecute op=Exists
                             node.kubernetes.io/pid-pressure:NoSchedule op=Exists
                             node.kubernetes.io/unreachable:NoExecute op=Exists
                             node.kubernetes.io/unschedulable:NoSchedule op=Exists
Events:
  Type    Reason     Age    From               Message
  ----    ------     ----   ----               -------
  Normal  Scheduled  2m42s  default-scheduler  Successfully assigned kube-system/tetragon-xls2q to k8s-w1
  Normal  Pulling    2m42s  kubelet            Pulling image "quay.io/cilium/hubble-export-stdout:v1.1.0"
  Normal  Pulled     2m37s  kubelet            Successfully pulled image "quay.io/cilium/hubble-export-stdout:v1.1.0" in 4.382s (4.382s including waiting). Image size: 1524888 bytes.
  Normal  Created    2m37s  kubelet            Created container: export-stdout
  Normal  Started    2m37s  kubelet            Started container export-stdout
  Normal  Pulling    2m37s  kubelet            Pulling image "quay.io/cilium/tetragon:v1.5.0"
  Normal  Pulled     2m14s  kubelet            Successfully pulled image "quay.io/cilium/tetragon:v1.5.0" in 10.055s (23.493s including waiting). Image size: 77257092 bytes.
  Normal  Created    2m14s  kubelet            Created container: tetragon
  Normal  Started    2m14s  kubelet            Started container tetragon

3. 데모 애플리케이션 배포

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl create -f https://raw.githubusercontent.com/cilium/cilium/v1.18.1/examples/minikube/http-sw-app.yaml

# 결과
service/deathstar created
deployment.apps/deathstar created
pod/tiefighter created
pod/xwing created

4. 프로세스 실행 모니터링 (Execution Events)

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# POD=$(kubectl -n kube-system get pods -l 'app.kubernetes.io/name=tetragon' -o name --field-selector spec.nodeName=$(kubectl get pod xwing -o jsonpath='{.spec.nodeName}'))
echo $POD

pod/tetragon-8h56q

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -ti -n kube-system $POD -c tetragon -- tetra getevents -o compact --pods xwing

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -ti xwing -- bash -c 'curl https://ebpf.io/applications/#tetragon'

✅ 출력

  
<!DOCTYPE html><html lang="en" prefix="og: http://ogp.me/ns#"><head><meta charSet="utf-8"/><meta http-equiv="x-ua-compatible" content="ie=edge"/><meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no, viewport-fit=cover"/><link rel="preload" href="/fonts/elza/elza-semibold.woff2" as="font" type="font/woff2" crossorigin="anonymous"/><link rel="preload" href="/fonts/elza-text/elza-text-medium.woff2" as="font" type="font/woff2" crossorigin="anonymous"/><meta name="generator" content="Gatsby 5.14.6"/><meta name="description" content="A directory of eBPF-based open source applications" data-gatsby-head="true"/><meta name="keywords" content="ebpf, bpf, landscape, directory, open source" data-gatsby-head="true"/><meta property="og:title" content="eBPF Applications Landscape" data-gatsby-head="true"/><meta property="og:description" content="A directory of eBPF-based open source applications" data-gatsby-head="true"/><meta property="og:url" content="https://ebpf.io/applications/" data-gatsby-head="true"/><meta property="og:image" content="https://ebpf.io/images/social-preview.jpg" data-gatsby-head="true"/><meta property="og:type" content="website" data-gatsby-head="true"/><meta name="twitter:card" content="summary_large_image" data-gatsby-head="true"/><style data-href="/styles.6dc50e72f6ab2660031c.css" data-identity="gatsby-global-css">/*
! tailwindcss v3.4.7 | MIT License | https://tailwindcss.com
....

  
🚀 process default/xwing /usr/bin/bash -c "curl https://ebpf.io/applications/#tetragon" 
🚀 process default/xwing /usr/bin/curl https://ebpf.io/applications/#tetragon 
💥 exit    default/xwing /usr/bin/curl https://ebpf.io/applications/#tetragon 0 

xwing 파드 내에서 curl 실행 시 Tetragon 로그에 다음 이벤트 기록됨
- bash -c "curl https://ebpf.io/applications/#tetragon" 실행 이벤트 감지
- curl 프로세스 실행 및 종료(exit 0) 이벤트 감지

5. 네트워크 액세스 모니터링

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -ti xwing -- bash -c 'curl https://httpbin.org'

✅ 출력

  
<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <title>httpbin.org</title>
    <link href="https://fonts.googleapis.com/css?family=Open+Sans:400,700|Source+Code+Pro:300,600|Titillium+Web:400,600,700"
        rel="stylesheet">
    <link rel="stylesheet" type="text/css" href="/flasgger_static/swagger-ui.css">
    <link rel="icon" type="image/png" href="/static/favicon.ico" sizes="64x64 32x32 16x16" />
    <style>
        html {
            box-sizing: border-box;
            overflow: -moz-scrollbars-vertical;
            overflow-y: scroll;
        }

        *,
        *:before,
        *:after {
            box-sizing: inherit;
        }

        body {
            margin: 0;
            background: #fafafa;
        }
    </style>
</head>
...

  
🚀 process default/xwing /usr/bin/bash -c "curl https://httpbin.org"      
🚀 process default/xwing /usr/bin/curl https://httpbin.org                
💥 exit    default/xwing /usr/bin/curl https://httpbin.org 0  

xwing 파드에서 curl https://httpbin.org 실행
외부 사이트 접근 이벤트가 프로세스 실행 이벤트로 기록됨
curl 프로세스 실행 및 종료 상태 코드(0) 확인

6. 파일 접근 모니터링

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -ti xwing -- bash -c 'cat /etc/passwd'

✅ 출력

root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/usr/sbin/nologin
man:x:6:12:man:/var/cache/man:/usr/sbin/nologin
lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin
mail:x:8:8:mail:/var/mail:/usr/sbin/nologin
news:x:9:9:news:/var/spool/news:/usr/sbin/nologin
uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin
proxy:x:13:13:proxy:/bin:/usr/sbin/nologin
www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin
backup:x:34:34:backup:/var/backups:/usr/sbin/nologin
list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin
irc:x:39:39:ircd:/run/ircd:/usr/sbin/nologin
_apt:x:42:65534::/nonexistent:/usr/sbin/nologin
nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin
node:x:1000:1000::/home/node:/bin/bash

  
🚀 process default/xwing /usr/bin/bash -c "cat /etc/passwd"               
🚀 process default/xwing /usr/bin/cat /etc/passwd                         
💥 exit    default/xwing /usr/bin/cat /etc/passwd 0  

xwing 파드에서 cat /etc/passwd 실행
해당 명령 실행 및 프로세스 종료 이벤트가 Tetragon 로그에 기록
시스템 내부 파일 접근까지 eBPF 레벨에서 추적 가능함을 확인

👀 파일 접속 모니터링 : 추적 정책으로 민감 파일 모니터링

1. 파일 모니터링 TracingPolicy 생성

  
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "file-monitoring-filtered"
spec:
  kprobes:
  - call: "security_file_permission"
    syscall: false
    return: true
    args:
    - index: 0
      type: "file" # (struct file *) used for getting the path
    - index: 1
      type: "int" # 0x04 is MAY_READ, 0x02 is MAY_WRITE
    returnArg:
      index: 0
      type: "int"
    returnArgAction: "Post"
    selectors:
    - matchArgs:      
      - index: 0
        operator: "Prefix"
        values:
        - "/boot"           # Reads to sensitive directories
        - "/root/.ssh"      # Reads to sensitive files we want to know about
        - "/etc/shadow"
        - "/etc/profile"
        - "/etc/sudoers"
        - "/etc/pam.conf"   # Reads global shell configs bash/csh supported
        - "/etc/bashrc"
        - "/etc/csh.cshrc"
        - "/etc/csh.login"  # Add additional sensitive files here
      - index: 1
        operator: "Equal"
        values:
        - "4" # MAY_READ
...

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl apply -f https://raw.githubusercontent.com/cilium/tetragon/main/examples/quickstart/file_monitoring.yaml

# 결과
tracingpolicy.cilium.io/file-monitoring-filtered created

(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl get tracingpolicy

✅ 출력

NAME                       AGE
file-monitoring-filtered   39s

2. 민감 파일 읽기 감지 (`/etc/shadow`)

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -ti -n kube-system $POD -c tetragon -- tetra getevents -o compact --pods xwing

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -ti xwing -- bash -c 'cat /etc/shadow'

✅ 출력

  
root:*:19709:0:99999:7:::
daemon:*:19709:0:99999:7:::
bin:*:19709:0:99999:7:::
sys:*:19709:0:99999:7:::
sync:*:19709:0:99999:7:::
games:*:19709:0:99999:7:::
man:*:19709:0:99999:7:::
lp:*:19709:0:99999:7:::
mail:*:19709:0:99999:7:::
news:*:19709:0:99999:7:::
uucp:*:19709:0:99999:7:::
proxy:*:19709:0:99999:7:::
www-data:*:19709:0:99999:7:::
backup:*:19709:0:99999:7:::
list:*:19709:0:99999:7:::
irc:*:19709:0:99999:7:::
_apt:*:19709:0:99999:7:::
nobody:*:19709:0:99999:7:::
node:!:19710:0:99999:7:::

  
🚀 process default/xwing /usr/bin/bash -c "cat /etc/shadow"               
🚀 process default/xwing /usr/bin/cat /etc/shadow                         
📚 read    default/xwing /usr/bin/cat /etc/shadow                         
📚 read    default/xwing /usr/bin/cat /etc/shadow                         
📚 read    default/xwing /usr/bin/cat /etc/shadow                         
📚 read    default/xwing /usr/bin/cat /etc/shadow                         
💥 exit    default/xwing /usr/bin/cat /etc/shadow 0    

xwing 파드에서 cat /etc/shadow 실행
Tetragon 이벤트 로그에서 다음 이벤트 기록 확인
- bash -c "cat /etc/shadow" 실행 감지
- cat /etc/shadow 프로세스 실행 이벤트 감지
- /etc/shadow에 대한 read 이벤트(📚) 다수 기록
- 프로세스 종료(exit 0)까지 추적됨

3. 민감 디렉토리 쓰기 감지 (`/etc/bar`)

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -ti xwing -- bash -c 'echo foo >> /etc/bar'

✅ 출력

  
🚀 process default/xwing /usr/bin/bash -c "echo foo >> /etc/bar"          
📝 write   default/xwing /usr/bin/bash /etc/bar                           
📝 write   default/xwing /usr/bin/bash /etc/bar                           
💥 exit    default/xwing /usr/bin/bash -c "echo foo >> /etc/bar" 0 

xwing 파드에서 echo foo >> /etc/bar 실행
Tetragon 이벤트 로그에서 다음 이벤트 기록 확인
- bash -c "echo foo >> /etc/bar" 실행 감지
- /etc/bar 파일에 대한 write 이벤트(📝) 발생
- 프로세스 종료(exit 0)까지 추적됨

4. TracingPolicy 삭제

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl delete -f https://raw.githubusercontent.com/cilium/tetragon/main/examples/quickstart/file_monitoring.yaml

# 결과
tracingpolicy.cilium.io "file-monitoring-filtered" deleted

🔐 Policy Enforcement : 커널 수준에서 정책 제한을 적용

https://tetragon.io/docs/getting-started/enforcement/

1. TracingPolicyNamespaced 생성 및 강제 액션 추가

  
apiVersion: cilium.io/v1alpha1
kind: TracingPolicyNamespaced
metadata:
  name: "file-monitoring-filtered"
spec:
  kprobes:
  - call: "security_file_permission"
    syscall: false
    return: true
    args:
    - index: 0
      type: "file" # (struct file *) used for getting the path
    - index: 1
      type: "int" # 0x04 is MAY_READ, 0x02 is MAY_WRITE
    returnArg:
      index: 0
      type: "int"
    returnArgAction: "Post"
    selectors:
    - matchArgs:
      - index: 0
        operator: "Prefix"
        values:
        - "/boot"           # Reads to sensitive directories
        - "/root/.ssh"      # Reads to sensitive files we want to know about
        - "/etc/shadow"
        - "/etc/profile"
        - "/etc/sudoers"
        - "/etc/pam.conf"   # Reads global shell configs bash/csh supported
        - "/etc/bashrc"
        - "/etc/csh.cshrc"
        - "/etc/csh.login"  # Add additional sensitive files here
      - index: 1
        operator: "Equal"
        values:
        - "4" # MAY_READ
      matchActions:
      - action: Sigkill
  ...      

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl apply -f https://raw.githubusercontent.com/cilium/tetragon/main/examples/quickstart/file_monitoring_enforce.yaml

# 결과
tracingpolicynamespaced.cilium.io/file-monitoring-filtered created

2. 민감 파일(/etc/shadow) 읽기 시도 차단 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -ti -n kube-system $POD -c tetragon -- tetra getevents -o compact --pods xwing

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -ti xwing -- bash -c 'cat /etc/shadow'

✅ 출력

command terminated with exit code 137

  
🚀 process default/xwing /usr/bin/bash -c "cat /etc/shadow"               
🚀 process default/xwing /usr/bin/cat /etc/shadow                         
📚 read    default/xwing /usr/bin/cat /etc/shadow                         
📚 read    default/xwing /usr/bin/cat /etc/shadow                         
💥 exit    default/xwing /usr/bin/cat /etc/shadow SIGKILL     

xwing 파드에서 cat /etc/shadow 실행
Tetragon 이벤트에서 read 이벤트(📚) 발생 후, SIGKILL로 프로세스 강제 종료됨 확인
실제 출력: command terminated with exit code 137 → 정책이 실제로 차단 동작한 것을 의미함

3. 민감하지 않은 파일 접근 허용 확인

  
(⎈|HomeLab:N/A) root@k8s-ctr:~# kubectl exec -ti xwing -- bash -c 'echo foo > /tmp/test.txt'

✅ 출력

  
🚀 process default/xwing /usr/bin/bash -c "echo foo > /tmp/test.txt"      
💥 exit    default/xwing /usr/bin/bash -c "echo foo > /tmp/test.txt" 0 

/tmp/test.txt 파일에 쓰기 시도 (echo foo > /tmp/test.txt)
Tetragon 이벤트 로그에서 write 이벤트(📝) 가 기록되었으나 차단되지 않고 정상 종료됨
정책에 포함되지 않은 파일에 대해서는 영향을 주지 않는 것을 확인

Cilium

This post is licensed under CC BY 4.0 by the author.

🔧 실습 환경 구성

1. Vagrantfile 다운로드 및 가상머신 구성

2. VM 부팅 타임아웃 오류 해결

3. 모니터링 및 관리 도구 접속 확인

📑 Identity

1. CiliumEndpoint 리소스로 엔드포인트 확인

2. CiliumIdentity 리소스 조회

3. Cilium Agent에서 Identity 라벨 상세 확인

4. 라벨 변경 후 Identity 재할당 확인

5. Special Identities 확인

🚀 [Lab1] DNS 기반 보안 정책

1. 데모 애플리케이션 배포

2. 외부 통신 동작 확인

🔎 DNS Egress 정책 적용 1:mediabot 파드가 api.github.com에만 액세스하도록 허용

1. CiliumNetworkPolicy 생성

2. 정책 생성 여부 확인

3. Policy Selector 확인

4. Cilium DNS 관련 설정 조회

5. api.github.com 허용 도메인 접근 테스트

6. 허용되지 않은 도메인 차단 확인

7. Hubble Relay 포트포워딩

8. Hubble Observe 통한 DNS Proxy 동작 확인

9. CoreDNS 로그 활성화 및 확인

10. 반복 질의 시도 및 로그 분석

11. DNS Proxy 설정값 확인

12. Cilium Agent 캐시 조회

13. FQDNPolicySelector 확인

🌐 DNS Egress 정책 적용 2: 모든 GitHub 하위 도메인에 액세스

1. FQDN 캐시 및 정책 초기화

2. GitHub 하위 도메인 허용 정책 생성 (*.github.com)

3. 정책 적용 결과 확인

4. GitHub 하위 도메인 접근 테스트

5. 비허용 도메인 접근 테스트

🏷️ DNS Egress 정책 적용 3 : DNS, Port 조합 적용

1. 기존 정책 및 캐시 초기화

2. DNS, Port 조합 정책 생성

3. 정책 적용 결과 확인

4. 도메인 접근 테스트

5. 실습 리소스 정리

📦 샘플 애플리케이션 배포 및 통신 문제 확인

1. 샘플 애플리케이션 배포

2. k8s-ctr 노드에 curl-pod 파드 배포

🔁 [Lab2] WireGuard 설정 및 실습 : 터널 모드는 두 번 캡슐화됨

1. 커널 WireGuard 지원 확인

2. WireGuard 활성화 전 네트워크 상태 확인

3. Cilium WireGuard 암호화 활성화

4. 암호화 상태 및 키 확인

5. WireGuard 인터페이스 확인

6. WireGuard 상세 정보 확인

7. Pod 간 통신 확인 (암호화 경로 테스트)

8. 물리 인터페이스 트래픽 캡처

9. 패킷 저장 및 복호화 불가 확인

10. Hubble 로깅 확인

11. 설정 원복

⚙️ Inspecting TLS Encrypted Connections with Cilium

1. cilium-secrets 네임스페이스 존재 확인

2. TLS 설정 파일 생성

3. Cilium 재배포 (TLS 설정 반영)

4. TLS Secret 동기화 설정 확인

🛡️ [Lab3] Cilium TLS Interception Demo 실습

1. Demo 애플리케이션 배포 (mediabot)

2. 내부 CA 키 생성

3. CA 인증서 생성

4. 대상 서비스용 개인 키 및 CSR 생성

5. CA로 서명된 인증서 발급

6. Kubernetes Secret 생성 (Terminating TLS)

7. 클라이언트 파드에 내부 CA 추가

8. Cilium에 신뢰할 수 있는 CA 목록 제공

📑 Apply DNS and TLS-aware Egress Policy

1. CiliumNetworkPolicy(l7-visibility-tls) 작성

2. 정책 적용 전 트래픽 상태 확인

3. 정책 적용 (TLS 인식 Egress Policy)

4. 정책 적용 후 L7 가시성 확인

5. 서버 인증서 확인 (변경 확인)

6. 실습 리소스 삭제

🕵️ Tetragon

1. Tetragon 설치

2. Tetragon Pod 및 구성 확인

3. 데모 애플리케이션 배포

4. 프로세스 실행 모니터링 (Execution Events)

🔎 DNS Egress 정책 적용 1:`mediabot` 파드가 `api.github.com`에만 액세스하도록 허용

5. `api.github.com` 허용 도메인 접근 테스트

**2. GitHub 하위 도메인 허용 정책 생성 (`*.github.com`)**

1. `cilium-secrets` 네임스페이스 존재 확인

1. `CiliumNetworkPolicy(l7-visibility-tls)` 작성

2. 민감 파일 읽기 감지 (`/etc/shadow`)

3. 민감 디렉토리 쓰기 감지 (`/etc/bar`)