Post

CI/CD 3์ฃผ์ฐจ ์ •๋ฆฌ

Posted
By GAGA
33 min read

๐Ÿš€ kind๋กœ k8s ๋ฐฐํฌ

1. kind ํด๋Ÿฌ์Šคํ„ฐ ์ƒ์„ฑ

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

kind create cluster --name myk8s --image kindest/node:v1.32.8 --config - <<EOF
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  apiServerAddress: "0.0.0.0"
nodes:
- role: control-plane
  extraPortMappings:
  - containerPort: 30000
    hostPort: 30000
  - containerPort: 30001
    hostPort: 30001
  - containerPort: 30002
    hostPort: 30002
  - containerPort: 30003
    hostPort: 30003
- role: worker
EOF

# ๊ฒฐ๊ณผ
Creating cluster "myk8s" ...
 โœ“ Ensuring node image (kindest/node:v1.32.8) ๐Ÿ–ผ
 โœ“ Preparing nodes ๐Ÿ“ฆ ๐Ÿ“ฆ  
 โœ“ Writing configuration ๐Ÿ“œ 
 โœ“ Starting control-plane ๐Ÿ•น๏ธ 
 โœ“ Installing CNI ๐Ÿ”Œ 
 โœ“ Installing StorageClass ๐Ÿ’พ 
 โœ“ Joining worker nodes ๐Ÿšœ 
Set kubectl context to "kind-myk8s"
You can now use your cluster with:

kubectl cluster-info --context kind-myk8s

Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community ๐Ÿ™‚

2. ๋…ธ๋“œ/๋„ค์ž„์ŠคํŽ˜์ด์Šค ํ™•์ธ

1
2
3
4
5

kind get nodes --name myk8s

# ๊ฒฐ๊ณผ
myk8s-worker
myk8s-control-plane

1
2
3
4

kubens default

# ๊ฒฐ๊ณผ
โœ” Active namespace is "default"

3. Docker ์ปจํ…Œ์ด๋„ˆ ํ™•์ธ ๋ฐ API ํฌํŠธ ๋งคํ•‘

1
2
3
4
5

docker ps

CONTAINER ID   IMAGE                  COMMAND                  CREATED              STATUS              PORTS                                                           NAMES
ea4fa45f8d82   kindest/node:v1.32.8   "/usr/local/bin/entrโ€ฆ"   About a minute ago   Up About a minute   0.0.0.0:30000-30003->30000-30003/tcp, 0.0.0.0:44501->6443/tcp   myk8s-control-plane
665b94422e49   kindest/node:v1.32.8   "/usr/local/bin/entrโ€ฆ"   About a minute ago   Up About a minute                                                                   myk8s-worker
  • ์ปจํŠธ๋กคํ”Œ๋ ˆ์ธ: 0.0.0.0:44501 -> 6443 (K8s API)
  • NodePort: 30000~30003 ํ˜ธ์ŠคํŠธ์™€ ๋งคํ•‘

4. ํ˜ธ์ŠคํŠธ IP ํ™•์ธ ๋ฐ K8s API ํ˜ธ์ถœ

1
2
3
4
5

ifconfig | grep 192.

# ๊ฒฐ๊ณผ
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        inet 192.168.219.107  netmask 255.255.255.0  broadcast 192.168.219.255

1
2
3
4
5
6
7
8
9
10
11
12
13
14

curl https://192.168.219.107:44501/version -k

# ๊ฒฐ๊ณผ
{
  "major": "1",
  "minor": "32",
  "gitVersion": "v1.32.8",
  "gitCommit": "2e83bc4bf31e88b7de81d5341939d5ce2460f46f",
  "gitTreeState": "clean",
  "buildDate": "2025-08-13T14:21:22Z",
  "goVersion": "go1.23.11",
  "compiler": "gc",
  "platform": "linux/amd64"
}%

๐Ÿงฉ docker compose

1. CI/CD ํ”„๋กœ์ ํŠธ ๋””๋ ‰ํ„ฐ๋ฆฌ ์ค€๋น„

1
2

mkdir cicd-labs
cd cicd-labs

1
2
3
4
5
6
7

docker network ls

# ๊ฒฐ๊ณผ
NETWORK ID     NAME      DRIVER    SCOPE
...
1da18f85ffec   kind      bridge    local
...

2. docker-compose ๊ตฌ์„ฑ (Jenkins + Gogs)

1

export DOCKER_GID=$(stat -c '%g' /var/run/docker.sock)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

cat <<EOT > docker-compose.yaml
services:

  jenkins:
    container_name: jenkins
    image: jenkins/jenkins
    restart: unless-stopped
    networks:
      - kind
    ports:
      - "8080:8080"
      - "50000:50000"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - jenkins_home:/var/jenkins_home
    group_add:
      - "${DOCKER_GID}"

  gogs:
    container_name: gogs
    image: gogs/gogs
    restart: unless-stopped
    networks:
      - kind
    ports:
      - "10022:22"
      - "3000:3000"
    volumes:
      - gogs-data:/data

volumes:
  jenkins_home:
  gogs-data:

networks:
  kind:
    external: true
EOT

3. ์ปจํ…Œ์ด๋„ˆ ๊ธฐ๋™ ๋ฐ ์ƒํƒœ ํ™•์ธ

1
2
3
4
5
6
7
8

docker compose up -d

# ๊ฒฐ๊ณผ
[+] Running 4/4
 โœ” Volume cicd-labs_jenkins_home  Created                                                                     0.0s 
 โœ” Volume cicd-labs_gogs-data     Created                                                                     0.0s 
 โœ” Container gogs                 Started                                                                     0.2s 
 โœ” Container jenkins              Started                                                                     0.2s 

1
2
3
4
5

docker compose ps

NAME      IMAGE             COMMAND                  SERVICE   CREATED          STATUS                             PORTS
gogs      gogs/gogs         "/app/gogs/docker/stโ€ฆ"   gogs      30 seconds ago   Up 29 seconds (health: starting)   0.0.0.0:3000->3000/tcp, [::]:3000->3000/tcp, 0.0.0.0:10022->22/tcp, [::]:10022->22/tcp
jenkins   jenkins/jenkins   "/usr/bin/tini -- /uโ€ฆ"   jenkins   30 seconds ago   Up 29 seconds                      0.0.0.0:8080->8080/tcp, [::]:8080->8080/tcp, 0.0.0.0:50000->50000/tcp, [::]:50000->50000/tcp

1
2
3
4
5
6
7
8
9

for i in gogs jenkins ; do echo ">> container : $i <<"; docker compose exec $i sh -c "whoami && pwd"; echo; done

>> container : gogs <<
root
/app/gogs

>> container : jenkins <<
jenkins
/

4. ๋„์ปค๋ฅผ ์ด์šฉํ•˜์—ฌ ๊ฐ ์ปจํ…Œ์ด๋„ˆ๋กœ ์ ‘์†

1
2
3

docker compose exec jenkins bash
jenkins@4aec55fcb34f:/$ exit
exit

1
2
3

docker compose exec gogs bash
f5292520283a:/app/gogs# exit
exit

5. Jenkins ์ปจํ…Œ์ด๋„ˆ ์ดˆ๊ธฐ ์„ค์ •

1
2

docker compose exec jenkins cat /var/jenkins_home/secrets/initialAdminPassword
dc92d4a5e233407f9fbb1dfaf986e75a # ํŒจ์Šค์›Œ๋“œ ์ž…๋ ฅ

(1) ์›น ์ ‘์†: http:<์ž์‹ ์˜ IP>:8080 ์ง„์ž… ํ›„, ๊ถŒ์žฅ ํ”Œ๋Ÿฌ๊ทธ์ธ ์„ค์น˜

(2) ์‚ฌ์šฉ์ž ์ƒ์„ฑ: devops / qwe123

(3) ์„ค์ •์™„๋ฃŒ

๐ŸŽฏ Jenkins ์ปจํ…Œ์ด๋„ˆ์—์„œ ํ˜ธ์ŠคํŠธ์— ๋„์ปค ๋ฐ๋ชฌ ์‚ฌ์šฉ ์„ค์ •

1. ์ปจํ…Œ์ด๋„ˆ ๋‚ด๋ถ€ Docker CLI ์„ค์น˜

1
2
3

docker compose exec --privileged -u root jenkins bash
root@4aec55fcb34f:/# id
uid=0(root) gid=0(root) groups=0(root)

1
2
3
4
5
6
7

root@4aec55fcb34f:/# curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
chmod a+r /etc/apt/keyrings/docker.asc
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  tee /etc/apt/sources.list.d/docker.list > /dev/null
apt-get update && apt install docker-ce-cli curl tree jq yq -y

2. ํ˜ธ์ŠคํŠธ ๋ฐ๋ชฌ ์—ฐ๊ฒฐ ํ™•์ธ

1
2
3
4
5
6

root@4aec55fcb34f:/# docker ps
CONTAINER ID   IMAGE                  COMMAND                  CREATED          STATUS                    PORTS                                                                                          NAMES
4aec55fcb34f   jenkins/jenkins        "/usr/bin/tini -- /uโ€ฆ"   14 minutes ago   Up 14 minutes             0.0.0.0:8080->8080/tcp, [::]:8080->8080/tcp, 0.0.0.0:50000->50000/tcp, [::]:50000->50000/tcp   jenkins
f5292520283a   gogs/gogs              "/app/gogs/docker/stโ€ฆ"   14 minutes ago   Up 14 minutes (healthy)   0.0.0.0:3000->3000/tcp, [::]:3000->3000/tcp, 0.0.0.0:10022->22/tcp, [::]:10022->22/tcp         gogs
ea4fa45f8d82   kindest/node:v1.32.8   "/usr/local/bin/entrโ€ฆ"   17 minutes ago   Up 17 minutes             0.0.0.0:30000-30003->30000-30003/tcp, 0.0.0.0:44501->6443/tcp                                  myk8s-control-plane
665b94422e49   kindest/node:v1.32.8   "/usr/local/bin/entrโ€ฆ"   17 minutes ago   Up 17 minutes                                                                                                            myk8s-worker

1
2

root@a0c3ece04d03:/# which docker
/usr/bin/docker

3. jenkins ์‚ฌ์šฉ์ž ๊ถŒํ•œ ๋ถ€์—ฌ ์‹œ๋„

1
2
3
4
5
6
7
8
9
10

root@4aec55fcb34f:/# chgrp docker /var/run/docker.sock
ls -l /var/run/docker.sock
usermod -aG docker jenkins
cat /etc/group | grep docker
chgrp: invalid group: โ€˜dockerโ€™
srw-rw---- 1 root 957 0 Oct 25 11:35 /var/run/docker.sock
usermod: group 'docker' does not exist

root@4aec55fcb34f:/# exit
exit
  • ์ปจํ…Œ์ด๋„ˆ ๋‚ด๋ถ€์—์„œ usermod -aG docker jenkins, chgrp docker /var/run/docker.sock ์‹œ๋„ํ–ˆ์œผ๋‚˜ docker ๊ทธ๋ฃน ๋ฏธ์กด์žฌ๋กœ ์‹คํŒจํ•จ

4. Jenkins ์ปจํ…Œ์ด๋„ˆ ์žฌ์‹œ์ž‘ ํ›„ ๊ถŒํ•œ ์—๋Ÿฌ ๊ด€์ฐฐ

1
2
3
4

docker compose restart jenkins

[+] Restarting 1/1
 โœ” Container jenkins  Started

1
2
3

docker compose exec jenkins docker ps

permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.51/containers/json": dial unix /var/run/docker.sock: connect: permission denied

5. docker-compose๋กœ ์†Œ์ผ“ GID ๋งคํ•‘ ์ ์šฉ

1

export DOCKER_GID=$(stat -c '%g' /var/run/docker.sock)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

...
services:
  jenkins:
    container_name: jenkins
    image: jenkins/jenkins
    restart: unless-stopped
    networks:
      - cicd-network
    ports:
      - "8080:8080"
      - "50000:50000"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - jenkins_home:/var/jenkins_home
    group_add:
      - "${DOCKER_GID}"
...      

1

docker compose up -d
  • ํ˜ธ์ŠคํŠธ ์†Œ์ผ“ GID๋ฅผ ํ™˜๊ฒฝ๋ณ€์ˆ˜๋กœ ์ถ”์ถœํ•ด group_add์— ๋ฐ˜์˜ํ•˜์—ฌ ๊ถŒํ•œ ๋ฌธ์ œ ํ•ด๊ฒฐํ•จ

6. ์ปจํ…Œ์ด๋„ˆ ์žฌ์ƒ์„ฑ์œผ๋กœ Docker CLI ํœ˜๋ฐœ ๋ฌธ์ œ์™€ ์žฌ์„ค์น˜

1
2

docker compose exec jenkins docker ps
OCI runtime exec failed: exec failed: unable to start container process: exec: "docker": executable file not found in $PATH

1
2
3
4
5
6
7
8
9

docker compose exec --privileged -u root jenkins bash

root@4aec55fcb34f:/# curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
chmod a+r /etc/apt/keyrings/docker.asc
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  tee /etc/apt/sources.list.d/docker.list > /dev/null
apt-get update && apt install docker-ce-cli curl tree jq yq -y

7. ์ตœ์ข… ๋™์ž‘ ๊ฒ€์ฆ

1
2
3
4
5
6
7

docker compose exec jenkins docker ps

CONTAINER ID   IMAGE                  COMMAND                  CREATED          STATUS                    PORTS                                                                                          NAMES
af793fe999f9   jenkins/jenkins        "/usr/bin/tini -- /uโ€ฆ"   6 minutes ago    Up 5 minutes              0.0.0.0:8080->8080/tcp, [::]:8080->8080/tcp, 0.0.0.0:50000->50000/tcp, [::]:50000->50000/tcp   jenkins
f5292520283a   gogs/gogs              "/app/gogs/docker/stโ€ฆ"   27 minutes ago   Up 27 minutes (healthy)   0.0.0.0:3000->3000/tcp, [::]:3000->3000/tcp, 0.0.0.0:10022->22/tcp, [::]:10022->22/tcp         gogs
ea4fa45f8d82   kindest/node:v1.32.8   "/usr/local/bin/entrโ€ฆ"   30 minutes ago   Up 30 minutes             0.0.0.0:30000-30003->30000-30003/tcp, 0.0.0.0:44501->6443/tcp                                  myk8s-control-plane
665b94422e49   kindest/node:v1.32.8   "/usr/local/bin/entrโ€ฆ"   30 minutes ago   Up 30 minutes                                                                                                            myk8s-worker

1
2
3
4
5
6
7
8
9

docker compose exec jenkins cat /etc/group

# ๊ฒฐ๊ณผ
root:x:0:
daemon:x:1:
bin:x:2:
sys:x:3:
...
jenkins:x:1000:
  • jenkins ์‚ฌ์šฉ์ž ์ปจํ…์ŠคํŠธ์—์„œ Docker ๋ช…๋ น ์ •์ƒ ๋™์ž‘ ํ™•์ธํ•จ

โš™๏ธ Gogs ์ปจํ…Œ์ด๋„ˆ ์ดˆ๊ธฐ ์„ค์ •

1. ์ดˆ๊ธฐ ์„ค์ • ์›น ์ ‘์†

**http://<**์ž์‹ ์˜ IP**>:3000/install**

1
2
3
4

๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค ์œ ํ˜•: SQLite3
์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ URL: http://<์ž์‹ ์˜ IP>:3000/
๊ธฐ๋ณธ ๋ธŒ๋žœ์น˜: main
๊ด€๋ฆฌ์ž ๊ณ„์ • ์„ค์ •: ์ด๋ฆ„(๊ณ„์ •๋ช… devops), ๋น„๋ฐ€๋ฒˆํ˜ธ, ์ด๋ฉ”์ผ ์ž…๋ ฅ

2. Token ์ƒ์„ฑ

Your Settings > Applications > Generate New Token - Token Name(devops) ๋ฉ”๋ชจํ•˜๊ธฐ

3. ๊ฐœ๋ฐœํŒ€์šฉ Repository ์ƒ์„ฑ

1
2
3
4
5
6

Repository Name : dev-app
Visibility : (Check) This repository is Private
.gitignore : Python
Readme : Default โ†’ (Check) initialize this repository with selected files and template

โ‡’ Create Repository

4. ๋ฐ๋ธŒ์˜ต์ŠคํŒ€์šฉ Repository ์ƒ์„ฑ

1
2
3
4

Repository Name : ops-deploy
Visibility : (Check) This repository is Private
.gitignore : Python
Readme : Default โ†’ (Check) initialize this repository with selected files and template

๐Ÿš Gogs ์‹ค์Šต์„ ์œ„ํ•œ ์ €์žฅ์†Œ ์„ค์ •

1. Gogs ์ปจํ…Œ์ด๋„ˆ ์…ธ ์ ‘์†

1
2

docker exec -it gogs bash
f5292520283a:/app/gogs# 

1
2
3
4
5
6
7
8

f5292520283a:/app/gogs# TMOUT=0
pwd
ls
cd /data
/app/gogs

# ๊ฒฐ๊ณผ
data    docker  gogs    log

1
2
3
4
5
6
7
8
9

f5292520283a:/data# ls -al

# ๊ฒฐ๊ณผ
total 20
drwxr-xr-x    5 git      git           4096 Nov  1 09:00 .
drwxr-xr-x    1 root     root          4096 Nov  1 09:00 ..
drwxr-xr-x    4 git      git           4096 Nov  1 09:36 git
drwxr-xr-x    5 git      git           4096 Nov  1 09:00 gogs
drwx------    2 git      git           4096 Nov  1 09:00 ssh

2. ํ† ํฐยทIP ํ™˜๊ฒฝ๋ณ€์ˆ˜ ์„ค์ •

1
2

f5292520283a:/data# TOKEN=<๊ฐ์ž Gogs Token>
f5292520283a:/data# MyIP=<๊ฐ์ž ์ž์‹ ์˜ IP>

3. dev-app ์ €์žฅ์†Œ ํด๋ก 

1
2
3
4
5
6
7
8
9

git clone http://devops:$TOKEN@$MyIP:3000/devops/dev-app.git

f5292520283a:/data# git clone http://devops:$TOKEN@$MyIP:3000/devops/dev-app.git
Cloning into 'dev-app'...
remote: Enumerating objects: 4, done.
remote: Counting objects: 100% (4/4), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 4 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
Unpacking objects: 100% (4/4), 690 bytes | 690.00 KiB/s, done.

1
2
3
4
5
6

f5292520283a:/data# cd /data/dev-app
f5292520283a:/data/dev-app# tree
.
โ””โ”€โ”€ README.md

0 directories, 1 files

4. Git ๋กœ์ปฌ ์„ค์ •

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

f5292520283a:/data/dev-app# git config --local user.name "devops"
git config --local user.email "a@a.com"
git config --local init.defaultBranch main
git config --local credential.helper store
git --no-pager config --local --list
cat .git/config

# ๊ฒฐ๊ณผ
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
remote.origin.url=http://devops:6f5bccf278f4e14b32d16f36db047b6440357acd@192.168.219.107:3000/devops/dev-app.git
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.main.remote=origin
branch.main.merge=refs/heads/main
user.name=devops
user.email=a@a.com
init.defaultbranch=main
credential.helper=store
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
[remote "origin"]
	url = http://devops:6f5bccf278f4e14b32d16f36db047b6440357acd@192.168.219.107:3000/devops/dev-app.git
	fetch = +refs/heads/*:refs/remotes/origin/*
[branch "main"]
	remote = origin
	merge = refs/heads/main
[user]
	name = devops
	email = a@a.com
[init]
	defaultBranch = main
[credential]
	helper = store

5. ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ํŒŒ์ผ ์ž‘์„ฑ

(1) server.py ํŒŒ์ผ

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

f5292520283a:/data/dev-app# cat > server.py <<EOF
from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler
from datetime import datetime
import socket

class RequestHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        match self.path:
            case '/':
                now = datetime.now()
                hostname = socket.gethostname()
                response_string = now.strftime("The time is %-I:%M:%S %p, VERSION 0.0.1\n")
                response_string += f"Server hostname: {hostname}\n"                
                self.respond_with(200, response_string)
            case '/healthz':
                self.respond_with(200, "Healthy")
            case _:
                self.respond_with(404, "Not Found")

    def respond_with(self, status_code: int, content: str) -> None:
        self.send_response(status_code)
        self.send_header('Content-type', 'text/plain')
        self.end_headers()
        self.wfile.write(bytes(content, "utf-8")) 

def startServer():
    try:
        server = ThreadingHTTPServer(('', 80), RequestHandler)
        print("Listening on " + ":".join(map(str, server.server_address)))
        server.serve_forever()
    except KeyboardInterrupt:
        server.shutdown()

if __name__== "__main__":
    startServer()
EOF

(2) Dockerfile

1
2
3
4
5
6
7

f5292520283a:/data/dev-app# cat > Dockerfile <<EOF
FROM python:3.12
ENV PYTHONUNBUFFERED 1
COPY . /app
WORKDIR /app 
CMD python3 server.py
EOF

(3) VERSION

1

f5292520283a:/data/dev-app# echo "0.0.1" > VERSION

6. ์ปค๋ฐ‹ ๋ฐ ํ‘ธ์‹œ

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

f5292520283a:/data/dev-app# tree
git status
git add .
git commit -m "Add dev-app"
git push -u origin main

# ๊ฒฐ๊ณผ
.
โ”œโ”€โ”€ Dockerfile
โ”œโ”€โ”€ README.md
โ”œโ”€โ”€ VERSION
โ””โ”€โ”€ server.py

0 directories, 4 files
On branch main
Your branch is up to date with 'origin/main'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	Dockerfile
	VERSION
	server.py

nothing added to commit but untracked files present (use "git add" to track)
[main 5e0b56b] Add dev-app
 3 files changed, 40 insertions(+)
 create mode 100644 Dockerfile
 create mode 100644 VERSION
 create mode 100644 server.py
Enumerating objects: 6, done.
Counting objects: 100% (6/6), done.
Delta compression using up to 18 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (5/5), 1014 bytes | 1014.00 KiB/s, done.
Total 5 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
To http://192.168.219.107:3000/devops/dev-app.git
   bcd7702..5e0b56b  main -> main
branch 'main' set up to track 'origin/main'.

๐Ÿณ ๋„์ปค ํ—ˆ๋ธŒ

1. ๋„์ปค ํ—ˆ๋ธŒ ํ† ํฐ ๋ฐœ๊ธ‰

2. Private ๋ฆฌํฌ์ง€ํ† ๋ฆฌ ์ƒ์„ฑ(dev-app)

๐Ÿค– Jenkins CI Pipeline

1. Jenkins ํ”Œ๋Ÿฌ๊ทธ์ธ ์„ค์น˜

  • Pipeline Stage View - Docs
  • Docker Pipeline : building, testing, and using Docker images from Jenkins Pipeline - Docs
  • Gogs : Webhook Plugin - Docs

2. ์ž๊ฒฉ์ฆ๋ช… ์„ค์ •

Jenkins ๊ด€๋ฆฌ โ†’ Credentials โ†’ Globals โ†’ Add Credentials

1
2
3
4
5

(1) Gogs Repo ์ž๊ฒฉ์ฆ๋ช… ์„ค์ • : gogs-crd
Kind : Username with password
Username : devops
Password : <Gogs ํ† ํฐ>
ID : gogs-crd


1
2
3
4
5

(2) ๋„์ปค ํ—ˆ๋ธŒ ์ž๊ฒฉ์ฆ๋ช… ์„ค์ • : dockerhub-crd
Kind : Username with password
Username : <๋„์ปค ๊ณ„์ •๋ช…>
Password : <๋„์ปค ๊ณ„์ • ์•”ํ˜ธ ํ˜น์€ ํ† ํฐ>
ID : dockerhub-crd

3. Jenkins Item ์ƒ์„ฑ

(1) item ์ง€์ •

(2) Pipeline script

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

pipeline {
    agent any
    environment {
        DOCKER_IMAGE = '<์ž์‹ ์˜ ๋„์ปค ํ—ˆ๋ธŒ ๊ณ„์ •>/dev-app' // Docker ์ด๋ฏธ์ง€ ์ด๋ฆ„
    }
    stages {
        stage('Checkout') {
            steps {
                 git branch: 'main',
                 url: 'http://<์ž์‹ ์˜ IP>:3000/devops/dev-app.git',  // Git์—์„œ ์ฝ”๋“œ ์ฒดํฌ์•„์›ƒ
                 credentialsId: 'gogs-crd'  // Credentials ID
            }
        }
        stage('Read VERSION') {
            steps {
                script {
                    // VERSION ํŒŒ์ผ ์ฝ๊ธฐ
                    def version = readFile('VERSION').trim()
                    echo "Version found: ${version}"
                    // ํ™˜๊ฒฝ ๋ณ€์ˆ˜ ์„ค์ •
                    env.DOCKER_TAG = version
                }
            }
        }
        stage('Docker Build and Push') {
            steps {
                script {
                    docker.withRegistry('https://index.docker.io/v1/', 'dockerhub-crd') {
                        // DOCKER_TAG ์‚ฌ์šฉ
                        def appImage = docker.build("${DOCKER_IMAGE}:${DOCKER_TAG}")
                        appImage.push()
                        appImage.push("latest")
                    }
                }
            }
        }
    }
    post {
        success {
            echo "Docker image ${DOCKER_IMAGE}:${DOCKER_TAG} has been built and pushed successfully!"
        }
        failure {
            echo "Pipeline failed. Please check the logs."
        }
    }
}

(3) ๋นŒ๋“œ ์ง„ํ–‰

๐Ÿงญ Deploying to Kubernetes

1. Deployment ์ƒ์„ฑ

1

DHUSER=<๋„์ปค ํ—ˆ๋ธŒ ๊ณ„์ •๋ช…>

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: timeserver
spec:
  replicas: 2
  selector:
    matchLabels:
      pod: timeserver-pod
  template:
    metadata:
      labels:
        pod: timeserver-pod
    spec:
      containers:
      - name: timeserver-container
        image: docker.io/$DHUSER/dev-app:0.0.1
        livenessProbe:
          initialDelaySeconds: 30
          periodSeconds: 30
          httpGet:
            path: /healthz
            port: 80
            scheme: HTTP
          timeoutSeconds: 5
          failureThreshold: 3
          successThreshold: 1
EOF
deployment.apps/timeserver created

2. ImagePullBackOff ์›์ธ ์ง„๋‹จ

1
2
3
4
5
6
7
8
9
10
11
12

kubectl get deploy,rs,pod -o wide

# ๊ฒฐ๊ณผ
NAME                         READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS             IMAGES                               SELECTOR
deployment.apps/timeserver   0/2     2            0           81s   timeserver-container   docker.io/shinminjin/dev-app:0.0.1   pod=timeserver-pod

NAME                                    DESIRED   CURRENT   READY   AGE   CONTAINERS             IMAGES                               SELECTOR
replicaset.apps/timeserver-6cd9654cc7   2         2         0       81s   timeserver-container   docker.io/shinminjin/dev-app:0.0.1   pod=timeserver-pod,pod-template-hash=6cd9654cc7

NAME                              READY   STATUS             RESTARTS   AGE   IP           NODE           NOMINATED NODE   READINESS GATES
pod/timeserver-6cd9654cc7-8jtdb   0/1     ImagePullBackOff   0          81s   10.244.1.2   myk8s-worker   <none>           <none>
pod/timeserver-6cd9654cc7-hkfw5   0/1     ImagePullBackOff   0          81s   10.244.1.3   myk8s-worker   <none>           <none>

1
2
3
4
5
6
7
8
9
10
11
12
13

kubectl describe pod

# ๊ฒฐ๊ณผ
...
Events:
  Type     Reason     Age                 From               Message
  ----     ------     ----                ----               -------
  Normal   Scheduled  104s                default-scheduler  Successfully assigned default/timeserver-6cd9654cc7-hkfw5 to myk8s-worker
  Normal   BackOff    22s (x5 over 102s)  kubelet            Back-off pulling image "docker.io/shinminjin/dev-app:0.0.1"
  Warning  Failed     22s (x5 over 102s)  kubelet            Error: ImagePullBackOff
  Normal   Pulling    10s (x4 over 104s)  kubelet            Pulling image "docker.io/shinminjin/dev-app:0.0.1"
  Warning  Failed     9s (x4 over 102s)   kubelet            Failed to pull image "docker.io/shinminjin/dev-app:0.0.1": failed to pull and unpack image "docker.io/shinminjin/dev-app:0.0.1": failed to resolve reference "docker.io/shinminjin/dev-app:0.0.1": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
  Warning  Failed     9s (x4 over 102s)   kubelet            Error: ErrImagePull
  • ๋ณดํ†ต ์ปจํ…Œ์ด๋„ˆ ์ด๋ฏธ์ง€ ์ •๋ณด๋ฅผ ์ž˜๋ชป ๊ธฐ์ž…ํ•˜๋Š” ๊ฒฝ์šฐ์— ๋ฐœ์ƒ
  • ํ˜น์€ ์ด๋ฏธ์ง€ ์ €์žฅ์†Œ์— ์ด๋ฏธ์ง€๊ฐ€ ์—†๊ฑฐ๋‚˜, ์ด๋ฏธ์ง€ ๊ฐ€์ ธ์˜ค๋Š” ์ž๊ฒฉ ์ฆ๋ช…์ด ์—†๋Š” ๊ฒฝ์šฐ์— ๋ฐœ์ƒ

3. Docker Hub ์ž๊ฒฉ์ฆ๋ช… ์‹œํฌ๋ฆฟ ์ƒ์„ฑ

1
2

DHUSER=<๋„์ปค ํ—ˆ๋ธŒ ๊ณ„์ •>
DHPASS=<๋„์ปค ํ—ˆ๋ธŒ ์•”ํ˜ธ ํ˜น์€ ํ† ํฐ>

1
2
3
4
5
6
7

kubectl create secret docker-registry dockerhub-secret \
  --docker-server=https://index.docker.io/v1/ \
  --docker-username=$DHUSER \
  --docker-password=$DHPASS
  
# ๊ฒฐ๊ณผ
secret/dockerhub-secret created

4. Deployment์— imagePullSecrets ์ ์šฉ

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: timeserver
spec:
  replicas: 2
  selector:
    matchLabels:
      pod: timeserver-pod
  template:
    metadata:
      labels:
        pod: timeserver-pod
    spec:
      containers:
      - name: timeserver-container
        image: docker.io/$DHUSER/dev-app:0.0.1
        livenessProbe:
          initialDelaySeconds: 30
          periodSeconds: 30
          httpGet:
            path: /healthz
            port: 80
            scheme: HTTP
          timeoutSeconds: 5
          failureThreshold: 3
          successThreshold: 1
      imagePullSecrets:
      - name: dockerhub-secret
EOF

# ๊ฒฐ๊ณผ
deployment.apps/timeserver configured

5. ๋ฆฌ์†Œ์Šค ์ƒํƒœ ํ™•์ธ

1
2
3
4
5
6
7
8
9
10
11
12
13

kubectl get deploy,rs,pod -o wide

# ๊ฒฐ๊ณผ
NAME                         READY   UP-TO-DATE   AVAILABLE   AGE    CONTAINERS             IMAGES                               SELECTOR
deployment.apps/timeserver   2/2     2            2           7m5s   timeserver-container   docker.io/shinminjin/dev-app:0.0.1   pod=timeserver-pod

NAME                                    DESIRED   CURRENT   READY   AGE    CONTAINERS             IMAGES                               SELECTOR
replicaset.apps/timeserver-69c87f9f8c   2         2         2       60s    timeserver-container   docker.io/shinminjin/dev-app:0.0.1   pod=timeserver-pod,pod-template-hash=69c87f9f8c
replicaset.apps/timeserver-6cd9654cc7   0         0         0       7m5s   timeserver-container   docker.io/shinminjin/dev-app:0.0.1   pod=timeserver-pod,pod-template-hash=6cd9654cc7

NAME                              READY   STATUS    RESTARTS   AGE   IP           NODE           NOMINATED NODE   READINESS GATES
pod/timeserver-69c87f9f8c-jr2j6   1/1     Running   0          60s   10.244.1.4   myk8s-worker   <none>           <none>
pod/timeserver-69c87f9f8c-tjlp2   1/1     Running   0          22s   10.244.1.5   myk8s-worker   <none>           <none>

6. Service ๊ณต๊ฐœ

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  name: timeserver
spec:
  selector:
    pod: timeserver-pod
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
    nodePort: 30000
  type: NodePort
EOF

# ๊ฒฐ๊ณผ
service/timeserver created

1
2
3
4
5

curl http://127.0.0.1:30000

# ๊ฒฐ๊ณผ
The time is 1:07:41 PM, VERSION 0.0.1
Server hostname: timeserver-69c87f9f8c-jr2j6
  • ํ˜ธ์ถœ ์‹œ ์‹œ๊ฐ„/ํ˜ธ์ŠคํŠธ๋ช… ์‘๋‹ต ํ™•์ธ

7. ๋ฐ˜๋ณต ์ ‘์†

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

while true; do curl -s --connect-timeout 1 http://127.0.0.1:30000 ; sleep 1 ; done

# ๊ฒฐ๊ณผ
The time is 1:08:23 PM, VERSION 0.0.1
Server hostname: timeserver-69c87f9f8c-jr2j6
The time is 1:08:24 PM, VERSION 0.0.1
Server hostname: timeserver-69c87f9f8c-tjlp2
The time is 1:08:25 PM, VERSION 0.0.1
Server hostname: timeserver-69c87f9f8c-tjlp2
The time is 1:08:26 PM, VERSION 0.0.1
Server hostname: timeserver-69c87f9f8c-tjlp2
The time is 1:08:27 PM, VERSION 0.0.1
Server hostname: timeserver-69c87f9f8c-tjlp2
The time is 1:08:28 PM, VERSION 0.0.1
...

๐Ÿ–ฅ๏ธ Updating your application : k8s Deploying an application with Jenkins

1. Git ๋ฐ˜์˜

์ƒ˜ํ”Œ ์•ฑ server.py, VERSION ์ฝ”๋“œ ๋ณ€๊ฒฝ(0.0.2)

1
2
3
4
5
6
7
8
9
10
11
12
13
14

f5292520283a:/data/dev-app# git add . && git commit -m "VERSION $(cat VERSION) Changed" && git push -u origin main

# ๊ฒฐ๊ณผ
[main cc68f85] VERSION 0.0.2 Changed
 2 files changed, 2 insertions(+), 2 deletions(-)
Enumerating objects: 7, done.
Counting objects: 100% (7/7), done.
Delta compression using up to 18 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (4/4), 340 bytes | 340.00 KiB/s, done.
Total 4 (delta 2), reused 0 (delta 0), pack-reused 0 (from 0)
To http://192.168.219.107:3000/devops/dev-app.git
   5e0b56b..cc68f85  main -> main
branch 'main' set up to track 'origin/main'.

2. VERSION 0.0.1 ํ™•์ธ

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

The time is 1:16:26 PM, VERSION 0.0.1
Server hostname: timeserver-69c87f9f8c-tjlp2
The time is 1:16:27 PM, VERSION 0.0.1
Server hostname: timeserver-69c87f9f8c-jr2j6
The time is 1:16:28 PM, VERSION 0.0.1
Server hostname: timeserver-69c87f9f8c-jr2j6
The time is 1:16:29 PM, VERSION 0.0.1
Server hostname: timeserver-69c87f9f8c-tjlp2
The time is 1:16:30 PM, VERSION 0.0.1
Server hostname: timeserver-69c87f9f8c-tjlp2
The time is 1:16:31 PM, VERSION 0.0.1
Server hostname: timeserver-69c87f9f8c-tjlp2
The time is 1:16:32 PM, VERSION 0.0.1
Server hostname: timeserver-69c87f9f8c-tjlp2
The time is 1:16:33 PM, VERSION 0.0.1
Server hostname: timeserver-69c87f9f8c-jr2j6
The time is 1:16:34 PM, VERSION 0.0.1
Server hostname: timeserver-69c87f9f8c-jr2j6
...

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

kubectl get deploy,rs,pod,svc,ep -owide

# ๊ฒฐ๊ณผ
NAME                         READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS             IMAGES                               SELECTOR
deployment.apps/timeserver   2/2     2            2           18m   timeserver-container   docker.io/shinminjin/dev-app:0.0.1   pod=timeserver-pod

NAME                                    DESIRED   CURRENT   READY   AGE   CONTAINERS             IMAGES                               SELECTOR
replicaset.apps/timeserver-69c87f9f8c   2         2         2       12m   timeserver-container   docker.io/shinminjin/dev-app:0.0.1   pod=timeserver-pod,pod-template-hash=69c87f9f8c
replicaset.apps/timeserver-6cd9654cc7   0         0         0       18m   timeserver-container   docker.io/shinminjin/dev-app:0.0.1   pod=timeserver-pod,pod-template-hash=6cd9654cc7

NAME                              READY   STATUS    RESTARTS   AGE   IP           NODE           NOMINATED NODE   READINESS GATES
pod/timeserver-69c87f9f8c-jr2j6   1/1     Running   0          12m   10.244.1.4   myk8s-worker   <none>           <none>
pod/timeserver-69c87f9f8c-tjlp2   1/1     Running   0          11m   10.244.1.5   myk8s-worker   <none>           <none>

NAME                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE     SELECTOR
service/kubernetes   ClusterIP   10.96.0.1      <none>        443/TCP        4h19m   <none>
service/timeserver   NodePort    10.96.88.179   <none>        80:30000/TCP   10m     pod=timeserver-pod

NAME                   ENDPOINTS                     AGE
endpoints/kubernetes   172.18.0.2:6443               4h19m
endpoints/timeserver   10.244.1.4:80,10.244.1.5:80   10m
  • CI ๋นŒ๋“œยทํ‘ธ์‹œ๋Š” ์™„๋ฃŒ๋˜์—ˆ์œผ๋‚˜ K8s ์ชฝ ์ด๋ฏธ์ง€ ํƒœ๊ทธ ์—…๋ฐ์ดํŠธ๊ฐ€ ์ž๋™์œผ๋กœ ๋ฐ˜์˜๋˜์ง€ ์•Š์Œ

3. ์ˆ˜๋™ ๋กค๋ง ์—…๋ฐ์ดํŠธ ์‹คํ–‰

1

kubectl set image deployment timeserver timeserver-container=$DHUSER/dev-app:0.0.2 && watch -d "kubectl get deploy,ep timeserver -owide; echo; kubectl get rs,pod"


1
2
3
4
5
6
7
8
9
10
11
12
13

Server hostname: timeserver-69c87f9f8c-jr2j6
The time is 1:18:15 PM, VERSION 0.0.1
Server hostname: timeserver-69c87f9f8c-tjlp2
The time is 1:18:16 PM, VERSION 0.0.1
Server hostname: timeserver-69c87f9f8c-tjlp2
The time is 1:18:17 PM, VERSION 0.0.2
Server hostname: timeserver-79d564b95c-28lmt
The time is 1:18:18 PM, VERSION 0.0.2
Server hostname: timeserver-79d564b95c-28lmt
The time is 1:18:19 PM, VERSION 0.0.2
Server hostname: timeserver-79d564b95c-28lmt
The time is 1:18:20 PM, VERSION 0.0.2
...
  • ์ด๋ฏธ์ง€ ํƒœ๊ทธ 0.0.2๋กœ ๊ฐฑ์‹ ํ•จ

๐Ÿ” Gogs Webhooks ์„ค์ •

1. Gogs ๋ณด์•ˆ ์„ค์ •(app.ini) ์—…๋ฐ์ดํŠธ

1
2

f5292520283a:/data/dev-app# cd ../gogs/conf/
f5292520283a:/data/gogs/conf# vi app.ini

1
2
3
4

[security]      
INSTALL_LOCK = true            
SECRET_KEY   = QYNoxvpdPUjpRf1
LOCAL_NETWORK_ALLOWLIST = 192.168.254.110 # ๊ฐ์ž ์ž์‹ ์˜ IP

1
2
3
4
5
6
7
8
9
10

f5292520283a:/data/gogs/conf# exit
exit

# gogs ์žฌ์‹œ์ž‘
docker compose restart gogs

# ๊ฒฐ๊ณผ
WARN[0000] The "DOCKER_GID" variable is not set. Defaulting to a blank string. 
[+] Restarting 1/1
 โœ” Container gogs  Started

2. Gogs Webhook โ†’ Jenkins ํŠธ๋ฆฌ๊ฑฐ ์„ค์ •

1
2
3
4
5

Payload URL : http://192.168.219.107:8080/gogs-webhook/?job=SCM-Pipeline/  # ๊ฐ์ž ์ž์‹ ์˜ IP
Content Type : application/json
Secret : qwe123
When should this webhook be triggered? : Just the push event
Active : Check

3. Jenkins Item ์ƒ์„ฑ(Pipeline) : SCM-Pipeline

1
2
3
4
5
6
7
8

GitHub project : http://<์ž์‹ ์˜ IP>:3000/<Gogs ๊ณ„์ •๋ช…>/dev-app
Use Gogs secret : qwe123
Pipeline script from SCM
- SCM : Git
	- Repo URL(http://<์ž์‹ ์˜ IP>:3000/<Gogs ๊ณ„์ •๋ช…>/dev-app)
	- Credentials(devops/***)
	- Branch(*/main)
- Script Path : Jenkinsfile

4. Gogs ์ปจํ…Œ์ด๋„ˆ ์žฌ์ ‘์† ๋ฐ ๋ฆฌํฌ์ง€ํ† ๋ฆฌ ๊ฒฝ๋กœ ์ด๋™

1
2
3

docker exec -it gogs bash
f5292520283a:/app/gogs# cd ../../data/dev-app/
f5292520283a:/data/dev-app#

5. Jenkinsfile ์ƒ์„ฑ ๋ฐ ํ‘ธ์‹œ

1
2
3
4
5
6
7
8

f5292520283a:/data/dev-app# tree
.
โ”œโ”€โ”€ Dockerfile
โ”œโ”€โ”€ README.md
โ”œโ”€โ”€ VERSION
โ””โ”€โ”€ server.py

0 directories, 4 files

1

f5292520283a:/data/dev-app# touch Jenkinsfile

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

pipeline {
    agent any
    environment {
        DOCKER_IMAGE = '<์ž์‹ ์˜ ๋„์ปค ํ—ˆ๋ธŒ ๊ณ„์ •>/dev-app' // Docker ์ด๋ฏธ์ง€ ์ด๋ฆ„
    }
    stages {
        stage('Checkout') {
            steps {
                 git branch: 'main',
                 url: 'http://<์ž์‹ ์˜ IP>:3000/devops/dev-app.git',  // Git์—์„œ ์ฝ”๋“œ ์ฒดํฌ์•„์›ƒ
                 credentialsId: 'gogs-crd'  // Credentials ID
            }
        }
        stage('Read VERSION') {
            steps {
                script {
                    // VERSION ํŒŒ์ผ ์ฝ๊ธฐ
                    def version = readFile('VERSION').trim()
                    echo "Version found: ${version}"
                    // ํ™˜๊ฒฝ ๋ณ€์ˆ˜ ์„ค์ •
                    env.DOCKER_TAG = version
                }
            }
        }
        stage('Docker Build and Push') {
            steps {
                script {
                    docker.withRegistry('https://index.docker.io/v1/', 'dockerhub-crd') {
                        // DOCKER_TAG ์‚ฌ์šฉ
                        def appImage = docker.build("${DOCKER_IMAGE}:${DOCKER_TAG}")
                        appImage.push()
                        appImage.push("latest")
                    }
                }
            }
        }
    }
    post {
        success {
            echo "Docker image ${DOCKER_IMAGE}:${DOCKER_TAG} has been built and pushed successfully!"
        }
        failure {
            echo "Pipeline failed. Please check the logs."
        }
    }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

f5292520283a:/data/dev-app# git add . && git commit -m "VERSION $(cat VERSION) Changed" && git push -u origin main

# ๊ฒฐ๊ณผ
[main 8d36c4b] VERSION 0.0.3 Changed
 3 files changed, 48 insertions(+), 2 deletions(-)
 create mode 100644 Jenkinsfile
Enumerating objects: 8, done.
Counting objects: 100% (8/8), done.
Delta compression using up to 18 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (5/5), 1.06 KiB | 1.06 MiB/s, done.
Total 5 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)
To http://192.168.219.107:3000/devops/dev-app.git
   cc68f85..8d36c4b  main -> main
branch 'main' set up to track 'origin/main'.

6. k8s์— ์‹ ๊ทœ ๋ฒ„์ „ ์ ์šฉ

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

kubectl set image deployment timeserver timeserver-container=$DHUSER/dev-app:0.0.3 && while true; do curl -s --connect-timeout 1 http://127.0.0.1:30000 ; sleep 1 ; done

# ๊ฒฐ๊ณผ
deployment.apps/timeserver image updated
Server hostname: timeserver-79d564b95c-28lmt
The time is 1:56:48 PM, VERSION 0.0.2
Server hostname: timeserver-79d564b95c-vpb6d
The time is 1:56:49 PM, VERSION 0.0.2
Server hostname: timeserver-79d564b95c-28lmt
The time is 1:56:50 PM, VERSION 0.0.2
Server hostname: timeserver-79d564b95c-vpb6d
The time is 1:56:51 PM, VERSION 0.0.2
Server hostname: timeserver-79d564b95c-vpb6d
The time is 1:56:52 PM, VERSION 0.0.3
Server hostname: timeserver-6566694f9d-vz2zr
The time is 1:56:53 PM, VERSION 0.0.3
Server hostname: timeserver-6566694f9d-vz2zr
The time is 1:56:54 PM, VERSION 0.0.3
Server hostname: timeserver-6566694f9d-vz2zr
The time is 1:56:55 PM, VERSION 0.0.3
...

๐ŸŒ Jenkins CD by K8S(Kind)

1. Jenkins ์ปจํ…Œ์ด๋„ˆ์— kubectl/helm ์„ค์น˜

1
2
3
4
5
6
7
8

docker compose exec --privileged -u root jenkins bash
root@af793fe999f9:/# curl -LO "https://dl.k8s.io/release/v1.32.8/bin/linux/amd64/kubectl"

# ๊ฒฐ๊ณผ
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   138  100   138    0     0    252      0 --:--:-- --:--:-- --:--:--   251
100 54.6M  100 54.6M    0     0  15.8M      0  0:00:03  0:00:03 --:--:-- 22.7M

1
2
3
4
5
6

root@af793fe999f9:/# install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
kubectl version --client=true

# ๊ฒฐ๊ณผ
Client Version: v1.32.8
Kustomize Version: v5.5.0

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

root@af793fe999f9:/# curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
helm version

# ๊ฒฐ๊ณผ
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 11928  100 11928    0     0  27938      0 --:--:-- --:--:-- --:--:-- 27934
Downloading https://get.helm.sh/helm-v3.19.0-linux-amd64.tar.gz
Verifying checksum... Done.
Preparing to install helm into /usr/local/bin
helm installed into /usr/local/bin/helm
version.BuildInfo{Version:"v3.19.0", GitCommit:"3d8990f0836691f0229297773f3524598f46bda6", GitTreeState:"clean", GoVersion:"go1.24.7"}

root@af793fe999f9:/# exit
exit

2. ํด๋ผ์ด์–ธํŠธ ๋ฒ„์ „ ํ™•์ธ

1
2
3
4
5

docker compose exec jenkins kubectl version --client=true

# ๊ฒฐ๊ณผ
Client Version: v1.32.8
Kustomize Version: v5.5.0

1
2
3
4

docker compose exec jenkins helm version

# ๊ฒฐ๊ณผ
version.BuildInfo{Version:"v3.19.0", GitCommit:"3d8990f0836691f0229297773f3524598f46bda6", GitTreeState:"clean", GoVersion:"go1.24.7"}

3. Kind ์ปจํŠธ๋กคํ”Œ๋ ˆ์ธ IP ํ™•์ธ

1
2
3
4
5
6

docker inspect myk8s-control-plane | grep IPAddress

# ๊ฒฐ๊ณผ
            "SecondaryIPAddresses": null,
            "IPAddress": "",
                    "IPAddress": "172.18.0.2",

4. Jenkins โ†’ K8s API ์—ฐ๊ฒฐ ํ™•์ธ

1
2
3
4
5
6
7
8
9
10
11
12
13
14

docker exec -it jenkins curl https://172.18.0.2:6443/version -k

# ๊ฒฐ๊ณผ
{
  "major": "1",
  "minor": "32",
  "gitVersion": "v1.32.8",
  "gitCommit": "2e83bc4bf31e88b7de81d5341939d5ce2460f46f",
  "gitTreeState": "clean",
  "buildDate": "2025-08-13T14:21:22Z",
  "goVersion": "go1.23.11",
  "compiler": "gc",
  "platform": "linux/amd64"
}%

5. Kubeconfig ์ค€๋น„ ๋ฐ ์ˆ˜์ •

1

cp ~/.kube/config ./kube-config

1
2

        server: https://0.0.0.0:44501
      name: kind-myk8s

1
2

				server: https://<myk8s-control-plane ์ปจํ…Œ์ด๋„ˆ IP>:6443 # ์ž์‹ ์˜ ํ™˜๊ฒฝ์— ๋งž๊ฒŒ ๋ณ€๊ฒฝ
		  name: kind-myk8s

6. Jenkins ์ž๊ฒฉ์ฆ๋ช…(k8s-crd) ์ƒ์„ฑ

7. ํŒŒ์ดํ”„๋ผ์ธ ์•„์ดํ…œ ์ƒ์„ฑ(k8s-cmd)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

pipeline {
    agent any
    environment {
        KUBECONFIG = credentials('k8s-crd')
    }
    stages {
        stage('List Pods') {
            steps {
                sh '''
                # Fetch and display Pods
                kubectl get pods -A --kubeconfig "$KUBECONFIG"
                '''
            }
        }
    }
}

๐Ÿ” Jenkins๋ฅผ ์ด์šฉํ•œ blue-green ๋ฐฐํฌ

1. ๊ธฐ์กด ๋ฆฌ์†Œ์Šค ์ •๋ฆฌ

1
2
3
4
5

kubectl delete deploy,svc timeserver

# ๊ฒฐ๊ณผ
deployment.apps "timeserver" deleted from default namespace
service "timeserver" deleted from default namespace

2. ๋ฐฐํฌ ๋งค๋‹ˆํŽ˜์ŠคํŠธ ๋””๋ ‰ํ„ฐ๋ฆฌ ์ƒ์„ฑ

1

f5292520283a:/data/dev-app# mkdir deploy

3. Blue/Green/Service ๋งค๋‹ˆํŽ˜์ŠคํŠธ ์ž‘์„ฑ

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68

f5292520283a:/data/dev-app# cat > deploy/echo-server-blue.yaml <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo-server-blue
spec:
  replicas: 2
  selector:
    matchLabels:
      app: echo-server
      version: blue
  template:
    metadata:
      labels:
        app: echo-server
        version: blue
    spec:
      containers:
      - name: echo-server
        image: hashicorp/http-echo
        args:
        - "-text=Hello from Blue"
        ports:
        - containerPort: 5678
EOF

cat > deploy/echo-server-service.yaml <<EOF
apiVersion: v1
kind: Service
metadata:
  name: echo-server-service
spec:
  selector:
    app: echo-server
    version: blue
  ports:
  - protocol: TCP
    port: 80
    targetPort: 5678
    nodePort: 30000
  type: NodePort
EOF

cat > deploy/echo-server-green.yaml <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo-server-green
spec:
  replicas: 2
  selector:
    matchLabels:
      app: echo-server
      version: green
  template:
    metadata:
      labels:
        app: echo-server
        version: green
    spec:
      containers:
      - name: echo-server
        image: hashicorp/http-echo
        args:
        - "-text=Hello from Green"
        ports:
        - containerPort: 5678
EOF

1
2
3
4
5
6
7
8
9
10
11
12
13

f5292520283a:/data/dev-app# tree
.
โ”œโ”€โ”€ Dockerfile
โ”œโ”€โ”€ Jenkinsfile
โ”œโ”€โ”€ README.md
โ”œโ”€โ”€ VERSION
โ”œโ”€โ”€ deploy
โ”‚   โ”œโ”€โ”€ echo-server-blue.yaml
โ”‚   โ”œโ”€โ”€ echo-server-green.yaml
โ”‚   โ””โ”€โ”€ echo-server-service.yaml
โ””โ”€โ”€ server.py

1 directories, 8 files

4. Git ์ปค๋ฐ‹/ํ‘ธ์‹œ๋กœ ๋งค๋‹ˆํŽ˜์ŠคํŠธ ๋ฐ˜์˜

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

f5292520283a:/data/dev-app# git add . && git commit -m "Add echo server yaml" && git push -u origin main

# ๊ฒฐ๊ณผ
[main b7b758f] Add echo server yaml
 3 files changed, 60 insertions(+)
 create mode 100644 deploy/echo-server-blue.yaml
 create mode 100644 deploy/echo-server-green.yaml
 create mode 100644 deploy/echo-server-service.yaml
Enumerating objects: 7, done.
Counting objects: 100% (7/7), done.
Delta compression using up to 18 threads
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 789 bytes | 789.00 KiB/s, done.
Total 6 (delta 2), reused 0 (delta 0), pack-reused 0 (from 0)
To http://192.168.219.107:3000/devops/dev-app.git
   273223f..b7b758f  main -> main
branch 'main' set up to track 'origin/main'.

5. Blue-Green ๊ด€์ฐฐ์šฉ ๋ฐ˜๋ณต ์Šคํฌ๋ฆฝํŠธ ์‹คํ–‰

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

while true; do curl -s --connect-timeout 1 http://127.0.0.1:30000 ; echo ; sleep 1  ; kubectl get deploy -owide ; echo ; kubectl get svc,ep echo-server-service -owide ; echo "------------" ; done

# ๊ฒฐ๊ณผ
No resources found in default namespace.

Error from server (NotFound): services "echo-server-service" not found
Error from server (NotFound): endpoints "echo-server-service" not found
------------

No resources found in default namespace.

Error from server (NotFound): services "echo-server-service" not found
Error from server (NotFound): endpoints "echo-server-service" not found
------------

No resources found in default namespace.

Error from server (NotFound): services "echo-server-service" not found
Error from server (NotFound): endpoints "echo-server-service" not found
------------
...

6. Jenkins ํŒŒ์ดํ”„๋ผ์ธ ์ƒ์„ฑ(k8s-bluegreen)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73

pipeline {
    agent any

    environment {
        KUBECONFIG = credentials('k8s-crd')
    }

    stages {
        stage('Checkout') {
            steps {
                 git branch: 'main',
                 url: 'http://<์ž์‹ ์˜ IP>:3000/devops/dev-app.git',  // Git์—์„œ ์ฝ”๋“œ ์ฒดํฌ์•„์›ƒ
                 credentialsId: 'gogs-crd'  // Credentials ID
            }
        }

        stage('container image build') {
            steps {
                echo "container image build"
            }
        }

        stage('container image upload') {
            steps {
                echo "container image upload"
            }
        }

        stage('k8s deployment blue version') {
            steps {
                sh "kubectl apply -f ./deploy/echo-server-blue.yaml --kubeconfig $KUBECONFIG"
                sh "kubectl apply -f ./deploy/echo-server-service.yaml --kubeconfig $KUBECONFIG"
            }
        }

        stage('approve green version') {
            steps {
                input message: 'approve green version', ok: "Yes"
            }
        }

        stage('k8s deployment green version') {
            steps {
	        	sh "kubectl apply -f ./deploy/echo-server-green.yaml --kubeconfig $KUBECONFIG"
            }
        }

        stage('approve version switching') {
            steps {
                script {
                    returnValue = input message: 'Green switching?', ok: "Yes", parameters: [booleanParam(defaultValue: true, name: 'IS_SWITCHED')]
                    if (returnValue) {
                        sh "kubectl patch svc echo-server-service -p '{\"spec\": {\"selector\": {\"version\": \"green\"}}}' --kubeconfig $KUBECONFIG"
                    }
                }
            }
        }

        stage('Blue Rollback') {
            steps {
                script {
                    returnValue = input message: 'Blue Rollback?', parameters: [choice(choices: ['done', 'rollback'], name: 'IS_ROLLBACk')]
                    if (returnValue == "done") {
                        sh "kubectl delete -f ./deploy/echo-server-blue.yaml --kubeconfig $KUBECONFIG"
                    }
                    if (returnValue == "rollback") {
                        sh "kubectl patch svc echo-server-service -p '{\"spec\": {\"selector\": {\"version\": \"blue\"}}}' --kubeconfig $KUBECONFIG"
                    }
                }
            }
        }
    }
}

7. ์ˆ˜๋™ ์Šน์ธ ๋‹จ๊ณ„ ์•ˆ๋‚ด

(1) ๊ทธ๋ฆฐ ํ™˜๊ฒฝ ๋ฐฐํฌ ์ง„ํ–‰ ์—ฌ๋ถ€

(2) ์„œ๋น„์Šค ์ „ํ™˜(ํŠธ๋ž˜ํ”ฝ ๊ทธ๋ฆฐ) ์—ฌ๋ถ€

(3) ๋ธ”๋ฃจ ์‚ญ์ œ ๋˜๋Š” ๋กค๋ฐฑ(์„œ๋น„์Šค blue๋กœ ์žฌ์ „ํ™˜) ์„ ํƒ

CI/CD
CI/CD
This post is licensed under CC BY 4.0 by the author.