环境

Ubuntu 22.04 LTS
Docker 27.2.1
Jenkins 2.462.2

描述

本文未讲如何构建 Gitlab,Harbor,Docker Swarm,Jenkins 等应用。
本文讲述 Jenkins 对接 Gitlab Harbor 使用 Pipeline Jenkinsfile 持续构建部署 uwsgi + django + sqlite3 轻小型 CMS 中后台应用至 Docker Swarm。
完成如下配置后,将实现当 Gitlab 中创建新标签时,将会自动进行,代码推送,镜像构建,镜像推送私有库,通知运行主机拉取镜像并发布。

步骤

目录结构

以下是一个完整 Django Project 目录。
空目录使用 .gitkeep(空文件) 占位使得 Gitlab 中保持目录结构。

项目
—App目录
—data # 存放数据库 sqlite3
——.gitkeep
—项目目录 # 含有 Django setting.py 的目录
——settings.py
—docker # Docker 相关文件
——Dockerfile-uwsgi
——Entrypoint-uwsgi.sh
——stack-compose.yml
—script # 脚本目录
——deploy-swarm.sh
—static # 静态文件目录
——.gitkeep
.gitignore
Jenkinsfile-swarm
manage.py
requirements.txt
uwsgi.ini

图示

点我查看文件内容

Django01

配置 Jenkins

安装插件

Pipeline Stage View Plugin
GitLab Plugin
Publish Over SSH
Pipeline
Credentials Plugin

创建并配置 Pipeline

New Item -> Pipeline -> 设置Jenkins 项目名称,如:django01

django01 -> Configure
Build Triggers:勾选Build when a change is pushed to GitLab. 并记录 Webhook 值,勾选子选项 Push Events
Pipeline:Pipeline script from SCM

  1. Repository URL:填写项目 URL,如:http://192.168.120.53:18001/t01/demo03.git
  2. Script Path:Jenkinsfile-swarm

创建凭证

Manage Jenkins -> Credentials -> Domain 下点击 global -> Add Credentials -> 添加 Harbor 用户帐号密码等信息:Kind:Username with password,帐号、密码、ID和描述。

配置 SSH Server

Manage Jenkins -> System

  1. 搜索 SSH Server,配置 Docker Swarm Node 节点信息,点开 Advanced 可设置帐号密码登录或密钥登录,Remote Directory 创建工作目录并授权用户访问,配置完成后,点击右下角 Test Configuration 测试。
  2. 搜索 Gitlab,取消勾选:Enable authentication for '/project' end-point

配置 Gitlab

Admin -> 设置 -> 网络 -> 出站请求 Outbound requests -> 一:勾选'允许系统钩子向本地网络发送请求',二(不执行也可):Webhook 和服务可以访问的本地 IP 地址和域名:Jenkins服务器IP地址。
项目 -> 设置 -> Webhook

  1. URL:前面记录的 Webhook 值
  2. 触发来源:标签推送事件
  3. 取消 SSL 验证。

相关文件

Dockerfile-uwsgi

注意修改 DJANGO_SUPERUSER_PASSWORD,若不修改则默认密码为 admin
注意修改 Docker 加速echo "Acquire::http::Proxy \"http://192.168.120.2:10809\";" > /etc/apt/apt.conf.d/90proxy && \;若无需 Docker 加速,请删除该行。

FROM python:3.10.15-slim-bookworm
LABEL org.opencontainers.image.authors="Shanks.Lee" \
      org.opencontainers.image.blog="https://www.yudelei.com"
ENV DJANGO_SUPERUSER_PASSWORD="admin" DJANGO_SUPERUSER_EMAIL="root@localhost"
WORKDIR /usr/src/app
VOLUME /usr/src/app/data
COPY . .
RUN echo "Acquire::http::Proxy \"http://192.168.120.2:10809\";" > /etc/apt/apt.conf.d/90proxy && \
    mv docker/Entrypoint-uwsgi.sh ./Entrypoint.sh && \
    chmod +x ./Entrypoint.sh && apt update && apt install gcc -y && \
    pip install --no-cache-dir -r requirements.txt && \
    apt remove gcc -y && apt autoremove -y && \
    rm -rf /var/cache/apt/archives /var/lib/apt/lists/* /etc/apt/apt.conf.d/90proxy
EXPOSE 8000
ENTRYPOINT ["./Entrypoint.sh"]

Entrypoint-uwsgi.sh

点我查看文件内容

#!/bin/bash

echo "Flush the manage.py command it any"
while ! python manage.py flush --no-input 2>&1; do
  echo "Flusing django manage command"
  sleep 3
done
echo "Migrate the Database at startup of project"
# Wait for few minute and run db migraiton
while ! python manage.py migrate  2>&1; do
   echo "Migration is in progress status"
   sleep 3
done

yes yes | python manage.py collectstatic

if ! env |grep "DJANGO_SUPERUSER_PASSWORD" ; then
  export DJANGO_SUPERUSER_PASSWORD=admin
fi
if ! env |grep "DJANGO_SUPERUSER_EMAIL" ; then
  email=$DJANGO_SUPERUSER_EMAIL
else
  email="root@localhost"
fi

check_do=$(echo "from django.contrib.auth.models import User; \
superuser_exists = User.objects.filter(is_superuser=True).exists(); \
print(superuser_exists)" | python manage.py shell)
if [ $check_do != "True" ] ; then
  python manage.py createsuperuser --username admin --noinput
fi

unset DJANGO_SUPERUSER_PASSWORD
unset DJANGO_SUPERUSER_EMAIL

uwsgi --ini uwsgi.ini

deploy-swarm.sh

若主机端口被占用,会自动变更,若不需要,请注释下方:自动变更主机映射端口口
若本地镜像冲突(以项目名称判断),会删除已有镜像,若不需要,请注释下方:清理冲突镜像
将该文件放置到每台主机的/usr/bin/目录下,添加执行权限chmod +x /usr/bin/deploy-swarm.sh

点我查看文件内容

harbor_addr=$1
harbor_repo=$2
proj=$3
ver=$4
local_path=$5
harbor_user=$6
harbor_password=$7
port_zj=${8:-17002}
port_rq=${9:-8000}

mkdir -p $local_path
deploy_image=$harbor_addr/$harbor_repo/$proj:$ver

# 自动变更主机映射端口口
while nc -z -w2 127.0.0.1 $port_zj ; do
  let port_zj++
  sleep 0.1
done

export DEPLOY_IMAGE=$deploy_image
export DEPLOY_PORT_PUBLISH=$port_zj
export DEPLOY_DATA=$local_path

# 清理冲突镜像
old_images=`docker images |grep ${proj} |awk '{print $2}'`
if [[ "old_images_ids" =~ "$ver" ]] ; then
  docker rmi -f $image_name
fi

docker login -u $harbor_user -p $harbor_password $harbor_addr
docker stack deploy --with-registry-auth -c stack-compose.yml ${proj}-stack2

stack-compose.yml

注意192.168.120.53:80/repo/demo03:v0.0.16为默认私有仓库中的镜像或其他,-17002为默认发布端口;
变量实际值由 Jenkins Pipeline 的 Jenkinfile 文件中设置,并传入 stack-compose.yml。
若 compose.yml 不需要默认值,则删除:-"192.168.120.53:80/repo/demo03:v0.0.16":-17002
volumes为 Node 主机t01-ubuntu本机目录,则需绑定运行t01-ubuntu(注意修改为实际 node 名称);
理论上建议使用共享卷;中小型应用本地目录与共享目录均可满足性能要求;若为共享目录,则可注释块placement:
共享目录不可为 smb 或 nfs,会有资源锁 Lock,会导致 Sqlite 无法使用。

点我查看文件内容

version: "3.8"
services:
  django1:
    image: ${DEPLOY_IMAGE:-"192.168.120.53:80/repo/demo03:v0.0.16"}
    ports:
      - published: ${DEPLOY_PORT_PUBLISH:-17002}
        target: 8000
    volumes:
      - type: bind
        source: ${DEPLOY_DATA}
        target: /usr/src/app/data
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.hostname == t01-ubuntu 
      restart_policy:
        condition: on-failure
        max_attempts: 3
    networks:
      - net73

networks:
  net73:
    driver: overlay

Jenkinsfile-swarm

变量均为在全局 environment 中,请根据后方注释修改,并删除注释。
若更改了目录结构,有绝对路径的地方调整,请仔细检查。

点我查看文件内容

pipeline {
    agent any

    environment {
        harbor_addr = '192.168.120.53:80' # 私有仓库地址
        harbor_repo = 'repo' # 私有仓库项目名称
        proj_run_host = 't01-ubuntu' # 运行主机
        proj_run_host_path = '/data/django/' # 将会挂载到容器中的目录
        proj_host_port = 17002 # 主机端口
        proj_container_port = 8000 # 容器端口
        proj_name = '' # 由参数提取设置,若手动设置,请删除"参数提取"stage
        proj_tag = '' # 由参数提取设置,若手动设置,请删除"参数提取"stage
        tmp = '' # 临时变量
    }

    stages {
        stage('参数提取') {
            steps {
                script {
                    proj_name = env.gitlabSourceRepoName
                    echo "$proj_name"

                    tmp = env.gitlabBranch
                    def tmp_tag = tmp.replace("refs/tags/", "")
                    proj_tag = tmp_tag
                    echo "$proj_tag"
                }
            }
        }
        stage('代码拉取') {
            steps {
                checkout scmGit(branches: [[name: "$proj_tag"]], extensions: [],
                userRemoteConfigs: [[url: "$GIT_URL"]])
            }
        }
        stage('镜像构建') {
            environment {
                HARBOR = credentials("$jks_harbor_user_password_credentials_id")
            }
            steps {
                sh "docker build -t $proj_name:$proj_tag -f docker/Dockerfile-uwsgi . && \
docker login -u " + '$HARBOR_USR' + " -p " + '$HARBOR_PSW' + " $harbor_addr && \
docker tag $proj_name:$proj_tag $harbor_addr/$harbor_repo/$proj_name:$proj_tag && \
docker push $harbor_addr/$harbor_repo/$proj_name:$proj_tag"
            }
        }
        stage('推送文件') {
            steps {
sshPublisher(publishers: [sshPublisherDesc(configName: "$proj_run_host", # 
transfers: [sshTransfer(cleanRemote: false, excludes: '', execCommand: '',
execTimeout: 120000, flatten: false, makeEmptyDirs: false, noDefaultExcludes: false,
patternSeparator: '[, ]+', remoteDirectory: '', remoteDirectorySDF: false,
removePrefix: 'docker/', sourceFiles: 'docker/stack-compose.yml')],
usePromotionTimestamp: false, useWorkspaceInPromotion: false, verbose: false)])
            }
        }
        stage('容器发布') {
            environment {
                HARBOR = credentials("$jks_harbor_user_password_credentials_id")
            }
            steps {
sshPublisher(publishers: [sshPublisherDesc(configName: "$proj_run_host", transfers:
[sshTransfer(cleanRemote: false, excludes: '',
execCommand: "cd $proj_work_dir && \
deploy-swarm.sh $harbor_addr $harbor_repo $proj_name $proj_tag $local_path $HARBOR_USR" + \
'$HARBOR_PSW' + " $proj_host_port $proj_container_port",
execTimeout: 120000, flatten: false, makeEmptyDirs: false, noDefaultExcludes: false, patternSeparator: '[, ]+',
remoteDirectory: '', remoteDirectorySDF: false, removePrefix: '', sourceFiles: '')],
usePromotionTimestamp: false, useWorkspaceInPromotion: false, verbose: false)])
            }
        }
    }
}

requirements.txt

若不需要 pip 加速,请删除行-i https://pypi.tuna.tsinghua.edu.cn/simple

-i https://pypi.tuna.tsinghua.edu.cn/simple
Django==4.2.16
shortuuid==1.0.13
django-simpleui==2024.8.28
uwsgi==2.0.27

uwsgi.ini

[uwsgi]
http-socket = 0.0.0.0:8000
chdir = /usr/src/app
wsgi-file = %(chdir)/项目名称/wsgi.py # 注意修改
static-map = /static=%(chdir)/static
module = serious_django.wsgi
master = True
processes = 2
enable-threads=true
threads = 2
workers=4
harakiri=30
vacuum = true
max-requests = 5000

settings.py

注意修改。

点我查看文件内容

...
DEBUG = False
...
INSTALLED_APPS = [
    'simpleui',
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'app'
]
...
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': BASE_DIR / 'data/db.sqlite3',
    }
}
...
LANGUAGE_CODE = 'zh-hans'
TIME_ZONE = 'Asia/Shanghai'
USE_I18N = True
USE_TZ = True
...
STATIC_URL = 'static/'
STATIC_ROOT = os.path.join(BASE_DIR, "static")
...
# 以下非必要
DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'
SIMPLEUI_DEFAULT_ICON = False
SIMPLEUI_LOGIN_PARTICLES = False
# SIMPLEUI_HOME_PAGE = '/'
SIMPLEUI_HOME_TITLE = 'Django01'
SIMPLEUI_INDEX = '/'
SIMPLEUI_HOME_INFO = False
SIMPLEUI_HOME_QUICK = True
SIMPLEUI_ANALYSIS = False
SIMPLEUI_STATIC_OFFLINE = True

最后修改:2024 年 10 月 09 日 09 : 49 PM
如果觉得文章帮助了您,您可以随意赞赏。