首页

    博客迁移到kubernetes

    标签:kubernetes,blog

    开始时,不想把nginx配置和lua代码放在repository,你懂的,就只有mount volume的方式,把它们放到pod里了

    创建configmap

    对lua文件夹

    `kubectl create configmap lua --from-file=path-to-lua-dir`
    

    同理,对nginx.conf

    kubectl create configmap nginx-conf --from-file=nginx.conf
    

    更新yaml文件

    ...
    spec:
      containers:
        volumeMounts:
        - name: lua-volume
          mountPath: /root/lua
        - name: conf-volume
          mountPath: /usr/local/openresty/nginx/conf
      volumes:
      - name: lua-volume
        configMap:
          name: lua
      - name: conf-volume
        configMap:
          name: nginx-conf
    

    执行后发现/usr/local/openresty/nginx/conf目录只有nginx.conf一个文件
    这就需要设置subPath

    Sometimes, it is useful to share one volume for multiple uses in a single Pod. The volumeMounts.subPath property can be used to specify a sub-path inside the referenced volume instead of its root.

    ...
    spec:
      containers:
        volumeMounts:
        ...
        - name: conf-volume
          mountPath: /usr/local/openresty/nginx/conf/nginx.conf
          subPath: nginx.conf
      volumes:
      ...
      - name: conf-volume
        configMap:
          name: nginx-conf
    

    更新configmap

    可以

    • 执行kubectl edit cm lua
    • 或者将configmap写成yaml格式
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: lua
    data:
      access.lua: |
        local host=ngx.var.host
        ...
    

    执行kubectl replace -f lua.yaml更新

    更新后pod需要几十秒才能同步更新,但是如果像上面mount nginx.conf那样,使用subPath,pod是不会同步更新nginx.conf修改的

    A container using a ConfigMap as a subPath volume will not receive ConfigMap updates

    使更新生效

    只有一个pod的话

    1. kubectl exec pod -it sh进入pod
    2. nginx -s reload

    多个pod的话,可以

    • kubectl scale deploy your_deployment --replicas=0,再kubectl scale deploy your_deployment --replicas=num将副本数设置为原本数量
    • 或者给deployment添加一个值为当前unix时间戳的label
    kubectl patch deployment blog -p "{\"spec\":{\"template\":{\"metadata\":{\"labels\":{\"date\":\"`date +'%s'`\"}}}}}"
    
    • 如果每次release都打一个新tag的话,可以
    kubectl set image deployment/blog blog=blog:new-tag
    

    触发rolling update

    注意只是执行kubectl apply的话,如果pod配置没有变化,是不会rolling update

    A Deployment’s rollout is triggered if and only if the Deployment’s pod template (that is, .spec.template) is changed, for example if the labels or container images of the template are updated.
    Other updates, such as scaling the Deployment, do not trigger a rollout.


    后来对原来的nginx进行了拆分,nginx配置和lua代码可以给别人看了,哈哈,就不用上面的mount volume了.

    ansible copy files to remote

    可以用ansible将它们复制到目标机器,再用上面的方法更新configmap,并使之生效.
    然而ansible的copy module只能复制文件,用synchronize module就能复制文件夹了

    ...
    - synchronize:
        src: ../lua
        dest: remote_path
        delete: yes
    ...
    

    Dockerfile ADD指令

    也可以在构建image时,把它们加到image,但是lua代码在文件夹,使用ADD指令

    #When using ADD with more than one source file, the destination must be a directory and end with a /
    ADD src lua /build/
    

    只会复制lua文件夹里的文件,不会复制文件夹本身,也就是说,/build目录没有src,lua文件夹,只有src和lua文件夹里的文件,文件夹

    If is a directory, the entire contents of the directory are copied, including filesystem metadata.
    Note: The directory itself is not copied, just its contents.

    可以在构建image前

    1. .gitlab-ci.yml build stage添加script
      script:
        ...
        - tar -cf source.tar nginx.conf lua src gulpfile
    

    将所有需要添加到image的文件,文件夹打包

    1. Dockerfile里ADD压缩包
    ADD source.tar /build
    

    ADD指令会自动解压常用压缩格式

    If is a local tar archive in a recognized compression format (identity, gzip, bzip2 or xz) then it is unpacked as a directory.

    1. release stage时,使用multi-stage builds,把nginx.conf和lua文件夹也添加到release image
    ...
    COPY --from=build /build/nginx.conf /usr/local/openresty/nginx/conf
    COPY --from=build /build/lua /root/lua
    COPY --from=build /build/blog /root/blog
    

    ansible when statement

    最后deploy,可以先kubectl apply,如果pod配置有变化,这里就会rolling update整个服务,输出的stdout里会有configured.
    否则,可以把kubectl apply的stdout保存在out变量,执行下一个command时,通过判断out变量中是否有configured(即kubectl apply是否更新了服务),决定是否为deployment添加value为当前unix时间戳的label(即强制更新)

        ...
        - name: kubernetes deploy
          command: /usr/bin/kubectl apply -f ~/blog/k8s.yaml
          register: out
    
        - name: kubernetes force pull
          when: out.stdout.find('configured')==-1
          command: /usr/bin/kubectl patch deployment blog -p {\"spec\":{\"template\":{\"metadata\":{\"labels\":{\"date\":\"{{ansible_date_time.epoch}}\"}}}}}
    

    另外,不是以root用户执行kubectl命令的话,需要额外设置身份认证

    mkdir -p $HOME/.kube
    sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    sudo chown $(id -u):$(id -g) $HOME/.kube/config
    

    不定期更新