一个非常无语的高危漏洞,任何人都可以重置任意账号的密码然后发送到指定邮箱
对于容器部署来说,直接修改 /var/opt/gitlab/nginx/conf/gitlab-http.conf 文件,在其中添加
location /users/password {
return 444;
}
然后重新加载nginx配置文件
gitlab-ctl hup nginx
一个非常无语的高危漏洞,任何人都可以重置任意账号的密码然后发送到指定邮箱
对于容器部署来说,直接修改 /var/opt/gitlab/nginx/conf/gitlab-http.conf 文件,在其中添加
location /users/password {
return 444;
}
然后重新加载nginx配置文件
gitlab-ctl hup nginx
升级至 gitlab-jh v16.5.0 以后,PostgreSQL 数据库的连接数会一直持续上升,直至触碰最大连接数上限,前台报500错误。升级至v16.5.2后恢复和以前版本一样的,连接数约200,峰值不超过500。
在HPC集群中通常有DNS和本地hosts提供解析服务,以便节点间通过主机名互相通信,而不是直接使用IP地址。但是如果在集群外有一个独立服务器中的容器需要与集群内的节点通过主机名通讯,就需要通过DNS来给容器提供解析服务。
通过自动化脚本将集群的hosts拷贝到独立服务器的一个目录下,如 /home/hpc/dns/hosts
自己做一个dnsmasq的容器:
[yaoge123]$ cat dnsmasq/Dockerfile
FROM alpine:latest
RUN apk update \
&& apk upgrade \
&& apk add --no-cache \
dnsmasq \
&& rm -rf /var/cache/apk/*
编写docker-compose.yml:
services:
dnsmasq:
build: ./dnsmasq
image: dnsmasq
container_name: dnsmasq
networks:
default:
ipv4_address: 192.168.100.200
volumes:
- /home/hpc/dns:/etc/dns:ro
command:
- dnsmasq
- --keep-in-foreground
#- --no-daemon
#- --log-queries
- --domain-needed
- --no-hosts
- --cache-size=3000
- --hostsdir=/etc/dns
abc:
image: abc
container_name: abc
dns:
- 192.168.100.200
…………
networks:
default:
driver: bridge
ipam:
driver: default
config:
- subnet: 192.168.100.0/24
测试解析和查看 dnsmasq 缓存情况,evictions为0最好
[yaoge123]# run --rm -it --network=docker_default --dns=192.168.100.200 alpine sh
/ # apk add bind-tools
/ # dig +short node_name
/ # for i in "cachesize.bind insertions.bind evictions.bind misses.bind hits.bind auth.bind servers.bind";do dig +short chaos txt $i;done
大体架构是:前端 Nginx 卸载SSL后反代给 OnlyOffice,OnlyOffice 将监控信息喂给 statsd-exporter,Prometheus 从 statsd-exporter 拉取数据,最终用 Grafana 可视化展示。
先来 docker-compose.yml 配置如下,有一些注意修改的地方:
services:
onlyoffice:
image: onlyoffice/documentserver-ee
container_name: onlyoffice
volumes:
- ./onlyoffice/logs:/var/log/onlyoffice
- ./onlyoffice/data:/var/www/onlyoffice/Data
- ./onlyoffice/lib:/var/lib/onlyoffice
- ./onlyoffice/db:/var/lib/postgresql
- ./onlyoffice/fonts/dejavu:/usr/share/fonts/dejavu
- ./onlyoffice/fonts/founder:/usr/share/fonts/founder
- ./onlyoffice/fonts/liberation:/usr/share/fonts/liberation
- ./onlyoffice/fonts/libertinus:/usr/share/fonts/libertinus
- ./onlyoffice/fonts/noto-cjk:/usr/share/fonts/noto-cjk
- ./onlyoffice/fonts/noto-emoji:/usr/share/fonts/noto-emoji
- ./onlyoffice/fonts/sarasa-gothic:/usr/share/fonts/sarasa-gothic
- ./onlyoffice/fonts/source-code-pro:/usr/share/fonts/source-code-pro
- ./onlyoffice/fonts/source-han-sans:/usr/share/fonts/source-han-sans
- ./onlyoffice/fonts/source-han-serif:/usr/share/fonts/source-han-serif
- ./onlyoffice/fonts/source-sans:/usr/share/fonts/source-sans
- ./onlyoffice/fonts/source-serif:/usr/share/fonts/source-serif
- ./onlyoffice/fonts/windows:/usr/share/fonts/windows
- ./onlyoffice/local-production-linux.json:/etc/onlyoffice/documentserver/local-production-linux.json
environment:
- TZ=Asia/Shanghai
- JWT_ENABLED=true
- JWT_SECRET=yaoge123
statsd-exporter:
image: prom/statsd-exporter
container_name: statsd-exporter
ports:
- 9102:9102
depends_on:
- onlyoffice
nginx:
image: nginx:alpine
container_name: nginx
ports:
- 80:80
- 443:443
volumes:
- ./nginx/conf.d/:/etc/nginx/conf.d/:ro
- ./nginx/ssl/:/etc/nginx/ssl/:ro
environment:
- TZ=Asia/Shanghai
nginx-exporter:
image: nginx/nginx-prometheus-exporter
container_name: nginx-prometheus-exporter
ports:
- 9113:9113
command:
- -nginx.scrape-uri
- http://nginx:8080/stub_status
depends_on:
- nginx
……
在 local-production-linux.json 对 OnlyOffice 进行持久化配置,主要是三部分:
{
"statsd": {
"useMetrics": true,
"host": "statsd-exporter",
"port": "9125",
"prefix": "ds."
},
"services": {
"CoAuthoring": {
"autoAssembly": {
"enable": true,
"interval": "5m"
}
}
},
"FileConverter": {
"converter": {
"maxDownloadBytes": 1073741824,
"downloadAttemptMaxCount": 3
}
}
}
Nginx 配置反代卸载SSL
map $http_host $this_host {
"" $host;
default $http_host;
}
map $http_x_forwarded_proto $the_scheme {
default $http_x_forwarded_proto;
"" $scheme;
}
map $http_x_forwarded_host $the_host {
default $http_x_forwarded_host;
"" $this_host;
}
map $http_upgrade $proxy_connection {
default upgrade;
"" close;
}
server {
listen 80;
listen [::]:80;
server_name onlyoffice.nju.edu.cn;
rewrite ^ https://$http_host$request_uri? permanent;
server_tokens off;
}
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name onlyoffice.nju.edu.cn;
server_tokens off;
include ssl/nju_edu_cn.conf;
add_header X-Content-Type-Options nosniff;
location / {
proxy_pass http://onlyoffice;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $proxy_connection;
proxy_set_header X-Forwarded-Host $the_host;
proxy_set_header X-Forwarded-Proto $the_scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
server {
listen 8080;
listen [::]:8080;
server_name localhost;
location /stub_status {
stub_status on;
access_log off;
}
}
在 Consul 中注册 OnlyOffice 和 Nginx 的监控
[yaoge123 ~]$ curl -X PUT -d '{"id": "onlyoffice.nju.edu.cn_statsd-exporter","name": "statsd_exporter","address": "onlyoffice.nju.edu.cn","port": 9102,"tags": ["prometheus","vm"],"checks": [{"http": "http://onlyoffice.nju.edu.cn:9102/metrics","interval": "30s"}]}' http://consul:8500/v1/agent/service/register
[yaoge123 ~]$ curl -X PUT -d '{"id": "onlyoffice.nju.edu.cn_nginx-exporter","name": "nginx_exporter","address": "onlyoffice.nju.edu.cn","port": 9113,"tags": ["prometheus","vm"],"checks": [{"http": "http://onlyoffice.nju.edu.cn:9113/metrics","interval": "30s"}]}' http://consul:8500/v1/agent/service/register
在Grafana中导入基于官方Dashboard的修改版
既然是集群,首先将域控变成两台:
将AD2加入域中,服务器角色中添加AD域服务,然后将其提升为域控,选择添加到现有域。
集群嘛,多安装几台 Office Online Server,但不导入SSL证书。
在第一台上用管理员打开PowerShell部署 Office Online Server farm
New-OfficeWebAppsFarm -InternalUrl "https://officeonline1.ad.local" -ExternalUrl "https://officeonline.nju.edu.cn" -SSLOffloaded -EditingEnabled
在其它台上用管理员打开PowerShell加入 Office Online Server farm
New-OfficeWebAppsMachine -MachineToJoin "officeonline1.ad.local"
在PowerShell中查看所有节点的状态
(Get-OfficeWebAppsFarm).Machines
前端 Nginx 负载均衡卸载SSL
upstream officeonline {
ip_hash;
server 192.168.1.11:80;
server 192.168.1.12:80;
server 192.168.1.13:80;
}
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
listen 80;
listen [::]:80;
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name officeonline.nju.edu.cn;
include ssl/nju_edu_cn.conf;
location / {
proxy_pass http://officeonline;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_cache_bypass $http_upgrade;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Real-PORT $remote_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
Office Online Server 必须加入域,且不能装在域控上,因此先装两台带桌面体验的 Windows Server 2022 并升级。
第一台用来做域控,给2C/4G的资源就够了,服务器角色中添加AD域服务,然后将其提升为域控,如添加新林的根域名为 ad.local
第二台用来装Office Online Server,要多给资源,首先将其DNS改为域控的IP,加入域并重启。
用管理员打开PowerShell执行安装要求的角色和服务,安装完成后重启。
Add-WindowsFeature Web-Server,Web-Mgmt-Tools,Web-Mgmt-Console,Web-WebServer,Web-Common-Http,Web-Default-Doc,Web-Static-Content,Web-Performance,Web-Stat-Compression,Web-Dyn-Compression,Web-Security,Web-Filtering,Web-Windows-Auth,Web-App-Dev,Web-Net-Ext45,Web-Asp-Net45,Web-ISAPI-Ext,Web-ISAPI-Filter,Web-Includes,NET-Framework-Features,NET-Framework-45-Features,NET-Framework-Core,NET-Framework-45-Core,NET-HTTP-Activation,NET-Non-HTTP-Activ,NET-WCF-HTTP-Activation45,Windows-Identity-Foundation,Server-Media-Foundation
安装如下依赖软件,部分软件在Windows 2022中已经存在(.NET和VC++ 2015),安装程序会有提示直接退出,这个没关系
安装 Office Online Server ,当前版本是2018年11月发布的16.0.10338.20039
0a764830ee8cca9a92f749c9a6a9cd6a0e99e592 en_office_online_server_last_updated_november_2018_x64_dvd_1b5ae10d.iso
从Microsoft Download Center下载语言包,在下载页面切换语言可以下载不同语言的语言包,当前是2018年11月发布的语言包,将多个语言包依次安装
87b77a9abf29cf11e95f74c07fb7419026882c05 wacserverlanguagepack-20181129cn.exe
33965054f9e93f6429cf3022dea0f6a575f8d178 wacserverlanguagepack-20181129en.exe
0432a9b8a27b7f56692e37c117b8cf79c45458fa wacserverlanguagepack-20181129tw.exe
从Microsoft Update Catalog查找下载最新的Microsoft Office Online Server 服务器场部署安全更新,当前版本是2022年11月8日发布的KB5002276,安装时也会自动更新语言包
5abacedb0f21d8629a0a0eab1b4d163ca0228f96 wacserver-x-none_5abacedb0f21d8629a0a0eab1b4d163ca0228f96.cab
在MMC控制台-证书(本地计算机)-个人中导入pfx格式含私钥的HTTPS证书,注意IIS不支持双证书,因此考虑兼容性导入RSA证书,导入后右击证书属性,设置友好名称(如nju.edu.cn)
用管理员打开PowerShell部署 Office Online Server farm
New-OfficeWebAppsFarm -InternalUrl "https://officeonline.ad.local" -ExternalUrl "https://officeonline.nju.edu.cn" -CertificateName "nju.edu.cn" -EditingEnabled
在PowerShell查看详细配置
Get-OfficeWebAppsFarm
构建一个安装了SST的CentOS8容器
singularity build –sandbox sst-build docker://centos:8.4.2105
cp sst-*.x86_64.rpm sst-build/home
singularity shell -w sst-build
rpm -ivh /home/sst-*.x86_64.rpm
rm /home/sst-*.x86_64.rpm
exit
singularity build sst.sif sst-build
到节点上升级
singularity shell –writable-tmpfs sst.sif
sst show -ssd
sst load -ssd 0
sst load -ssd 1
……
exit
reboot
VPN:
在以上配置的情况下,单一客户端可以跑满总带宽,多个客户端时可以保证每个客户端至少能获得最小带宽。
GPFS提供两种高可用NFS服务的方式,分别是Cluster NFS (CNFS)和Cluster Export Services (CES),二者互斥只能选其一。CNFS只支持NFS、CES支持NFS/SMB/Object。CNFS基于Linux kernel的NFS server,NFS的配置不由GPFS管理,元数据性能较好;CES基于用户空间的Ganesha NFS server,GPFS管理NFS配置,数据流式访问性能好。注意两者切换必将导致NFS停机。
设置CES共享目录:此目录每个CES节点应均能访问,此步骤需要整个GPFS集群停机
mmshutdown -a
mmchconfig cesSharedRoot=/share/ces
mmstartup -a
添加CES节点:
mmchnode –ces-enable -N ces1,ces2
配置CES IP:CES IP是专用于提供NFS/SMB/Object服务的虚IP,不可用于内部GPFS通讯,CES IP必须可通过DNS或/etc/hosts解析。每个CES节点上应配置有和CES虚IP相同网段IP的网络接口,GPFS只能给这些网络接口添加子IP。如ces1配置有192.168.1.101/24、ces2配置有192.168.1.102/24,CES IP为192.168.1.11和192.168.1.12
mmces address add –ces-ip 192.168.1.11,192.168.1.12
验证CES IP:
[root@ces1 ~]# mmces address list –full-list
cesAddress cesNode attributes cesGroup preferredNode unhostableNodes
192.168.1.11 ces2 none none none none
192.168.1.12 ces1 none none none none
安装NFS:
yum install pyparsing pygobject2 libwbclient
rpm -ivh gpfs.nfs-ganesha-2.7.5-ibm058.12.el7.x86_64.rpm gpfs.nfs-ganesha-gpfs-2.7.5-ibm058.12.el7.x86_64.rpm gpfs.nfs-ganesha-utils-2.7.5-ibm058.12.el7.x86_64.rpm
安装SMB:
yum install libarchive gdb
rpm -ivh gpfs.smb-4.11.16_gpfs_19-2.el7.x86_64.rpm
启用CES NFS:
mmces service enable nfs
在所有CES节点上启动NFS服务:
mmces service start NFS -a
验证CES NFS:
[root@ces1 ~]# mmces service list -a
Enabled services: NFS
ces1: NFS is running
ces2: NFS is running
推荐创建一个独立的fileset用于NFS:
mmcrfileset share data –inode-space new
mmlinkfileset share data -J /share/data
设置用户认证方式:
mmuserauth service create –data-access-method file –type userdefined
创建NFS共享:
mmnfs export add /share/data –client “192.168.1.100/32(Access_Type=RW)”
检查NFS共享:
[root@ces1 ~]# mmnfs export list
Path Delegations Clients
—————— ———– ————-
/share/data NONE 192.168.1.100/32