Merge pull request #404 from bytedance/ypb

Ypb

elkeidup/README-zh_CN.md

@@ -19,8 +19,6 @@ Elkeid 自动化部署工具
 * 安装后不要删除 `~/.elkeidup` 目录
 * 不要修改任何组件的密码，包括Elkeid Console(Manager)初始默认用户
 
-> 注意：由于v1.9.1版本与v1.7之间差异较大，不支持从v1.7直接平滑升级到v1.9.1，可参考[重装指定组件](./deploy-zh_CN.md#5、重装指定组件)。v1.9.1往后的版本都会支持平滑升级。
-
 ### 自动下载缺失预编译ko服务开启提示
 
 **服务背景**：
@@ -48,6 +46,10 @@ Elkeid Driver是在内核态工作的，由于内核要求加载的内核模块
 ### Elkeid HUB 单独部署
 [Elkeid HUB 单独部署](./deploy_hub-zh_CN.md)
 
+### Elkeid 升级和扩容
+- [Elkeid 升级指南](./update-zh_CN.md)
+- [Elkeid 扩容指南](./expansion-zh_CN.md)
+
 ## Raw Data Usage Tutorial
 - [Elkeid 数据说明](../server/docs/ElkeidData.xlsx)
 - [Raw Data Usage Tutorial](raw_data_usage_tutorial/raw_data_usage_tutorial-zh_CN.md)

elkeidup/README.md

@@ -19,8 +19,6 @@ Automated deployment of Elkeid tools
 * Don't remove `~/.elkeidup` dir
 * Don't fix any components used user's password, Include the Console(Elkeid Manager)
 
-> > Note: Due to the large difference between v1.9.1 and v1.7, direct smooth upgrade from v1.7 to v1.9.1 is not supported,  Refer to [Upgrade specified components](./deploy.md#5、Upgrade specified components). Versions after v1.9.1 will support smooth upgrades.
-
 ## Awareness of Auto-download missing kernel driver service
 
 In this open-source version, we have integrated a service to provide auto-download capabilities for kernel driver files of those kernel versions that are missing from pre-compiled lists.
@@ -53,6 +51,10 @@ The codes for collecting information and downloading KO files from Elkeid servic
 ### Elkeid HUB Deployment
 [Elkeid HUB Deployment Only](./deploy_hub.md)
 
+### Elkeid Upgrading and Expansion
+- [Elkeid Upgrade Guide](./update.md)
+- [Elkeid Expansion Guide](./expansion.md)
+
 ## Raw Data Usage Tutorial
 - [Elkeid Data Description](../server/docs/ElkeidData.xlsx)
 - [Raw Data Usage Tutorial](raw_data_usage_tutorial/raw_data_usage_tutorial-zh_CN.md)

elkeidup/expansion-zh_CN.md

@@ -0,0 +1,41 @@
+# Elkeid 社区版 扩容指南
+
+## ServiceDiscovery
+
+### 自身扩容(依赖elkeidup)
+
+1.  修改config.yaml 在sd中添加其他的host，登录条件与安装时相同。
+2.  执行以下命令 elkeidup reinstall --component ServiceDiscovery --re-init
+
+### 自身扩容(手动操作)
+
+1.  拷贝已安装好SD机器的 /elkeid/service_discovery 到待扩容机器上。
+2.  更新全部SD的配置文件 /elkeid/service_discovery/conf/conf.yaml 的 Cluster.Members项，该项为所有sd实例的数组，每台sd都要填写全部实例的地址。
+3.  执行新SD实例的 /elkeid/service_discovery/install.sh ，会自动启动sd。
+4.  重启所有旧的sd实例 `systemctl restart elkeid_sd` 。
+
+### 同步修改上下游配置
+
+sd目前同时被AgentCenter，Manager和Nginx所依赖，扩容SD后，需要同步重启。
+
+-   AgentCenter: 配置文件位于/elkeid/agent_center/conf/svr.yml 的 sd.addrs，重启命令 `systemctl restart elkeid_ac`。
+-   Manager: 配置文件位于/elkeid/manager/conf/svr.yml 的 sd.addrs，重启命令 `systemctl restart elkeid_manager`。
+-   Nginx: 配置文件位于/elkeid/nginx/nginx/nginx.conf 的 upstream sd_list，重启命令 `systemctl restart elkeid_nginx`。
+
+## AgentCenter
+
+### 自身扩容(依赖elkeidup)
+
+1.  修改config.yaml 在ac中添加其他的host，登录条件与安装时相同。
+2.  执行以下命令 elkeidup reinstall --component AgentCenter --re-init
+
+### 自身扩容(手动操作)
+
+1.  拷贝已安装好AC机器的 /elkeid/agent_center 到待扩容机器上。
+2.  执行新AC实例的 /elkeid/agent_center/install.sh ，会自动安装和启动AC。
+
+### 同步修改上下游配置
+
+若agent通过服务发现的方式连接到AC，则不需要手动同步上下游配置。
+
+若agent通过编码的AC地址连接AC，需要重新编译agent，将新的AC地址加入到agent连接配置中。

elkeidup/expansion.md

@@ -0,0 +1,41 @@
+# Elkeid Community Edition, Expansion Guide
+
+## ServiceDiscovery
+
+### Self-expansion (dependency elkeidup)
+
+1. Modify config.yaml add other hosts in sd, and the login conditions are the same as when installing.
+2. Execute the following command elkeidup reinstall --component ServiceDiscovery --re-init
+
+### Self-expansion (manual operation)
+
+1. Copy the /elkeid/service_discovery of the installed SD machine to the machine to be expanded.
+2. Update all SD configuration file /elkeid/service_discovery/conf/conf.yaml Cluster. Members item, which is an array of all SD instances, and each SD must fill in the addresses of all instances.
+3. Execute the /elkeid/service_discovery/install.sh of the new SD instance, which will automatically start SD.
+4. Restart all old sd instances'systemctl restart elkeid_sd '.
+
+### sync up the upstream and downstream configuration
+
+SD is currently a dependency of AgentCenter, Manager and Nginx. After expanding SD, you need to sync up and restart.
+
+- AgentCenter: The configuration file is located sd.addrs/elkeid/agent_center/conf/svr.yml, restart the command'systemctl restart elkeid_ac '.
+- Manager: configuration file is sd.addrs/elkeid/manager/conf/svr.yml, restart command'systemctl restart elkeid_manager '.
+- Nginx: configuration file is located in the upstream sd_list of/elkeid/nginx/nginx/nginx.conf, restart command'systemctl restart elkeid_nginx '.
+
+## AgentCenter
+
+### Self-expansion (dependency elkeidup)
+
+1. Modify config.yaml add other hosts in ac, and the login conditions are the same as when installing.
+2. Execute the following command elkeidup reinstall --component AgentCenter --re-init
+
+### Self-expansion (manual operation)
+
+1. Copy the /elkeid/agent_center of the installed AC machine to the machine to be expanded.
+2. Executing the /elkeid/agent_center/install.sh of the new AC instance installs and starts AC automatically.
+
+### sync up the upstream and downstream configuration
+
+If the agent is linked to the AC by means of service discovery, there is no need to manually sync up the upstream and downstream configurations.
+
+If the agent is linkage AC through the AC address of the code, you need to re-compile the agent and add the new AC address to the agent linkage configuration.

elkeidup/update-zh_CN.md

@@ -0,0 +1,102 @@
+Elkeidup 社区版升级指南 1.7.1 --> 1.9.1
+
+# 前言
+
+首先需要配置elkeidup 1.7.1 与 1.9.1 版本共存，然后按情况进行切换。
+
+详细操作请同时参照1.7.1 与 1.9.1 的文档。
+
+```
+# rename .elkeidup dir
+cd /root
+mv .elkeidup .elkeidup_v1.7.1
+ln -s .elkeidup_v1.7.1 .elkeidup
+
+# copy cert to v1.9.1
+mkdir -p /root/.elkeidup_v1.9.1
+cp -r /root/.elkeidup_v1.7.1/elkeid_password /root/.elkeidup_v1.9.1
+cp -r /root/.elkeidup_v1.7.1/cert /root/.elkeidup_v1.9.1
+# download v1.9.1 package to /root/.elkeidup_v1.9.1
+```
+
+切换到 1.7.1
+
+```
+rm /root/.elkeidup && ln -s /root/.elkeidup_v1.7.1 /root/.elkeidup
+```
+
+切换到 1.9.1
+
+```
+rm /root/.elkeidup && ln -s /root/.elkeidup_v1.9.1 /root/.elkeidup
+```
+
+# 后端
+
+v1.9.1后端目前无法与v1.7.1兼容，需要卸载v1.7.1后端后重新安装v1.9.1。
+
+## 备份数据
+
+根据需要选择备份数据：
+
+1.  备份 MongoDB：目录位于 /elkeid/mongodb 仅是备份DB，备份的数据无法直接使用，如果有恢复需求，目前尚无自动化脚本，需要手动转换。
+2.  备份Hub策略：目录位于 /elkeid/hub 策略可以在Hub web界面中进行导入。
+
+## 卸载v1.7.1
+
+> 在卸载v1.7.1后端后，Agent将在1min后自动关闭所有插件，并进入守护状态，直到新的后端被安装
+
+```
+# 按照前言操作切换到 v1.7.1
+
+cd /root/.elkeidup 
+./elkeidup undeploy
+```
+
+## 安装v1.9.1
+
+> 在安装v1.9.1后端后，Agent将在1min内重连，但此时还尚未加载任何插件，您可以在Console上看到这个状态
+
+```
+# 按照前言操作切换到 v1.9.1
+# 安装文档详见v1.9.1 安装文档
+cd /root/.elkeidup 
+./elkeidup deploy
+```
+
+# Agent
+
+## 确认配置及状态
+
+-   `/root/.elkeidup_v1.7.1/cert` `/root/.elkeidup_v1.9.1/cert` 三个目录内的所有文件内容均保持一致
+
+-   `/root/.elkeidup_v1.7.1/elkeid_server.yaml` `/root/.elkeidup_v1.9.1/elkeidup_config.yaml` 三个文件中，下述相关配置均保持一致。
+
+    -     注：具体字段filed值以`v1.9.1`为准，请勿直接覆盖。
+
+    -   nginx
+
+        -   domain
+        -   ssh_host
+        -   public_addr
+
+    -   mg
+
+        -   ssh_host
+
+-   确认后端更新完成后，所有v1.7.1的Agent均已成功上线
+
+## Build v1.9.1组件
+
+```
+./elkeidup agent init
+./elkeidup agent build
+./elkeidup agent policy create
+```
+
+## 下发任务
+
+> 可根据需要进行灰度升级，此时新上线/重连的客户端会自动拉取最新配置升级，其他客户端需要手动同步配置升级
+
+1.  在 [Elkeid Console - 任务管理](../server/docs/console_tutorial/Elkeid_Console_manual.md#任务管理) 界面，点击“新建任务”，选择单台主机，点击下一步，选择“同步配置”任务类型，点击确认。随后，在此页面找到刚刚创建的任务，点击运行，观察升级后的主机是否符合预期。
+2.  在 [Elkeid Console - 任务管理](../server/docs/console_tutorial/Elkeid_Console_manual.md#任务管理) 界面，点击“新建任务”，选择全部主机，点击下一步，选择“同步配置”任务类型，点击确认。随后，在此页面找到刚刚创建的任务，点击运行，即可对存量旧版本Agent进行升级。

elkeidup/update.md

@@ -0,0 +1,102 @@
+Elkeidup Community Edition Upgrade Guide 1.7.1 -- > 1.9.1
+
+# Foreword
+
+First you need to configure elkeidup 1.7.1 to coexist with version 1.9.1, and then switch as the case may be.
+
+For detailed operation, please refer to the documentation of 1.7.1 and 1.9.1 at the same time.
+
+```
+# rename .elkeidup dir
+cd /root
+mv .elkeidup .elkeidup_v1.7.1
+ln -s .elkeidup_v1.7.1 .elkeidup
+
+# copy cert to v1.9.1
+mkdir -p /root/.elkeidup_v1.9.1
+cp -r /root/.elkeidup_v1.7.1/elkeid_password /root/.elkeidup_v1.9.1
+cp -r /root/.elkeidup_v1.7.1/cert /root/.elkeidup_v1.9.1
+# download v1.9.1 package to /root/.elkeidup_v1.9.1
+```
+
+Switch to 1.7.1
+
+```
+rm /root/.elkeidup && ln -s /root/.elkeidup_v1.7.1 /root/.elkeidup
+```
+
+Switch to 1.9.1
+
+```
+rm /root/.elkeidup && ln -s /root/.elkeidup_v1.9.1 /root/.elkeidup
+```
+
+# Backend
+
+The v1.9.1 backend is currently not compatible with v1.7.1, you need to uninstall the v1.7.1 backend and reinstall v1.9.1.
+
+## backup data
+
+Select backup data as needed:
+
+1. Backup MongoDB: The directory is located /elkeid/mongodb is only a backup DB, and the backed up data cannot be used directly. If there is a recovery need, there is no automated script at present, and manual conversion is required.
+2. Backup Hub Policies: The directory is located /elkeid/hub Policies can be imported in the Hub web interface.
+
+## uninstall v1.7.1
+
+> After uninstalling the v1.7.1 backend, Agent will automatically close all plugins after 1 minute and enter the daemon state until the new backend is installed
+
+```
+#switch to v1.7.1 according to the preface
+
+cd /root/.elkeidup 
+./elkeidup undeploy
+```
+
+## install v1.9.1
+
+> After installing the v1.9.1 backend, the Agent will be reconnected within 1min, but no plugins have been loaded at this time, you can see this state on the Console
+
+```
+#switch to v1.9.1 according to the preface
+#For installation documentation, see v1.9.1 installation documentation
+cd /root/.elkeidup
+./elkeidup deploy
+```
+
+# Agent
+
+## Confirm configuration and state
+
+- '/root/elkeidup_v1/cert'/root/elkeidup_v1/cert 'The contents of all files in the three directories are consistent
+
+- '/root/elkeidup_v1/elkeid_server.yaml'/root/elkeidup_v1/elkeidup_config.yaml 'The following related configurations are consistent.
+
+    - Note: The filed value of the specific field is subject to'v1.9.1 ', please do not directly cover.
+
+    - nginx
+
+        - domain
+        - ssh_host
+        - public_addr
+
+    - mg
+
+        - ssh_host
+
+- After confirming that the backend update is complete, all v1.7.1 Agents have been successfully launched
+
+## Build v1.9.1 component
+
+```
+./elkeidup agent init
+./elkeidup agent build
+./elkeidup agent policy created
+```
+
+## Submit a task
+
+> Grey release upgrade can be performed as needed. At this time, the newly launched/reconnected client/client side/client end will automatically pull the latest configuration upgrade, and other client/client side/client ends need to manually sync up configuration upgrade
+
+1. In the [Elkeid Console - Task Management](../server/docs/console_tutorial/Elkeid_Console_manual.md#任务管理) interface, click "New Task", select a single host, click Next, select the "sync up configuration" task type, and click Confirm. Then, find the task you just created on this page, click Run, and observe whether the upgraded host meets expectations.
+2. In the [Elkeid Console - Task Management](../server/docs/console_tutorial/Elkeid_Console_manual.md#任务管理) interface, click "New Task", select all hosts, click Next, select "sync up configuration" task type, and click Confirm. Then, find the task you just created on this page and click Run to upgrade the old version of Agent.