Task manager tier 2

[Dwordcito]

Description

This objective is a continuation of the command manager issue: wazuh/wazuh-indexer#349.
The goal is to expand the available commands and complete any pending tasks that were left unfinished in the previously mentioned objective.

From now on, what was call command manager, will now be referred as task manager. The task manager will accept task request and will output commands.
We will create a new module named command manager in the indexer that will process commands aimed at the indexer. After processing them all of this commands will generate a new commands aimed at server or/and agents.

The reason for this naming is that the current commands are too granular from the user’s perspective, and they need to be grouped or abstracted under parent tasks.

List of commands and their purpose:

Command	Description	Trigger	Target	Requires Expansion
Group assignment change	Changes the groups an agent belongs to	Group assignment change or group deletion	Indexer	Yes
Group configuration change	Updates the configuration of a specific group	Configuration change in the associated group	Indexer	Yes
Agent remote upgrade	On-demand upgrade request for an agent	Upgrade request	Agent	Yes
New agent package	Server downloads agent packages	New agent package	Server	No
Agent name change	Renames the agent	Name change request	Indexer	No
SCA policy change	Modifies or deletes a specific policy for a group of agents	Policy update or removal triggered by content manager or user	Indexer	Yes
SCA content update	Indexer updates its local sca database	SCA content update	Agent	Yes
Vulnerability DB rescan	Forces the agent to re-scan its vulnerabilities	CVE content update	Agent	Yes
CVE content update	Server updates its local vulnerability database	CVE content update	Server	No
Active response configuration	Modifies or deletes a specific active response for a group of agents	Active response update or removal via content manager or user	Indexer	Yes
Active response execution	Executes an active response command	Detection rule triggers active response	Indexer	Yes
Ruleset update	Server updates its local ruleset	Ruleset content update	Indexer	No
Server configuration change	Server receives a configuration change	Configuration update via dashboard/API	Server	No
Agent deletion	Agent notifies the system to stop connecting	Agent deletion from API or dashboard	Indexer	No
Management API RBAC changes	Server refreshes its RBAC cache	RBAC updates via Management API or dashboard	Server	No

Functional requirements

Generic

1. The agent must request the group configuration when executing the group assignment change command.
2. The agent must hot-apply the configuration after receiving it.
3. The server must provide a new endpoint for requesting the configuration of a specific group.
4. All existing implementation related to the agent reboot command must be removed.
5. A configuration change that implies disabling specific content (e.g., SCA, Active response) must cause the agent to remove such content.

Group assignment change

6. The group content is attached to the command sent to the agent to prevent a unnecessary flows.

flowchart LR
    subgraph Backend["Server"]
        direction TB
        ManagementAPI("Management API")
        Server("Server")
        CommsAPI("Comms API")
        ManagementAPI --> Server
    end

    Agent("Agent")

    subgraph IndexSystem["Indexer"]
        direction TB
        TaskManager("Task manager")
        CommandsStream("Commands stream")
        CommandManager("Command manager")
        AgentsIndex("Agents index")
    end

    Dashboard("Dashboard")

    Dashboard -- "1-Generate task" --> TaskManager
    ManagementAPI -- "1- Generate task" --> TaskManager

    TaskManager -- "2- Write command" --> CommandsStream

    CommandManager -- "3- Pull command" --> CommandsStream
    CommandManager -- "4- Change group in agent index" --> AgentsIndex
    CommandManager -- "5- Write child command \n    & update first status command" --> CommandsStream
    
    Server -- "6- Command pulling" --> CommandsStream
    Server -- "7- Redirect command" --> CommsAPI
    CommsAPI -- "8- Propagate command" --> Agent
    Agent -- "9- Command result" --> CommsAPI
    CommsAPI -- "10- Command result" --> CommandsStream

Loading

Group configuration change

7. The group content is attached to the command sent to the agent to prevent a unnecessary flows.

flowchart LR
    subgraph Backend["Server"]
        direction TB
        ManagementAPI("Management API")
        Server("Server")
        CommsAPI("Comms API")
        ManagementAPI --> Server
    end

    Agent("Agent")

    subgraph IndexSystem["Indexer"]
        direction TB
        TaskManager("Task manager")
        CommandsStream("Commands stream")
        CommandManager("Command manager")
        GroupsIndex("Groups index")
    end

    Dashboard("Dashboard")

    Dashboard -- "1-Generate task" --> TaskManager
    ManagementAPI -- "1- Generate task" --> TaskManager

    TaskManager -- "2- Write command" --> CommandsStream

    CommandManager -- "3- Pull command" --> CommandsStream
    CommandManager -- "4- Change group content" --> GroupsIndex
    CommandManager -- "5- Write child command \n    & update first status command" --> CommandsStream
    
    Server -- "6- Command pulling" --> CommandsStream
    Server -- "7- Redirect command" --> CommsAPI
    CommsAPI -- "8- Propagate command" --> Agent
    Agent -- "9- Command result" --> CommsAPI
    CommsAPI -- "10- Command result" --> CommandsStream

Loading

Agent remote upgrade

flowchart LR
    subgraph Backend["Server"]
        direction TB
        ManagementAPI("Management API")
        Server("Server")
        CommsAPI("Comms API")
        ManagementAPI --> Server
    end

    Agent("Agent")

    subgraph IndexSystem["Indexer"]
        direction TB
        TaskManager("Task manager")
        CommandsStream("Commands stream")
    end

    Dashboard("Dashboard")

    Dashboard -- "1-Generate task" --> TaskManager
    ManagementAPI -- "1- Generate task" --> TaskManager

    TaskManager -- "2- Write command" --> CommandsStream
  
    Server -- "3- Command pulling" --> CommandsStream
    Server -- "4- Redirect command" --> CommsAPI
    CommsAPI -- "5- Propagate command" --> Agent
    Agent -- "6- Download package" --> CommsAPI
    Agent -- "7- Update agent" --> Agent
    Agent -- "8- Command result" --> CommsAPI
    CommsAPI -- "9- Command result" --> CommandsStream

Loading

New agent package

flowchart LR
    subgraph Backend["Server"]
        direction TB
        Server("Server")
    end

    subgraph IndexSystem["Indexer"]
        direction TB
        TaskManager("Task manager")
        CommandsStream("Commands stream")
        ContentManager("Content manager")
    end

    CTI("CTI")

    ContentManager -- "1-Download package" --> CTI
    ContentManager -- "2-Generate task" --> TaskManager

    TaskManager -- "3- Write command" --> CommandsStream
  
    Server -- "4- Command pulling" --> CommandsStream
    Server -- "5- Get package" --> ContentManager
    Server -- "6- Command result" --> CommandsStream

Loading

Agent name change

flowchart LR
    subgraph Backend["Server"]
        direction TB
        ManagementAPI("Management API")
        Server("Server")
        CommsAPI("Comms API")
        ManagementAPI --> Server
    end

    Agent("Agent")

    subgraph IndexSystem["Indexer"]
        direction TB
        TaskManager("Task manager")
        CommandsStream("Commands stream")
        CommandManager("Command manager")
        AgentsIndex("Agents index")
    end

    Dashboard("Dashboard")

    Dashboard -- "1-Generate task" --> TaskManager
    ManagementAPI -- "1- Generate task" --> TaskManager

    TaskManager -- "2- Write command" --> CommandsStream

    CommandManager -- "3- Pull command" --> CommandsStream
    CommandManager -- "4- Change name of agent" --> AgentsIndex
    CommandManager -- "5- Write child command \n    & update first status command" --> CommandsStream
    
    Server -- "6- Command pulling" --> CommandsStream
    Server -- "7- Redirect command" --> CommsAPI
    CommsAPI -- "8- Propagate command" --> Agent
    Agent -- "9- Command result" --> CommsAPI
    CommsAPI -- "10- Command result" --> CommandsStream

Loading

SCA Policy change

flowchart LR
    subgraph Backend["Server"]
        direction TB
        ManagementAPI("Management API")
        Server("Server")
        CommsAPI("Comms API")
        ManagementAPI --> Server
    end

    Agent("Agent")

    subgraph IndexSystem["Indexer"]
        direction TB
        TaskManager("Task manager")
        CommandsStream("Commands stream")
        CommandManager("Command manager")
        SCAIndex("SCA index")
        ContentManager("Content manager")
    end

    Dashboard("Dashboard")
    Dashboard -- "1-Generate task" --> TaskManager
    ManagementAPI -- "1- Generate task" --> TaskManager

    TaskManager -- "2- Write command" --> CommandsStream

    CommandManager -- "3- Pull command" --> CommandsStream
    CommandManager -- "4- Update SCA policy" --> SCAIndex
    CommandManager -- "5- Write child command \n    & update first status command" --> CommandsStream
    
    Server -- "6- Command pulling" --> CommandsStream
    Server -- "7- Redirect command" --> CommsAPI
    CommsAPI -- "8- Propagate command" --> Agent
    Agent -- "9- Command result" --> CommsAPI
    CommsAPI -- "10- Command result" --> CommandsStream

Loading

SCA Content update

flowchart LR
    subgraph Backend["Server"]
        direction TB
        Server("Server")
        CommsAPI("Comms API")
    end

    Agent("Agent")

    subgraph IndexSystem["Indexer"]
        direction TB
        TaskManager("Task manager")
        CommandsStream("Commands stream")
        SCAIndex("SCA index")
        ContentManager("Content manager")
    end

    CTI("CTI")

    ContentManager -- "0- Download SCA content" --> CTI
    ContentManager -- "1- Update SCA index" --> SCAIndex
    ContentManager -- "2-Generate task" --> TaskManager
    TaskManager -- "3- Write N(agents) command" --> CommandsStream
    
    Server -- "4- Command pulling" --> CommandsStream
    Server -- "5- Redirect command" --> CommsAPI
    CommsAPI -- "6- Propagate command" --> Agent
    Agent -- "7- Command result" --> CommsAPI
    CommsAPI -- "8- Command result" --> CommandsStream

Loading

Vulnerability DB rescan

flowchart LR
    subgraph Backend["Server"]
        direction TB
        Server("Server")
        CommsAPI("Comms API")
    end

    Agent("Agent")

    subgraph IndexSystem["Indexer"]
        direction TB
        TaskManager("Task manager")
        CommandsStream("Commands stream")
        CVEIndex("CVE index")
        ContentManager("Content manager")
    end

    CTI("CTI")

    ContentManager -- "0- Download CVE content" --> CTI
    ContentManager -- "1- Update CVE index" --> CVEIndex
    ContentManager -- "2-Generate task" --> TaskManager
    TaskManager -- "3- Write N(agents) command" --> CommandsStream
    
    Server -- "4- Command pulling" --> CommandsStream
    Server -- "5- Redirect command" --> CommsAPI
    CommsAPI -- "6- Propagate command" --> Agent
    Agent -- "7- Command result" --> CommsAPI
    CommsAPI -- "8- Command result" --> CommandsStream

Loading

CVE content update

flowchart LR
    subgraph Backend["Server"]
        direction TB
        Server("Server")
        Engine("Engine")
    end

    subgraph IndexSystem["Indexer"]
        direction TB
        TaskManager("Task manager")
        CommandsStream("Commands stream")
        CVEIndex("CVE index")
        ContentManager("Content manager")

    end

    CTI("CTI")

    ContentManager -- "0- Download CVE content" --> CTI
    ContentManager -- "1- Update CVE index" --> CVEIndex
    ContentManager -- "2-Generate task" --> TaskManager
    TaskManager -- "3- Write command" --> CommandsStream
    
    Server -- "4- Command pulling" --> CommandsStream
    Server -- "5- Redirect command" --> Engine

Loading

Active response configuration

flowchart LR
    subgraph Backend["Server"]
        direction TB
        ManagementAPI("Management API")
        Server("Server")
        CommsAPI("Comms API")
    end

    Agent("Agent")

    subgraph IndexSystem["Indexer"]
        direction TB
        TaskManagerAttended("Task manager attended")
        TaskManagerUnattended("Task manager unattended")
        CommandsStream("Commands stream")
        ARIndex("Active Response index")
        ContentManager("Content manager")
    end
    CTI("CTI")
    Dashboard("Dashboard")

    Dashboard -- "1A-Generate task" --> TaskManagerAttended
    ManagementAPI -- "1A- Generate task" --> TaskManagerAttended
    TaskManagerAttended -- "2A- Update AR policy" --> ARIndex
    TaskManagerAttended -- "3A- Write command" --> CommandsStream


    ContentManager -- "0B-Download content" --> CTI
    ContentManager -- "1B- Update AR policy" --> ARIndex
    ContentManager -- "2B- Generate task" --> TaskManagerUnattended
    TaskManagerUnattended -- "3B- Write command" --> CommandsStream  

    Server -- "4- Command pulling" --> CommandsStream
    Server -- "5- Redirect command" --> CommsAPI
    CommsAPI -- "6- Propagate command" --> Agent
    Agent -- "7- Command result" --> CommsAPI
    CommsAPI -- "8- Command result" --> CommandsStream

Loading

Active response execution

flowchart LR
    subgraph Backend["Server"]
        direction TB
        Server("Server")
        CommsAPI("Comms API")
        Engine("Engine")
    end

    Agent("Agent")

    subgraph IndexSystem["Indexer"]
        direction TB
        TaskManager("Task manager")
        CommandsStream("Commands stream")
        CommandManager("Command manager")
        AgentsIndex("Agents index")
    end

    Engine -- "1-Generate Active response task" --> TaskManager

    TaskManager -- "2- Write command" --> CommandsStream

    CommandManager -- "3- Pull command" --> CommandsStream
    CommandManager -- "4- Read agents and groups" --> AgentsIndex
    CommandManager -- "5- Write child commands   & update first status command" --> CommandsStream
    
    Server -- "6- Command pulling" --> CommandsStream
    Server -- "7- Redirect command" --> CommsAPI
    CommsAPI -- "8- Propagate command" --> Agent
    Agent -- "9- Command result" --> CommsAPI
    CommsAPI -- "10- Command result" --> CommandsStream

Loading

Ruleset update

flowchart LR
    subgraph Backend["Server"]
        direction TB
        Server("Server")
        Engine("Engine")
    end

    subgraph IndexSystem["Indexer"]
        direction TB
        TaskManager("Task manager")
        CommandsStream("Commands stream")
        RulesetIndex("Ruleset index")
        ContentManager("Content manager")

    end

    CTI("CTI")

    ContentManager -- "0- Download ruleset content" --> CTI
    ContentManager -- "1- Update ruleset index" --> RulesetIndex
    ContentManager -- "2-Generate task" --> TaskManager
    TaskManager -- "3- Write command" --> CommandsStream
    
    Server -- "4- Command pulling" --> CommandsStream
    Server -- "5- Redirect command" --> Engine

Loading

Ruleset update

flowchart LR
    subgraph Backend["Server"]
        direction TB
        ManagementAPI("Management API")
        Server("Server")
        Engine("Engine")
    end

    subgraph IndexSystem["Indexer"]
        direction TB
        TaskManagerAttended("Task manager attended")
        TaskManagerUnattended("Task manager unattended")
        CommandsStream("Commands stream")
        RulesetIndex("Ruleset index")
        ContentManager("Content manager")
    end
    CTI("CTI")
    Dashboard("Dashboard")


    Dashboard -- "1A-Generate task" --> TaskManagerAttended
    ManagementAPI -- "1A- Generate task" --> TaskManagerAttended
    TaskManagerAttended -- "2A- Update ruleset policy" --> RulesetIndex
    TaskManagerAttended -- "3A- Write command" --> CommandsStream

    ContentManager -- "0B-Download ruleset content" --> CTI
    ContentManager -- "1B- Update ruleset policy" --> RulesetIndex
    ContentManager -- "2B- Generate task" --> TaskManagerUnattended
    TaskManagerUnattended -- "3B- Write command" --> CommandsStream
    
    Server -- "4- Command pulling" --> CommandsStream
    Server -- "5- Redirect command" --> Engine
    Engine -- "6- Command result" --> CommandsStream

Loading

Server configuration

flowchart LR
    subgraph Backend["Server"]
        direction TB
        ManagementAPI("Management API")
        Server("Server")
    end

    subgraph IndexSystem["Indexer"]
        direction TB
        TaskManagerAttended("Task manager attended")
        CommandsStream("Commands stream")
        ServerConfigurationIndex("Server configuration index")
    end
    Dashboard("Dashboard")
    Dashboard -- "1-Generate task" --> TaskManagerAttended
    ManagementAPI -- "1- Generate task" --> TaskManagerAttended
    TaskManagerAttended -- "2- Update server configuration" --> ServerConfigurationIndex
    TaskManagerAttended -- "3- Write command" -->CommandsStream

    Server -- "4- Command pulling" --> CommandsStream
    Server -- "5- Notify observers" -->Server

Loading

Implementation restrictions

Plan

Spike

Development

Requirements Traceability Matrix

• Mapping of requirements to implementation and test cases.

Notes

• Agent initialization ( ask for current group configuration and assignment, etc)