[Fleet][Agentless] Improve agentless APIs

[nchaulet]

Description

Currently to create an agentless integration, a user need to create both an agent policy and a package policy, this is not ideal, and could be confusing for the user. (The same happen for the deletion where the user need to delete the package policy, than the agent policy).

There is a lot of orphaned agent policies, as we hide them from the UI.

Proposed solution

We could use introduce a new dedicated API for agentless policies

Creation

If we get an agentless request without an agent policy id we could create one

POST /api/fleet/agentless_policies
{
  ...SimplifiedPackagePolicy,
  ...cloud_connector
}

Instead of

POST /api/fleet/agent_policies
{
 "supports_agentless": true,
}
POST /api/fleet/package_policies
{
 "policy_ids": [<previous_id>]
 "supports_agentless": true,
}

Deletion

Same on deletion we could delete both the package policy and agent policy

DELETE /api/fleet/agentless_policies/{id} => will delete both agent and package policy and 

Instead of

POST /api/fleet/package_policy/delete

DELETE /api/fleet/agent_policies/{id}

Notes

• We considered using only the package policy API, but that API is already complex, and it seems more future proof to have a dedicated API.
• This is kind of breaking, but agentless is not GA.

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[cmacknz]

Is it worth introducing a new agentless_policy API, that hides the underlying composition of agent policy and agentless policy, so that we have freedom to do whatever is needed behind the scenes now and later?

[nchaulet]

This could be another option yes, we tried to avoid this to avoid having another APIs to maintains, but it seems it could make sense to have a dedicated API, if we go that way we should probably deprecate/than block the creation of agentless policies through the existing APIs to avoid introducing another way to create a policy

[cmacknz]

Yes I was thinking you'd deprecate and then block use of the supports_agentless API.

I suspect right now use of supports_agentless is creating another API, it's just an implicit one across the various existing API calls.

[nchaulet]

I suspect right now use of supports_agentless is creating another API, it's just an implicit one across the various existing API calls.

Yes it's an implicit and complex to use, it probably make sense to have a dedicated API, more over as we are adding things like cloud connectors, it will allow to not add that complexity to our already too complex package policy one.

[nchaulet]

An update on how I plan to land this one as we discovered some work on the cloud connectors API are done in the same time:

• Add Create/Delete agentless policies API behind a feature flag without cloud connector supports related PR
• security team will adapt their work https://github.com/elastic/security-team/issues/14277 to use the new agentless API instead of a new internal one

Once code merged:

• we can communicate to all team using Agentless that we have a new API and UI will use it, ask for some testing with instruction
• Turn the feature flag on to use the new API
• remove deprecated Agentless policy creation through package/agent policy