[Logs] Document the central logs source setting

[mdbirnstiehl]

A new central setting as described in https://github.com/elastic/logs-dev/issues/138 should be introduced. Once in place, it can be used in progressively more places throughout the observability solution.

Project Link

[mdbirnstiehl]

@flash1293 @Kerry350 I was hoping I could get some input from you on this issue and on what needs to be documented. @gbamparop mentioned that it would be a good candidate for 8.16 docs, but I don't have much background on this one so I would appreciate your help filling in the gaps here.

[Kerry350]

@mdbirnstiehl No problem. I'll write up some bullet points tomorrow on what this affects and how 👍

[Kerry350]

@mdbirnstiehl From a high level what we've done is added a new advanced setting (called Log sources) which now influences how multiple Observability solutions access logs data (rather than having disjointed methods - hardcoded indices lists etc). The hope is that this brings consistency to data access and less surprises for users (https://github.com/elastic/logs-dev/issues/138).

Right now these changes affect the following UIs:

• Logs UI settings:

The setting is now the default option associated with the logs UI settings. This also means the setting still influences the indices queried in log threshold rules unless the user has manually changed to "indices" or "data view". Those two options are now deprecated. (https://github.com/elastic/logs-dev/issues/170)

• For the two deprecated options above a migration assistant option was added, with a "quick resolve" option, to help migrate users to the advanced setting option with their Logs UI settings (Provide help to migrate logs settings saved object kibana#186824):

• Both the log categories and log anomalies ML jobs now read their indices from this setting (under choose indices) (https://github.com/elastic/logs-dev/issues/168)

• In Logs Explorer this is now the default data source (e.g. "All logs") (https://github.com/elastic/logs-dev/issues/169):

• The Observability AI Assistant specific advanced setting has been removed in favour of, and consolidation with, this setting for choosing where to access logs data (https://github.com/elastic/logs-dev/issues/167)

• In Discover this setting is now used as part of the context resolution process to determine if we are viewing logs data (https://github.com/elastic/logs-dev/issues/171)