[AI-5153] DDS: Mac Audit Logs Integration v1.0.0

[tirthrajchaudhari-crest]

What does this PR do?

This is an initial release PR of Mac Audit Logs integration including all the required assets. This is agent based integration.

Additional Notes

• OOTB detection rules JSON would be shared separately with the required teams as a part of separate repository .
• Since during the standard attribute remapping we are not preserving the source attributes as per suggested best practices, it would result in filters using these standard attributes populating the values of other integrations as well as per current datadog behaviour.

Review checklist (to be filled by reviewers)

• Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
• Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
• If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

[codecov]

Codecov Report

Attention: Patch coverage is 60.55046% with 43 lines in your changes missing coverage. Please review.

Project coverage is 88.65%. Comparing base (2d13353) to head (39b8924).
Report is 67 commits behind head on master.

Additional details and impacted files

Flag	Coverage Δ
activemq	?
cassandra	?
hive	?
hivemq	?
hudi	?
ignite	?
jboss_wildfly	?
kafka	?
mac_audit_logs	60.55% <60.55%> (?)
presto	?
solr	?

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[drichards-87]

Created DOCS-10537 for Docs Team editorial review.

[nubtron]

Hi @tirthrajchaudhari-crest, just a quick update - I’m working on the review and discussing some aspects with the team, I'll share as soon as it's ready!

[nubtron]

Setting the time zone in the backend is inconvenient for the customer, and it would cause complications if there are clients in different time zones. Could you please modify the integration so that timezones are handled automatically?

You can read the time zone with, for eg., the date "+%z" command. Then you can
convert the timestamp and send the log data with the send_log function:

from datadog_checks.base import AgentCheck
from datadog_checks.base.utils.time import get_timestamp

class HelloCheck(AgentCheck):
    def check(self, instance):
        data = dict()
        data['timestamp'] = get_timestamp()
        data['message'] = "this is a custom log message!"
        data['ddtags'] = "env:dev,bar:foo"
        self.send_log(data)

You also need to change the type entry in logs section to integration, for eg:

logs:
  - type: integration
    source: my_source
    service: my_service

Octopus Deploy and SAP HANA have usages of send_log that can be used as examples.

You can parse the XML in the logs with Python code. You should use the lxml library for that, instead of the built-in Python xml library, which has security issues. The IBM WAS integration has an example of lxml usage.

[estherk15]

Suggested change

	This integration collects mac audit logs and sends them to Datadog for analysis, providing visual insights through out-of-the-box dashboards and Log Explorer. It also helps monitor and respond to security threats with ready-to-use Cloud SIEM detection rules.
	This integration collects Mac audit logs and sends them to Datadog for analysis, providing visual insights through out-of-the-box dashboards and the Log Explorer. It also helps monitor and respond to security threats with ready-to-use Cloud SIEM detection rules.

[tirthrajchaudhari-crest]

Done

[estherk15]

Suggested change

	**Note**: The following steps are required for the Mac version >=14.

[tirthrajchaudhari-crest]

Done

[estherk15]

Recommend moving this note to the beginning of the configuration.

Suggested change

**Note**: The above steps are needed for the mac version >=14.

[tirthrajchaudhari-crest]

Done

[estherk15]

Suggested change

	3. Hover over the Mac Audit Logs pipeline and click on the **clone** button. This will create an editable clone of the Mac Audit Logs pipeline.
	3. Hover over the Mac Audit Logs pipeline and click **clone**. This creates an editable clone of the Mac Audit Logs pipeline.

[tirthrajchaudhari-crest]

Timezone support has been added in code itself. Hence, Removed this section.

[estherk15]

Suggested change

	4. Edit the Grok Parser using the below steps:
	4. Edit the Grok Parser by following these steps:

[tirthrajchaudhari-crest]

Timezone support has been added in code itself. Hence, Removed this section.

[estherk15]

Recommend consistent capitalization of Mac, unless this is all caps for a reason.

Suggested change

	- Change the string `UTC` to the [TZ identifier][9] of the time zone of your MAC machine. For example, if your timezone is IST, you would change the value to`Asia/Calcutta`.
	- Change the string `UTC` to the [TZ identifier][9] of the time zone of your Mac machine. For example, if your timezone is IST, you would change the value to`Asia/Calcutta`.

[tirthrajchaudhari-crest]

Timezone support has been added in code itself. Hence, Removed this section.

[estherk15]

Suggested change

	- Click the **update** button.
	- Click **Update**.

[estherk15]

To confirm, is the button lowercase?

[tirthrajchaudhari-crest]

Timezone support has been added in code itself. Hence, Removed this section.

[estherk15]

To confirm, did you intend to send the reader into the logs pipelines page or to a reference for TZ identifier? If you meant to link to the logs pipeline page, it seems redundant since the user is directed there in step 1.

Suggested change

	- Change the string `UTC` to the [TZ identifier][9] of the time zone of your MAC machine. For example, if your timezone is IST, you would change the value to`Asia/Calcutta`.
	- Change the string `UTC` to the TZ identifier of the time zone of your MAC machine. For example, if your timezone is IST, you would change the value to`Asia/Calcutta`.

[tirthrajchaudhari-crest]

Timezone support has been added in code itself. Hence, Removed this section.

[tirthrajchaudhari-crest]

self.send_log(data)

Hey @nubtron , Thank you for the feedback. We are currently discussing this internally and will get back to you soon.​

[tirthrajchaudhari-crest]

Hi @estherk15, I'll do the suggested changes for README along with this change.

[tirthrajchaudhari-crest]

Setting the time zone in the backend is inconvenient for the customer, and it would cause complications if there are clients in different time zones. Could you please modify the integration so that timezones are handled automatically?

You can read the time zone with, for eg., the date "+%z" command. Then you can convert the timestamp and send the log data with the send_log function:

from datadog_checks.base import AgentCheck
from datadog_checks.base.utils.time import get_timestamp

class HelloCheck(AgentCheck):
    def check(self, instance):
        data = dict()
        data['timestamp'] = get_timestamp()
        data['message'] = "this is a custom log message!"
        data['ddtags'] = "env:dev,bar:foo"
        self.send_log(data)

You also need to change the type entry in logs section to integration, for eg:

logs:
  - type: integration
    source: my_source
    service: my_service

Octopus Deploy and SAP HANA have usages of send_log that can be used as examples.

You can parse the XML in the logs with Python code. You should use the lxml library for that, instead of the built-in Python xml library, which has security issues. The IBM WAS integration has an example of lxml usage.

Hey @nubtron, We have made the necessary changes as per your recommendations.

[nubtron]

Hi @tirthrajchaudhari-crest, there is an issue with how the main check function is blocked by the praudit process, I'm looking into what could be a good alternative.