parse_description in windows_eventlog2 is broken

[ichintu]

I think parse_description true is broken. My config:

<source>
  @type windows_eventlog2
  @id windows_eventlog2
  channels Windows PowerShell,Microsoft-Windows-Sysmon/Operational,Security
  tag winevt.raw
  render_as_xml true
  parse_description true
  read_existing_events false
  <storage>
    @type local
    persistent true
    path C:\opt\td-agent\winlog.json
  </storage>
</source>

<match winevt.raw>
  @type file
  path C:\logs_out\fluentd\winlog
  # compress gzip
  <format>
    @type json
  </format>
  <buffer>
    timekey 5m
    timekey_use_utc true
    timekey_wait 1m
  </buffer>
</match>

What i am expecting is something that looks like this:

{
  "ProviderName": "Microsoft-Windows-Security-Auditing",
  "ProviderGUID": "{D441060A-9695-472B-90BC-24DCA9D503A4}",
  "EventID": "4798",
  "Qualifiers": "",
  "Level": "0",
  "Task": "13824",
  "Opcode": "0",
  "Keywords": "0x8020000000000000",
  "TimeCreated": "2019-06-19T03:10:01.982940200Z",
  "EventRecordID": "87028",
  "ActivityID": "",
  "RelatedActivityID": "{2599DE71-2F70-44AD-9DC8-C5FF2AE8D1EF}",
  "ThreadID": "16888",
  "Channel": "Security",
  "Computer": "DESKTOP-TEST",
  "UserID": "",
  "Version": "0",
  "DescriptionTitle": "A user's local group membership was enumerated.",
  "subject.security_id": "S-X-Y-Z",
  "subject.account_name": "DESKTOP-TEST$",
  "subject.account_domain": "WORKGROUP",
  "subject.logon_id": "0x3e7",
  "user.security_id": "S-XXX-YYY-ZZZ",
  "user.account_name": "Administrator",
  "user.account_domain": "DESKTOP-TEST",
  "process_information.process_id": "0xbac",
  "process_information.process_name": "C:\\Windows\\System32\\svchost.exe"
}

However what I am getting is:

{
  "ProviderName": "Microsoft-Windows-Sysmon",
  "ProviderGUID": "{5770385f-c22a-43e0-bf4c-06f5698ffbd9}",
  "EventID": "1",
  "Qualifiers": "",
  "Level": "4",
  "Task": "1",
  "Opcode": "0",
  "Keywords": "0x8000000000000000",
  "TimeCreated": "2023-03-06T18:15:15.560133900Z",
  "EventRecordID": "7154",
  "ActivityID": "",
  "RelatedActivityID": "",
  "ProcessID": "2936",
  "ThreadID": "4880",
  "Channel": "Microsoft-Windows-Sysmon/Operational",
  "Computer": "W10-SANDY-001",
  "UserID": "S-1-5-18",
  "Version": "5",
  "DescriptionTitle": "Process Create:\r\nRuleName: \r\nUtcTime: 2023-03-06 18:15:15.558\r\nProcessGuid: {84faa657-2db3-6406-0000-0010e59d5202}\r\nProcessId: 1336\r\nImage: C:\\opt\\td-agent\\bin\\ruby.exe\r\nFileVersion: 2.7.6p219\r\nDescription: Ruby interpreter (CUI) 2.7.6p219 [x64-mingw32]\r\nProduct: Ruby interpreter 2.7.6p219 [x64-mingw32]\r\nCompany: http://www.ruby-lang.org/\r\nOriginalFileName: ruby.exe\r\nCommandLine: C:\\opt\\td-agent\\bin\\ruby.exe -Eascii-8bit:ascii-8bit -h\r\nCurrentDirectory: c:\\opt\\td-agent\\\r\nUser: W10-SANDY-001\\Johnny Douche\r\nLogonGuid: {84faa657-1673-6405-0000-0020b4830100}\r\nLogonId: 0x183B4\r\nTerminalSessionId: 1\r\nIntegrityLevel: High\r\nHashes: MD5=949C42C68A780AB3AB2EE4730B56230D,SHA256=4815AC3DA6906A52031D0A21314493F66635D717125D644CFB3784396838F7DC\r\nParentProcessGuid: {84faa657-2dae-6406-0000-0010bf854f02}\r\nParentProcessId: 5988\r\nParentImage: C:\\opt\\td-agent\\bin\\ruby.exe\r\nParentCommandLine: \"C:\\opt\\td-agent\\bin\\ruby.exe\"  \"C:\\opt\\td-agent\\bin\\fluentd\" -c etc\\td-agent\\td-agent.conf"
}

Am I doing something wrong or something broken with the parser?

[ichintu]

I wanted to share of what happens when i set parse_description false
so the configuration is definitely trying to parse but its not parsing properly.

{
  "ProviderName": "Microsoft-Windows-Sysmon",
  "ProviderGUID": "{5770385f-c22a-43e0-bf4c-06f5698ffbd9}",
  "EventID": "1",
  "Qualifiers": "",
  "Level": "4",
  "Task": "1",
  "Opcode": "0",
  "Keywords": "0x8000000000000000",
  "TimeCreated": "2023-03-06T17:32:19.497163500Z",
  "EventRecordID": "7137",
  "ActivityID": "",
  "RelatedActivityID": "",
  "ProcessID": "2936",
  "ThreadID": "4880",
  "Channel": "Microsoft-Windows-Sysmon/Operational",
  "Computer": "W10-SANDY-001",
  "UserID": "S-1-5-18",
  "Version": "5",
  "Description": "Process Create:\r\nRuleName: \r\nUtcTime: 2023-03-06 17:32:19.495\r\nProcessGuid: {84faa657-23a3-6406-0000-0010d577d201}\r\nProcessId: 7668\r\nImage: C:\\opt\\td-agent\\bin\\ruby.exe\r\nFileVersion: 2.7.6p219\r\nDescription: Ruby interpreter (CUI) 2.7.6p219 [x64-mingw32]\r\nProduct: Ruby interpreter 2.7.6p219 [x64-mingw32]\r\nCompany: http://www.ruby-lang.org/\r\nOriginalFileName: ruby.exe\r\nCommandLine: \"C:\\opt\\td-agent\\bin\\ruby.exe\"  \"C:\\opt\\td-agent\\bin\\fluentd\" -c etc\\td-agent\\td-agent.conf\r\nCurrentDirectory: c:\\opt\\td-agent\\\r\nUser: W10-SANDY-001\\Johnny Douche\r\nLogonGuid: {84faa657-1673-6405-0000-0020b4830100}\r\nLogonId: 0x183B4\r\nTerminalSessionId: 1\r\nIntegrityLevel: High\r\nHashes: MD5=949C42C68A780AB3AB2EE4730B56230D,SHA256=4815AC3DA6906A52031D0A21314493F66635D717125D644CFB3784396838F7DC\r\nParentProcessGuid: {84faa657-02a1-6406-0000-0010ffd05401}\r\nParentProcessId: 1204\r\nParentImage: C:\\Windows\\SysWOW64\\cmd.exe\r\nParentCommandLine: \"C:\\Windows\\SysWOW64\\cmd.exe\" /k \"C:\\opt\\td-agent\\td-agent-prompt.bat\"",
  "EventData": [
    "",
    "2023-03-06 17:32:19.495",
    "{84FAA657-23A3-6406-0000-0010D577D201}",
    "7668",
    "C:\\opt\\td-agent\\bin\\ruby.exe",
    "2.7.6p219",
    "Ruby interpreter (CUI) 2.7.6p219 [x64-mingw32]",
    "Ruby interpreter 2.7.6p219 [x64-mingw32]",
    "http://www.ruby-lang.org/",
    "ruby.exe",
    "\"C:\\opt\\td-agent\\bin\\ruby.exe\"  \"C:\\opt\\td-agent\\bin\\fluentd\" -c etc\\td-agent\\td-agent.conf",
    "c:\\opt\\td-agent\\",
    "W10-SANDY-001\\Johnny Douche",
    "{84FAA657-1673-6405-0000-0020B4830100}",
    "0x00000000000183b4",
    "1",
    "High",
    "MD5=949C42C68A780AB3AB2EE4730B56230D,SHA256=4815AC3DA6906A52031D0A21314493F66635D717125D644CFB3784396838F7DC",
    "{84FAA657-02A1-6406-0000-0010FFD05401}",
    "1204",
    "C:\\Windows\\SysWOW64\\cmd.exe",
    "\"C:\\Windows\\SysWOW64\\cmd.exe\" /k \"C:\\opt\\td-agent\\td-agent-prompt.bat\""
  ]
}

[AmeOnCoffee]

Same issue here, using Fluentd and Loki. doesnt seem to be fixed