Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for escaped macros #379

Open
NassimBtk opened this issue Sep 2, 2024 · 1 comment
Open

Add support for escaped macros #379

NassimBtk opened this issue Sep 2, 2024 · 1 comment
Assignees

Comments

@NassimBtk
Copy link
Member

NassimBtk commented Sep 2, 2024

Description:

We need to extend our macro replacement functionality to support various escape formats. The goal is to allow the engine to automatically replace macros with their corresponding inputs in different escaped formats.

Current Functionality:
Currently, our code supports the replacement of macros with their corresponding inputs. For example, the macro %{USERNAME} is replaced with the actual username from the user's configuration.

New Functionality:
We want to introduce a new set of macros that apply different types of escaping to the input values. The new macros should follow the pattern %{SOURCE_ESC_FORMAT}.

Supported Escape Formats:

  • Regex: %{SOURCE_ESC_REGEX}
  • URL: %{SOURCE_ESC_URL}
  • JSON: %{SOURCE_ESC_JSON}
  • XML: %{SOURCE_ESC_XML}
  • Windows Shell: %{SOURCE_ESC_WINDOWS} and %{SOURCE_ESC_CMD}
  • Linux Shell: %{SOURCE_ESC_LINUX} and %{SOURCE_ESC_BASH}
  • SQL: %{SOURCE_ESC_SQL}

Supported Input Sources (SOURCE)

  • USERNAME
  • HOSTNAME
  • PASSWORD
  • AUTHENTICATIONTOKEN
  • BASIC_AUTH_BASE64
  • SHA256_AUTH
  • PASSWORD_BASE64

Example Usage:

ℹ️ We must ensure that all possible scenarios are accounted for in each escape format. These are just a few examples:

  • JSON Escape:
    %{USERNAME_ESC_JSON}: Na"ssimNa\"ssim
    Explanation: In JSON, the double quote character " must be escaped with a backslash, so Na"ssim becomes Na\"ssim.

  • URL Encoding:
    %{USERNAME_ESC_URL}: Na ssimNa%20ssim
    Explanation: In URLs, spaces are encoded as %20, so Na ssim becomes Na%20ssim.

  • Regex Escape:
    %{USERNAME_ESC_REGEX}: Na.ssimNa\.ssim
    Explanation: In regular expressions, the dot . is a special character and must be escaped with a backslash, so Na.ssim becomes Na\.ssim.

  • XML Escape:
    %{USERNAME_ESC_XML}: Na<ssim & "N"Na&lt;ssim &amp; &quot;N&quot;
    Explanation: In XML, <, &, and " are special characters that must be escaped as &lt;, &amp;, and &quot;, respectively.

  • Windows CMD Escape:
    %{USERNAME_ESC_WINDOWS}: Na&ssimNa^&ssim
    Explanation: In Windows CMD, the ampersand & is a special character that must be escaped with a caret ^, so Na&ssim becomes Na^&ssim.

  • Linux Bash Escape:
    %{USERNAME_ESC_BASH}: Na'ssimNa\'ssim
    Explanation: In Linux Bash, single quotes inside a quoted string must be escaped with a backslash, so Na'ssim becomes Na\'ssim.

  • SQL Escape:
    %{USERNAME_ESC_SQL}: Na'ssimNa''ssim
    Explanation: In SQL, single quotes are escaped by doubling them, so Na'ssim becomes Na''ssim.

Additional Notes:

  • Ensure that the implementation does not break existing functionality.
  • Consider the security implications of these escapes, particularly in Shell and SQL escapes.
  • Remove %{USERNAME_JSON} and %{PASSWORD_JSON} and replace them with %{USERNAME_ESC_JSON} and %{PASSWORD_ESC_JSON}

Specification

The possible escape characters for all the source formats are as follow: (@bertysentry Can you please review the escape sequences below ?)

  • URL
Character Escape Sequence Context
(Space) %20 Represents a space
! %21 Represents the exclamation mark
" %22 Represents a double quote
# %23 Represents the hash or pound symbol
$ %24 Represents the dollar sign
% %25 Represents the percent sign
& %26 Represents the ampersand
' %27 Represents the single quote or apostrophe
( %28 Represents the opening parenthesis
) %29 Represents the closing parenthesis
* %2A Represents the asterisk
+ %2B Represents the plus sign
, %2C Represents the comma
/ %2F Represents the forward slash
: %3A Represents the colon
; %3B Represents the semicolon
= %3D Represents the equals sign
? %3F Represents the question mark
@ %40 Represents the at symbol (@)
[ %5B Represents the opening square bracket
] %5D Represents the closing square bracket
^ %5E Represents the caret symbol
` %60 Represents the backtick or grave accent
{ %7B Represents the opening curly brace
} %7D Represents the closing curly brace
| %7C Represents the pipe
~ %7E Represents the tilde
  • Regex
Character Escape Sequence Context
. \. Represents any single character (wildcard)
^ \^ Represents the start of a line
$ \$ Represents the end of a line
* \* Represents zero or more occurrences
+ \+ Represents one or more occurrences
? \? Represents zero or one occurrence (optional)
{ \{ Represents the start of a quantifier
} \} Represents the end of a quantifier
[ \[ Represents the start of a character class
] \] Represents the end of a character class
( \( Represents the start of a capture group
) \) Represents the end of a capture group
| \| Represents alternation (logical OR)
\ \\ Represents a literal backslash
/ \/ Represents a literal forward slash
  • XML
Character Escape Sequence Context
< &lt; Represents the less than symbol in XML tags
> &gt; Represents the greater than symbol in XML tags
& &amp; Represents the ampersand (used for entities)
" &quot; Represents double quotes in attribute values
' &apos; Represents single quotes in attribute values
  • Windows CMD
Character Escape Sequence Context
& ^& Used to escape the ampersand, which is used for command chaining (e.g., && for "AND") in CMD
| ^| Used to escape the pipe, which is used for piping the output of one command to another
< ^< Used to escape the less than symbol, which is used for input redirection in CMD
> ^> Used to escape the greater than symbol, which is used for output redirection in CMD
^ ^^ Used to escape the caret symbol itself, which is the escape character in CMD
% ^% Used to escape the percent symbol, which is used for variable expansion in CMD
( ^( Used to escape the opening parenthesis, which is used to group commands
) ^) Used to escape the closing parenthesis, which is used to close grouped commands
" ^" Used to escape the double quote, which is used for enclosing strings or paths in CMD
  • Linux Bash
Character Escape Sequence Context
' \' Used to escape the single quote inside single-quoted strings
" \" Used to escape the double quote inside double-quoted strings
\ \\ Used to escape the backslash itself
$ \$ Used to escape the dollar sign, which is used for variable expansion
! \! Used to escape the exclamation mark (for history expansion)
* \* Used to escape the asterisk, which is used as a wildcard
? \? Used to escape the question mark, which is used as a wildcard
[ \[ Used to escape the opening square bracket, which is part of character classes
] \] Used to escape the closing square bracket, which is part of character classes
( \( Used to escape the opening parenthesis, which is used for subshells or grouping
) \) Used to escape the closing parenthesis, which is used for subshells or grouping
{ \{ Used to escape the opening curly brace, which is used in brace expansion
} \} Used to escape the closing curly brace, which is used in brace expansion
| \| Used to escape the pipe, which is used to pipe commands
& \& Used to escape the ampersand, which is used to run commands in the background
< \< Used to escape the less than symbol, which is used for input redirection
> \> Used to escape the greater than symbol, which is used for output redirection
~ \~ Used to escape the tilde, which is used for home directory expansion
  • SQL queries
Character Escape Sequence Context
' '' Used to escape single quotes inside string literals in SQL
" \" (if applicable) Used to escape double quotes in some SQL databases (not all databases require this)
\ \\ (if applicable) Used to escape the backslash in certain SQL databases (depends on the SQL dialect)
% % Represents the percent symbol used in SQL for pattern matching (wildcards in LIKE)
_ _ Represents the underscore symbol used in SQL for pattern matching (wildcards in LIKE)
Newline (\n) \\n Used to represent a newline in string literals or text fields in SQL
Carriage Return (\r) \\r Used to represent a carriage return in string literals or text fields in SQL
Tab (\t) \\t Used to represent a tab in string literals or text fields in SQL
@NassimBtk NassimBtk changed the title Add Support for Escaped Macros Add support for escaped macros Sep 2, 2024
@bertysentry
Copy link
Member

For URL and Regex escaping, we can use native Java methods.

For Windows CMD, I think there is a problem in the table (2nd row), where we should see | must be replaced with ^|.

This will require some manual testing, to make sure the behavior is as intended, especially for Windows and Linux commands, before implementing unit tests.

Also, the source can be an actual source reference (${source::monitors.enclosure.discovery.source(1)_ESC_JSON}).

MedMaalej added a commit that referenced this issue Sep 27, 2024
* Delete HttpMacrosUpdator
* Add MacrosUpdater
* Create new macros
* Add unit tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants