Skip to content

Commit

Permalink
Remove deprecated model elements in 3x (#4078)
Browse files Browse the repository at this point in the history
  • Loading branch information
Cisphyx authored Jan 20, 2025
1 parent 14449ba commit b573e71
Show file tree
Hide file tree
Showing 44 changed files with 129 additions and 1,931 deletions.
9 changes: 5 additions & 4 deletions docs/synapse/adminguide.rstorm
Original file line number Diff line number Diff line change
Expand Up @@ -1487,13 +1487,14 @@ be used to show the current lock status of all deprecated model elements.

**Examples:**

Lock the ``ps:person:img`` property:
Lock the ``inet:fqdn:_depr`` property:

.. storm-cli:: model.deprecated.lock ps:person:img
.. storm-pre:: $lib.model.ext.addFormProp(inet:fqdn, _depr, (str, ({})),({'deprecated': true}))
.. storm-cli:: model.deprecated.lock inet:fqdn:_depr

Unlock the ``ps:person:img`` property:
Unlock the ``inet:fqdn:_depr`` property:

.. storm-cli:: model.deprecated.lock --unlock ps:person:img
.. storm-cli:: model.deprecated.lock --unlock inet:fqdn:_depr

Lock all deprecated model elements:

Expand Down
2 changes: 1 addition & 1 deletion docs/synapse/userguides/data_model.rstorm
Original file line number Diff line number Diff line change
Expand Up @@ -504,7 +504,7 @@ In the above cases, generic forms may be used to capture data where a more speci
forms reside in within the ``meta:*`` forms in the data model.

The ``meta:rule`` form is an example of a generic form. Synapse includes more specific forms to represent common
detection logic such as antivirus (``it:av:sig`` and ``it:av:filehit``) or YARA rules (``it:app:yara:rule`` and
detection logic such as antivirus (``it:av:signame`` and ``it:av:scan:result``) or YARA rules (``it:app:yara:rule`` and
``it:app:yara:match``). Other technologies or organizations may have their own specific (and often "black box")
detection logic.

Expand Down
12 changes: 6 additions & 6 deletions docs/synapse/userguides/storm_adv_vars.rstorm
Original file line number Diff line number Diff line change
Expand Up @@ -323,16 +323,16 @@ You can assign an explicit, unchanging value to a variable.

*Example:*

- Tag ``file:bytes`` nodes that have a number of AV signature hits higher than a given threshold for review:
- Tag ``file:bytes`` nodes that have a number of malicious AV scan results higher than a given threshold for review:

.. storm-pre:: [file:bytes=sha256:0000746c55336cd8d34885545f9347d96607d0391fbd3e76dae7f2b3447775b4 it:av:filehit=(sha256:0000746c55336cd8d34885545f9347d96607d0391fbd3e76dae7f2b3447775b4, (0bfef0179bf358f3fe7bad67fa529c77, trojan.gen.2)) it:av:filehit=(sha256:0000746c55336cd8d34885545f9347d96607d0391fbd3e76dae7f2b3447775b4, (325cd5a01724fa0c63907eac044f4961, trojan.agent/gen-onlinegames)) it:av:filehit=(sha256:0000746c55336cd8d34885545f9347d96607d0391fbd3e76dae7f2b3447775b4, (ac8d9645c6cdf123683a73a02e231052, w32/imestartup.a.gen!eldorado))]
.. storm-pre:: [file:bytes=sha256:00007694135237ec8dc5234007043814608f239befdfc8a61b992e4d09e0cf3f it:av:filehit=(sha256:00007694135237ec8dc5234007043814608f239befdfc8a61b992e4d09e0cf3f, (be9793d772d23269ab0c165af819e74a, troj_gen.r002c0gkj17)) it:av:filehit=(sha256:00007694135237ec8dc5234007043814608f239befdfc8a61b992e4d09e0cf3f, (eef2ccb70945fb28a45c7f14f2a0f11d, malicious.1b8fb7)) it:av:filehit=(sha256:00007694135237ec8dc5234007043814608f239befdfc8a61b992e4d09e0cf3f, (ce4e34d2f9207095aa7351986bbad357, trojan-ddos.win32.stormattack.c)) it:av:filehit=(sha256:00007694135237ec8dc5234007043814608f239befdfc8a61b992e4d09e0cf3f, (ed344310e3203ec4348c4ee549a3b188, "trojan ( 00073eb11 )")) it:av:filehit=(sha256:00007694135237ec8dc5234007043814608f239befdfc8a61b992e4d09e0cf3f, (f5b5daeda10e487fccc07463d9df6b47, tool.stormattack.win32.10)) it:av:filehit=(sha256:00007694135237ec8dc5234007043814608f239befdfc8a61b992e4d09e0cf3f, (a0f25a5ba637d5c8e7c42911c4336085, trojan/w32.agent.61440.eii))]
.. storm-cli:: $threshold=5 file:bytes +{ -> it:av:filehit } >= $threshold [ +#review ]
.. storm-pre:: [file:bytes=sha256:0000746c55336cd8d34885545f9347d96607d0391fbd3e76dae7f2b3447775b4 (it:av:scan:result=(1,) it:av:scan:result=(2,) it:av:scan:result=(3,) :target:file=sha256:0000746c55336cd8d34885545f9347d96607d0391fbd3e76dae7f2b3447775b4 :verdict=malicious) ]
.. storm-pre:: [file:bytes=sha256:00007694135237ec8dc5234007043814608f239befdfc8a61b992e4d09e0cf3f (it:av:scan:result=(4,) it:av:scan:result=(5,) it:av:scan:result=(6,) it:av:scan:result=(7,) it:av:scan:result=(8,) it:av:scan:result=(9,) :target:file=sha256:00007694135237ec8dc5234007043814608f239befdfc8a61b992e4d09e0cf3f :verdict=malicious) ]
.. storm-cli:: $threshold=5 file:bytes +{ -> it:av:scan:result +:verdict=malicious } >= $threshold [ +#review ]

.. TIP::

The example above uses a subquery filter (:ref:`filter-subquery`) to pivot to the ``it:av:filehit`` nodes
associated with the ``file:bytes`` node, and compares the number of AV hits to the value of the ``$threshold``
The example above uses a subquery filter (:ref:`filter-subquery`) to pivot to the ``it:av:scan:result`` nodes
associated with the ``file:bytes`` node, and compares the number of malicious AV scan results to the value of the ``$threshold``
variable.


Expand Down
8 changes: 4 additions & 4 deletions docs/synapse/userguides/storm_ref_automation.rstorm
Original file line number Diff line number Diff line change
Expand Up @@ -327,13 +327,13 @@ new cron jobs. See the help for each command for options and additional details.
**One-time (non-recurring) cron job example**

You want to use a one-time cron job that runs during off hours to perform some data cleanup. Specifically, you
want a job that will lift all existing ``media:news`` nodes and remove (delete) the deprecated ``:author`` property
want a job that will lift all existing ``media:news`` nodes and remove (delete) the ``:publisher:name`` property
at 0200 UTC.

.. storm-pre:: [ (media:news=* :author='ron the cat' :title='my article') (media:news=* :title='your article') ]
.. storm-pre:: media:news:author [ -:author ]
.. storm-pre:: [ (media:news=* :publisher:name='ron the cat' :title='my article') (media:news=* :title='your article') ]
.. storm-pre:: media:news:publisher:name [ -:publisher:name ]

.. storm-cli:: cron.at --hour 2 { media:news:author [ -:author ] }
.. storm-cli:: cron.at --hour 2 { media:news:publisher:name [ -:publisher:name ] }

We can view the details of this cron job using the ``cron.list`` command:

Expand Down
8 changes: 0 additions & 8 deletions docs/synapse/userguides/storm_ref_cmd.rstorm
Original file line number Diff line number Diff line change
Expand Up @@ -1107,14 +1107,6 @@ on the form and value of the object scanned and the signature name used for the
You can optionally include the name of the scanner / scan engine and/or the time the scan was performed
for additional deconfliction.

.. NOTE::

The command can be used to generate ``it:av:scan:result`` nodes from existing ``it:av:filehit`` nodes.
The ``it:av:filehit`` form has been marked as deprecated and will be removed in a future version of Synapse.
Some :ref:`synapse_powerups` that previously created ``it:av:filehit`` nodes may include dedicated commands
to assist with migration; if a migration tool exists, it will be documented in the Admin Guide section of
the Power-Up's Help.

**Syntax:**

.. storm-cli:: gen.it.av.scan.result --help
Expand Down
6 changes: 1 addition & 5 deletions synapse/cortex.py
Original file line number Diff line number Diff line change
Expand Up @@ -3666,11 +3666,7 @@ async def _initDeprLocks(self):
# TODO: 3.0.0 conversion will truncate this hive key

if self.inaugural:
locks = (
# 2.87.0 - lock out incorrect crypto model
('crypto:currency:transaction:inputs', True),
('crypto:currency:transaction:outputs', True),
)
locks = ()
for k, v in locks:
await self._hndlsetDeprLock(k, v)

Expand Down
4 changes: 0 additions & 4 deletions synapse/lib/stormlib/gen.py
Original file line number Diff line number Diff line change
Expand Up @@ -678,7 +678,6 @@ class LibGen(s_stormtypes.Lib):
),
'storm': 'yield $lib.gen.langByName($cmdopts.name)',
},
# todo: remove it:av:filehit example in 3.x.x
{
'name': 'gen.it.av.scan.result',
'descr': '''
Expand All @@ -693,9 +692,6 @@ class LibGen(s_stormtypes.Lib):
// Also deconflict by scanner name and scan time
gen.it.av.scan.result inet:fqdn fqdn vertex.link foosig --scanner-name barscanner --time 2022-11-03
// Generate an it:av:scan:result node from an it:av:filehit node
it:av:filehit#foo | gen.it.av.scan.result file:bytes :file :sig:name
''',
'cmdargs': (
('form', {'help': 'The target form.'}),
Expand Down
Loading

0 comments on commit b573e71

Please sign in to comment.