Skip to content

Commit 1bbf2a7

Browse files
pranjalg1331pranjalg1331
authored
Gsoc25 refactor analyzer tests (#2886)
* unit tests Signed-off-by: pranjalg1331 <[email protected]> * unittests framework Signed-off-by: pranjalg1331 <[email protected]> * cleanup Signed-off-by: pranjalg1331 <[email protected]> * correct python error Signed-off-by: pranjalg1331 <[email protected]> * making class method Signed-off-by: pranjalg1331 <[email protected]> * mock location changed Signed-off-by: pranjalg1331 <[email protected]> * python deepsource Signed-off-by: pranjalg1331 <[email protected]> * abusix Signed-off-by: pranjalg1331 <[email protected]> * file base class initial draft Signed-off-by: pranjalg1331 <[email protected]> * blint_scan test Signed-off-by: pranjalg1331 <[email protected]> * boxjs Signed-off-by: pranjalg1331 <[email protected]> * debloat Signed-off-by: pranjalg1331 <[email protected]> * cleanup Signed-off-by: pranjalg1331 <[email protected]> * updated base class Signed-off-by: pranjalg1331 <[email protected]> * python errors Signed-off-by: pranjalg1331 <[email protected]> * added analyzer unit tests Signed-off-by: pranjalg1331 <[email protected]> * new file analyzers Signed-off-by: pranjalg1331 <[email protected]> * checkphish Signed-off-by: pranjalg1331 <[email protected]> * greynoise_labs Signed-off-by: pranjalg1331 <[email protected]> * ja4db Signed-off-by: pranjalg1331 <[email protected]> * netlas Signed-off-by: pranjalg1331 <[email protected]> * polyswarm Signed-off-by: pranjalg1331 <[email protected]> * thug_url Signed-off-by: pranjalg1331 <[email protected]> * zoomeye Signed-off-by: pranjalg1331 <[email protected]> * file analyzers test+ base class improved Signed-off-by: pranjalg1331 <[email protected]> * commenting old tests Signed-off-by: pranjalg1331 <[email protected]> * commenting old tests -2 Signed-off-by: pranjalg1331 <[email protected]> * commenting old tests -3 Signed-off-by: pranjalg1331 <[email protected]> * add init Signed-off-by: pranjalg1331 <[email protected]> * trial workflow Signed-off-by: pranjalg1331 <[email protected]> * correct error Signed-off-by: pranjalg1331 <[email protected]> * greynosise error Signed-off-by: pranjalg1331 <[email protected]> * greynosise error-2 Signed-off-by: pranjalg1331 <[email protected]> * comment out Signed-off-by: pranjalg1331 <[email protected]> * remove monkeypatch Signed-off-by: pranjalg1331 <[email protected]> * remove monkeypatch from file_analyzers Signed-off-by: pranjalg1331 <[email protected]> * zippy_scan Signed-off-by: pranjalg1331 <[email protected]> * yaraify test class Signed-off-by: pranjalg1331 <[email protected]> * uncomment Signed-off-by: pranjalg1331 <[email protected]> * delete Signed-off-by: pranjalg1331 <[email protected]> * uncomment analyzer tests Signed-off-by: pranjalg1331 <[email protected]> * new analyzer tests Signed-off-by: pranjalg1331 <[email protected]> * clearing monkeppatch Signed-off-by: pranjalg1331 <[email protected]> * modify class to remove mock response in docker analyzers Signed-off-by: pranjalg1331 <[email protected]> * no changes to pr automation Signed-off-by: pranjalg1331 <[email protected]> * test errors Signed-off-by: pranjalg1331 <[email protected]> * correct errors Signed-off-by: pranjalg1331 <[email protected]> * separated transactions tests Signed-off-by: pranjalg1331 <[email protected]> * hfinger Signed-off-by: pranjalg1331 <[email protected]> * loggin improved Signed-off-by: pranjalg1331 <[email protected]> * remove f Signed-off-by: pranjalg1331 <[email protected]> * merge logs Signed-off-by: pranjalg1331 <[email protected]> * deep python errors Signed-off-by: pranjalg1331 <[email protected]> * pr template update Signed-off-by: pranjalg1331 <[email protected]> * new analyzers + update pr template Signed-off-by: pranjalg1331 <[email protected]> * update dir structure for integration tests Signed-off-by: pranjalg1331 <[email protected]> * more tests Signed-off-by: pranjalg1331 <[email protected]> * lint Signed-off-by: pranjalg1331 <[email protected]> --------- Signed-off-by: pranjalg1331 <[email protected]>
1 parent 2af860a commit 1bbf2a7

File tree

333 files changed

+6986
-7733
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

333 files changed

+6986
-7733
lines changed

‎.github/pull_request_template.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,8 +26,8 @@ Please delete options that are not relevant.
2626
- [ ] Check if it could make sense to add that analyzer/connector to other [freely available playbooks](https://intelowlproject.github.io/docs/IntelOwl/usage/#list-of-pre-built-playbooks).
2727
- [ ] I have provided the resulting raw JSON of a finished analysis and a screenshot of the results.
2828
- [ ] If the plugin interacts with an external service, I have created an attribute called precisely `url` that contains this information. This is required for Health Checks (HEAD HTTP requests).
29-
- [ ] If the plugin requires mocked testing, `_monkeypatch()` was used in its class to apply the necessary decorators.
30-
- [ ] I have added that raw JSON sample to the `MockUpResponse` of the `_monkeypatch()` method. This serves us to provide a valid sample for testing.
29+
- [ ] If a new analyzer has beed added, I have created a unittest for it in the appropriate dir. I have also mocked all the external calls, so that no real calls are being made while testing.
30+
- [ ] I have added that raw JSON sample to the `get_mocker_response()` method of the unittest class. This serves us to provide a valid sample for testing.
3131
- [ ] I have created the corresponding `DataModel` for the new analyzer following the [documentation](https://intelowlproject.github.io/docs/IntelOwl/contribute/#how-to-create-a-datamodel)
3232
- [ ] I have inserted the copyright banner at the start of the file: ```# This file is a part of IntelOwl https://github.com/intelowlproject/IntelOwl # See the file 'LICENSE' for copying permission.```
3333
- [ ] Please avoid adding new libraries as requirements whenever it is possible. Use new libraries only if strictly needed to solve the issue you are working for. In case of doubt, ask a maintainer permission to use a specific library.

‎.github/workflows/pull_request_automation.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,9 @@ jobs:
115115
- name: Run test
116116
run: |
117117
docker exec intelowl_uwsgi coverage run manage.py test --keepdb tests
118+
- name: Run async tests
119+
run: |
120+
docker exec intelowl_uwsgi coverage run manage.py test --keepdb async_tests
118121
119122
frontend-tests:
120123
runs-on: ubuntu-latest

‎api_app/analyzers_manager/classes.py‎

Lines changed: 0 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,6 @@
1212
from django.conf import settings
1313

1414
from certego_saas.apps.user.models import User
15-
from tests.mock_utils import MockUpResponse, if_mock_connections, patch
1615

1716
from ..choices import Classification, PythonModuleBasePaths
1817
from ..classes import Plugin
@@ -487,43 +486,6 @@ def _docker_get(self):
487486
raise AssertionError
488487
return resp
489488

490-
@staticmethod
491-
def mocked_docker_analyzer_get(*args, **kwargs):
492-
return MockUpResponse(
493-
{"key": "test", "returncode": 0, "report": {"test": "This is a test."}}, 200
494-
)
495-
496-
@staticmethod
497-
def mocked_docker_analyzer_post(*args, **kwargs):
498-
return MockUpResponse({"key": "test", "status": "running"}, 202)
499-
500-
def _monkeypatch(self, patches: list = None):
501-
"""
502-
Here, `_monkeypatch` is an instance method and not a class method.
503-
This is because when defined with `@classmethod`, we were getting the error
504-
```
505-
'_patch' object has no attribute 'is_local'
506-
```
507-
whenever multiple analyzers with same parent class were being called.
508-
"""
509-
if patches is None:
510-
patches = []
511-
# no need to sleep during tests
512-
self.poll_distance = 0
513-
patches.append(
514-
if_mock_connections(
515-
patch(
516-
"requests.get",
517-
side_effect=self.mocked_docker_analyzer_get,
518-
),
519-
patch(
520-
"requests.post",
521-
side_effect=self.mocked_docker_analyzer_post,
522-
),
523-
)
524-
)
525-
return super()._monkeypatch(patches)
526-
527489
def health_check(self, user: User = None) -> bool:
528490
"""
529491
basic health check: if instance is up or not (timeout - 10s)

‎api_app/analyzers_manager/file_analyzers/artifacts.py‎

Lines changed: 0 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
import logging
22

33
from api_app.analyzers_manager.classes import DockerBasedAnalyzer, FileAnalyzer
4-
from tests.mock_utils import MockUpResponse
54

65
logger = logging.getLogger(__name__)
76

@@ -28,47 +27,3 @@ def run(self):
2827
)
2928
result = self._docker_run(req_data, req_files, analyzer_name=self.analyzer_name)
3029
return result
31-
32-
# flake8: noqa
33-
@staticmethod
34-
def mocked_docker_analyzer_get(*args, **kwargs):
35-
return MockUpResponse(
36-
{
37-
"report": {
38-
"name": "APK_Artifacts",
39-
"process_time": 5.07,
40-
"status": "SUCCESS",
41-
"end_time": "2024-08-27T10:03:15.563886Z",
42-
"parameters": {},
43-
"type": "analyzer",
44-
"id": 72,
45-
"report": {
46-
"dex": ["classes.dex"],
47-
"md5": "8a05a189e58ccd7275f7ffdf88c2c191",
48-
"root": [],
49-
"family": {
50-
"name": "CryCrypto",
51-
"match": 11.11,
52-
"value": {
53-
"intent": 33.33,
54-
"permission": 0.0,
55-
"application": 0.0,
56-
},
57-
},
58-
"string": {"known": [], "base64": [], "telegram_id": []},
59-
"library": [],
60-
"network": {"ip": [], "url": [], "param": []},
61-
"sandbox": [
62-
"https://tria.ge/s?q=8a05a189e58ccd7275f7ffdf88c2c191",
63-
"https://www.joesandbox.com/analysis/search?q=8a05a189e58ccd7275f7ffdf88c2c191",
64-
"https://www.virustotal.com/gui/search/8a05a189e58ccd7275f7ffdf88c2c191",
65-
"https://bazaar.abuse.ch/browse.php?search=md5:8a05a189e58ccd7275f7ffdf88c2c191",
66-
"https://koodous.com/apks?search=8a05a189e58ccd7275f7ffdf88c2c191",
67-
],
68-
"version": "1.1.1",
69-
"elapsed_time": 0.02,
70-
},
71-
}
72-
},
73-
200,
74-
)

‎api_app/analyzers_manager/file_analyzers/boxjs_scan.py‎

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,3 @@ def run(self):
6767
report["uris"] = list(set(report["uris"])) # uniq
6868

6969
return report
70-
71-
# disable mockup connections for this class
72-
@classmethod
73-
def _monkeypatch(cls, patches: list = None) -> None: ... # noqa: E704

‎api_app/analyzers_manager/file_analyzers/cape_sandbox.py‎

Lines changed: 0 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,6 @@
1111

1212
from api_app.analyzers_manager.classes import FileAnalyzer
1313
from api_app.analyzers_manager.exceptions import AnalyzerRunException
14-
from tests.mock_utils import MockUpResponse, if_mock_connections, patch
1514

1615
logger = logging.getLogger(__name__)
1716

@@ -335,31 +334,3 @@ def __poll_for_result(
335334
)
336335

337336
return results
338-
339-
@classmethod
340-
def _monkeypatch(cls):
341-
patches = [
342-
if_mock_connections(
343-
patch(
344-
"requests.Session.get",
345-
return_value=MockUpResponse(
346-
{"error": False, "data": "completed"}, 200
347-
),
348-
),
349-
patch(
350-
"requests.Session.post",
351-
return_value=MockUpResponse(
352-
{
353-
"error": False,
354-
"data": {
355-
"task_ids": [1234],
356-
},
357-
"errors": [],
358-
"url": ["http://fake_url.com/submit/status/1234/"],
359-
},
360-
200,
361-
),
362-
),
363-
)
364-
]
365-
return super()._monkeypatch(patches=patches)

‎api_app/analyzers_manager/file_analyzers/clamav.py‎

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,6 @@
44
import logging
55

66
from api_app.analyzers_manager.classes import DockerBasedAnalyzer, FileAnalyzer
7-
from tests.mock_utils import MockUpResponse
87

98
logger = logging.getLogger(__name__)
109

@@ -46,9 +45,3 @@ def run(self):
4645
logger.error(f"no detections extracted? {self.job_id}")
4746

4847
return {"detections": list(set(detections)), "raw_report": report}
49-
50-
@staticmethod
51-
def mocked_docker_analyzer_get(*args, **kwargs):
52-
return MockUpResponse(
53-
{"key": "test", "returncode": 0, "report": "OK real_signature\n"}, 200
54-
)

‎api_app/analyzers_manager/file_analyzers/cuckoo_scan.py‎

Lines changed: 0 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,6 @@
1010

1111
from api_app.analyzers_manager.classes import FileAnalyzer
1212
from api_app.analyzers_manager.exceptions import AnalyzerRunException
13-
from tests.mock_utils import MockUpResponse, if_mock_connections, patch
1413

1514
logger = logging.getLogger(__name__)
1615

@@ -261,19 +260,3 @@ def __cuckoo_retrieve_and_create_report(self):
261260
logger.info(f"report generated for ({self.filename},{self.md5})")
262261

263262
return result
264-
265-
@classmethod
266-
def _monkeypatch(cls):
267-
patches = [
268-
if_mock_connections(
269-
patch(
270-
"requests.Session.get",
271-
return_value=MockUpResponse({"task": {"status": "reported"}}, 200),
272-
),
273-
patch(
274-
"requests.Session.post",
275-
return_value=MockUpResponse({}, 200),
276-
),
277-
)
278-
]
279-
return super()._monkeypatch(patches=patches)

0 commit comments

Comments
 (0)