Add Beaconing Analysis

[Cyb3r-Monk]

I have a beaconing detection algorithm which is ported from RITA
https://github.com/Cyb3r-Monk/RITA-J

The algorithm can be adopted and added as a feature and perform beaconing analysis for based on a schema provided:

1. Bulk dataset
2. Specific source (user, src_host, src_ip)
3. Specific destination (dst_host, dst_ip)
4. Source-Destination

2nd, 3rd, and 4th option can be done one the query level and may not be required (query the required data, perform bulk analysis which has only the specified source/destination logs)

[dan-stats-1]

Hi @Cyb3r-Monk
I'm currently working on the implementation of this for the polling detection module I've been working on (#515)

I had a couple of quick questions I wanted to clarify

1. Bowleys skewness coefficient [1] uses the quantiles at 0.25, 0.5 and 0.75 whereas in your notebook the quantiles are 0.2, 0.5 and 0.8.
   What's the reason for this variation of Bowleys method?

2. The tsConnCountScore is equal to conn_count / ( (max(timestamp) - min(timestamp)) / 90 ).
   What's the purpose of the 90 in the above calculation?

Once I've finished the whole the thing I'll ask you to review it :)

Cheers
Dan

References

[1] https://www.statisticshowto.com/bowley-skewness/

[dan-stats-1]

Hi @Cyb3r-Monk
I've found a few differences between the code in your notebook and the go code from the rita repo (https://github.com/activecm/rita/blob/master/pkg/beacon/analyzer.go)

Should I follow your code or should I be implementing a translation of the go code?

Cheers
Dan

[Cyb3r-Monk]

Hi @Cyb3r-Monk I'm currently working on the implementation of this for the polling detection module I've been working on (#515)

I had a couple of quick questions I wanted to clarify

1. Bowleys skewness coefficient [1] uses the quantiles at 0.25, 0.5 and 0.75 whereas in your notebook the quantiles are 0.2, 0.5 and 0.8.
   What's the reason for this variation of Bowleys method?
2. The tsConnCountScore is equal to conn_count / ( (max(timestamp) - min(timestamp)) / 90 ).
   What's the purpose of the 90 in the above calculation?

Once I've finished the whole the thing I'll ask you to review it :)

Cheers Dan

References

[1] https://www.statisticshowto.com/bowley-skewness/

1. Regarding skewness, I just made a tweak according to my experience. Some normal traffic was creating false positives. We could make the parameter configurable with default values maybe?

2. tsConnCount score is based on how many connections would a beacon make during a session(session = the start and end time of the data we query). I increased it since a becaon can have more than 10sec of sleep value. This could also be configurable by the analyst. Btw, 90 is 90sec if I remember correctly. RITA was using 10sec but now uses 3600sec.

[Cyb3r-Monk]

Hi @Cyb3r-Monk I've found a few differences between the code in your notebook and the go code from the rita repo (https://github.com/activecm/rita/blob/master/pkg/beacon/analyzer.go)

Should I follow your code or should I be implementing a translation of the go code?

Cheers Dan

Seems like they made some changes during the last few months. I haven't analyzed new algorithm and can't say anything. Hopefully this year, I'll come up with a better detection logic(I already have something but haven't tested it yet enough and I want to use it as a CFP for a conf first). So, it's up to you!

[dan-stats-1]

Hi @Cyb3r-Monk I've found a few differences between the code in your notebook and the go code from the rita repo (https://github.com/activecm/rita/blob/master/pkg/beacon/analyzer.go)
Should I follow your code or should I be implementing a translation of the go code?
Cheers Dan

Seems like they made some changes during the last few months. I haven't analyzed new algorithm and can't say anything. Hopefully this year, I'll come up with a better detection logic(I already have something but haven't tested it yet enough and I want to use it as a CFP for a conf first). So, it's up to you!

Ah I suspected that was the case

I'll translate the latest go code to python but I'll make the values (that you tweaked originally) parameters passed to the function

Thanks for your help!