Custom RPZ feeds from Infoblox TIDE
Infoblox Threat Intelligence Data Exchange (TIDE) leverages highly accurate machine-readable threat intelligence (MRTI) data to aggregate and selectively distribute data across a broad range of security infrastructure. Infoblox threat intelligence team curates, normalizes, and refines the high-quality threat data to minimize false positives. Threat feeds begin with information gained from native investigations and harvesting techniques. Then they combined with verified and observed data from trusted partners including government agencies, academic institutions, several premier Internet infrastructure providers, and law enforcement. The end result is a highly refined feed with a very low historical false-positive rate. It also works with several premium data providers, including SURBL, CrowdStrike, Emerging Threats, Farsight Security, iSight Partners, ThreatTrack Security etc to create an in-solution third-party threat indicator feed data marketplace.
Infoblox ActiveTrust solutions use TIDE to create various response policy zone (RPZ) feeds. Right now you can subscribe up to 26 different RPZ feeds including partners feeds from DHS, SURBL, Proofpoint Emerging Threats, Farsight Security, ThreatTrack Security. The list is really comprehensive which allows you to implement a superb DNS protection layer.
What is really missing is possibility to create custom feeds. Why should you care about it?
- Infoblox DDI and ISC Bind have a limitation - maximum 32 response policy zones per a DNS view. If you want to use additional feeds (e.g. own or 3rd party) you will be limited by the remaining number of RPZs or should disregard some feeds. Infoblox is also working to introduce new response policy zones so quantity of remaining zones will be tight;
- RPZ Feeds use memory and affect DNS service performance. Some feeds can contain the same indicators especially if you take a look on SURBL and Farsight newly observed domains;
- You may want to have fine control when indicators are expired and/or what kind of indicators to include to produce different feeds for different devices/networks.
To create custom RPZ feeds you may use following approaches:
- Create a local RPZ zone, pull indicators from TIDE and update the RPZ via API or CSV import. The zone will be stored on Grid Manager even if it is provisioned on a member.
- Create an RPZ zone on a separate DNS server (Infoblox, Bind, PowerDNS) and use a zone transfer to distribute it to Grid members.
Both approaches require scripting/developers skills.
A year ago there were no open source tool (or I was not aware about it) which will be able to automate the whole RPZ creation and maintenance process. So instead of working on the scripts (actually I have my own scripts to manage RPZ on Infoblox) I’ve decided to develop a DNS server which will do all the magic in the most efficient way.
ioc2rpz is a custom DNS server which automatically converts indicators (e.g. malicious FQDNs, IPs) from various sources into RPZ feeds and automatically maintains/updates them. The feeds can be distributed to any open source and/or commercial DNS servers which support RPZ (DNS Firewall), e.g. Infoblox, ISC Bind, PowerDNS. With ioc2rpz you can define your own feeds, actions and prevent undesired communications.
With ioc2rpz you can mix feeds to generate a single RPZ or multiple RPZs. Trusted domains and IPs can be whitelisted. ioc2rpz supports expiration of indicators and accordingly rebuilds zones. An incremental zone transfer (IXFR) is supported as well.
The current ioc2rpz release supports: local files, remote http/https/ftp files, REST API requests via http/https protocols with or without basic HTTP authentication. The username and password should be passed in the URL. You can use any file format if you can write a REGEX to extract indicators and the indicators are separated by a newline or/and return carriage char (/n, /r, /r/n).
Infoblox TIDE can be leverage as a data source for ioc2rpz and the RPZ feeds can be distributed to Infoblox appliances. The expiration time is supported and ioc2rpz will automatically retire old indicators independently of standard zone refresh time. ioc2rpz.gui is a separate project which significantly simplify ioc2rpz management.
Infoblox TIDE provides you ability to request any indicators you want with filtering by profile (e.g. IID, SURBL, CrowdStrike, iSIGHTPARTNERS, FarsightSecurity, ThreatTrackSecurity, EmergingThreats, AISCOMM depending on your subscription), threat class(e.g. Bot, Cryptocurrency, MalwareC2 etc), threat property (e.g. Bot_Citadel, Cryptocurrency_Cryptojacking, Policy_NewlyObservedDomains etc), TLD (e.g. com, net, ru etc) etc and in different export formats.
Right now Infoblox TIDE provides 27 different classes and 461 different properties. The comprehensive descriptions of the classes and properties you can find on the Infoblox’s ActiveTrust website under “Resources” → “Threat Classification Guide”.
Below you can find a sample response in JSON format which includes all fields.
{
"id": "fce2243e-c59f-11e7-9273-696e900af2b0",
"type": "HOST",
"host": "dns3.alphadyns.ru",
"domain": "alphadyns.ru",
"tld": "ru",
"profile": "IID",
"property": "MaliciousNameserver_Generic",
"class": "MaliciousNameserver",
"threat_level": 100,
"detected": "2017-11-09T22:47:39.106Z",
"received": "2017-11-09T22:47:39.106Z",
"imported": "2017-11-09T22:47:39.106Z",
"expiration": "2019-11-09T22:47:39.106Z",
"dga": false,
"up": true,
"batch_id": "fcde2c70-c59f-11e7-9273-696e900af2b0",
"extended": {
"threat_family": "Fast flux nameserver for Dark Cloud/Zbot malware/fraud network."
}
},
For RPZ zone generation we should use the host or the domain (e.g. if you want to implement strict rules or redirect traffic to SWG) or the ip field and the expiration date if it exists and required.
In the table below you can find several sample requests to fetch different data from Infoblox TIDE in csv format which is easy and efficient to parse on ioc2rpz.
| Request* | Description |
|---|---|
| /api/data/threats/host? field=host,expiration &data_format=csv&tld=ru &profile=IID,SURBL | All active malicious domains in TLD “RU” submitted by Infoblox and SURBL. |
| /api/data/threats/host?field=host,expiration &data_format=csv &property=Policy_NewlyObservedDomains |
All newly observed domains independently of a data provider. The domains will be deduplicated on ioc2rpz. |
| /api/data/threats/ip?field=ip,expiration &data_format=csv &property=Policy_BogonRFC1122, Policy_BogonRFC1918, Policy_BogonRFC2544, Policy_BogonRFC3927, Policy_BogonRFC5736, Policy_BogonRFC5737, Policy_BogonRFC6598 |
IP-addresses and networks which should not be used in internet. E.g. to protect against DNS rebinding attack. |
| /api/data/threats/host?field=host,expiration &data_format=csv &class=MalwareC2 | All domains in Malware C2 class |
| /api/data/threats/ip?field=ip,expiration &data_format=csv &property=Proxy_TorExitNode, Proxy_TorNode, Proxy_TorRelayNode | IP-addresses of Tor nodes |
*Spaces were added to format the table. Before using the requests - remove spaces.
ioc2rpz will cleanup the feed with a REGEX and whitelists, deduplicate indicators and effectively manage response policy zones.
On [ioc2rpz wiki](https://github.com/Homas/ioc2rpz/wiki/IOC-Sources) page you can find examples of various IOC feeds including Infoblox TIDE. The examples can be imported (ioc2rpz.gui) or used “as is” in the ioc2rpz configuration. Let’s take a look on the following example:
{source,{"at_ip_w_exp","https://**APIKEY**@api.activetrust.net:8000/api/data/threats/state/ip?profile=IID&field=ip,expiration&data_format=csv","[:AXFR:]&from_date=[:FTimestamp:]","^(?!host)(?!ip)\"?\'?([A-Za-z0-9][A-Za-z0-9\-\._]+)\"?\'?,?([0-9:TZtz -.]+)?$"}}.
The configuration line contains: the source name, URL to fetch full dataset (AXFR), URL to fetch incremental (new indicators only) dataset (IXFR) and REGEX to extract an IP and an expiration date per line. The screenshot below shows how the source is presented in ioc2rpz.gui
The full zone transfer URL (AXFR) fetches all active IP addresses which were detected by Infoblox Cyber intelligence team (profile=IID) along with expiration date in CSV format. The incremental zone transfer fetches only new indicators which were added after a previous request. The timestamp is automatically maintained by ioc2rpz. The universal REGEX for Infoblox TIDE filters out headers (host, ip) and quotes. It extracts an indicator and an expiration date. If an indicator has an expiration date/time it will be used to retire the indicator from an RPZ feed. If there is no expiration date, an indicator can be removed only with the full source update (AXFR).
The next example is similar to the previous one except that it is fetching hosts without expiration date and we use a different regular expression (w/o an expiration date).
{source,{"at_hosts","https://**APIKEY**@api.activetrust.net:8000/api/data/threats/state/host?profile=IID&field=host&data_format=csv","[:AXFR:]&from_date=[:FTimestamp:]","^(?!host)(?!ip)\"?\'?([A-Za-z0-9][A-Za-z0-9\-\._]+)[^A-Za-z0-9\-\._]*.*$"}}.
To make the provided examples working you need to substitute “**APIKEY**” with your TIDE API key.
When you finish provisioning RPZs you may export DNS server configuration in the Infoblox CSV import format and import it in Infoblox. Please take a look how it works on the following video:
If you export RPZs in the GUI they will be automatically sorted by a recommended zone precedence: RPZ containing only domain based rules, mixed zones, RPZ containing IP/Network-based rules only. It is recommended to sort RPZ by this order for performance optimization and DNS based exfiltration protection. RPZs which contain domain based rules only do not require a recursion to block a DNS request.
The next video demonstrates ioc2rpz technology and ioc2rpz.gui.
More information about ioc2rpz technology you can find on ioc2rpz.com. If you have any question feel free to reach me.
I’m going to demo the ioc2rpz technology at Def Con 26 Demo Labs. So if you are attending it let’s meet there.