New/Established Tag showing incorrect after configuration

[tdeering]

I ran domain-stats-settings and configured the established setting to be 30 days, however I am still seeing domains coming across that are greater than 30 days old showing as NEW still.

This is on a CENTOS 7 running as a service.

It was installed via python3 -m pip install domain-stats

cat gunicorn_config.py

import gunicorn
import multiprocessing
import os

os.environ["SERVER_SOFTWARE"] = "domain_stats"
bind = "127.0.0.1:5730"
workers = 25
threads = 36
gunicorn.SERVER_SOFTWARE = 'domain_stats'

cat domain_stats.yaml
cache_browse_limit: 100
count_rdap_errors: false
enable_freq_scores: true
established_days_age: 30
freq_avg_alert: 5.0
freq_table: freqtable2018.freq
freq_word_alert: 4.0
ip_address: 127.0.0.1
local_port: 5730
log_detail: 0
mode: rdap
rdap_error_ttl_days: 7
threads_per_worker: 36
timezone_offset: 0
workers: 25

[Unit]
Description=Domain Stats Service
After=network.target

[Service]
#You may need to change the path so it points to where you are storing your data
WorkingDirectory=/opt/domain-stats
ExecStart=/usr/local/bin/domain-stats ./
PIDFile=/var/run/domainstats.pid
Restart=always

[Install]
WantedBy=multi-user.target


{"alerts":[],"category":"NEW","freq_score":[8.515,8.5607],"seen_by_isc":"top1m","seen_by_web":"Wed, 29 Jul 2020 14:23:24 GMT","seen_by_you":"Tue, 17 Aug 2021 19:14:46 GMT"}

[MarkBaggett]

Hi.
The categorization of a domain as "NEW" or "ESTABLISHED" happens at the time the entry is added to the local database (ie its First Contact), and not for each individual query. This is done for performance reasons so that domain_stats can keep up with a large volume of requests. As a result if you change your establish date on an running system only new queries (ie "First-contact") domains will reflect the changes. "Cached" entries that are in the database will be updated when the domain registration expires. If you want to reclassify all the domains then you would have to delete your local database and let it begin rebuilding it.

[tdeering]

The first run should have been with the 30 day setting in place. I'll delete the database and see if things come in as expected.

[MarkBaggett]

Ok. Please do let me know if that doesn’t fix it. Mark

…

________________________________ From: tdeering ***@***.***> Sent: Wednesday, August 18, 2021 2:40:43 PM To: MarkBaggett/domain_stats ***@***.***> Cc: MarkBaggett ***@***.***>; Comment ***@***.***> Subject: Re: [MarkBaggett/domain_stats] New/Established Tag showing incorrect after configuration (#25) The first run should have been with the 30 day setting in place. I'll delete the database and see if things come in as expected. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#25 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAFVSDOELQZ3WQD5BBNF3ODT5P5CXANCNFSM5CMOI3HA>.