Releases: mode-network/synth-subnet
v1.3.2-rc
Upgrade Process
- Commands:
git fetchgit checkout v1.3.2-rcsource bt_venv/bin/activatealembic upgrade head
- Restart validator
pm2 start validator.local.config.js
What's Changed
- update api doc link by @Thykof in #137
- add ip address in metagraph history by @Thykof in #138
- add columns created at and updated at with server default by @Thykof in #132
- Remove scores duplicates by @Thykof in #133
- axon: 16-chunk prompt request by @parkhomenko in #102
- Score unique and on conflict update by @Thykof in #139
Full Changelog: v1.3.1...v1.3.2-rc
v1.3.1
v1.3.0
Schedule the launch of ETH prompts
What's Changed
Full Changelog: v1.2.2...v1.3.0
Upgrade Process
- Commands:
git fetchgit checkout v1.3.0source bt_venv/bin/activate
- Restart validator
pm2 start validator.local.config.js
CRPS Calculation Update for BTC and ETH Prediction Prompts
Prompt scores are calculated individually for each asset prediction—either BTC or ETH. When we report the overall smoothed score, we take a moving average of these individual prompt scores across all assets.
How Prompt Scores are Calculated:
Individual interval CRPS values are now computed by comparing the real price change in the interval as a fraction of the price at the start of the interval, measured in basis points (100th of a percentage point), against the distribution of the price changes across the paths predicted by the miner, also measured in basis points.
For example, if BTC is priced at $100K at time T₀ and rises to $102K at T₀ + 5 minutes, the price change is +200 basis points. Each miner’s predicted distribution is then compared against this 200bps change to calculate the CRPS for that interval.
CRPS is calculated across multiple time intervals: 288 x 5 minutes, 48 x 30 minutes, and 8 x 3 hours.
A second component of the CRPS scoring is applied only to the final path price provided by miners. This compares the distribution of the absolute predictions of the final price, with the actual crypto price at that time. This final CRPS value is then divided by the real price and multiplied by 10,000, to also give it the units of basis points. These 344 interval scores and 1 absolute score are summed to give the 'CRPS Sum' for the miner for that prompt
Prompt Scores are obtained by capping very bad CRPS sums, or miners with missing predictions, to the 90th percentile CRPS Sum (i.e. top 10% worst value). and then subtracting the best CRPS Sum (lowest score). So a prompt score of 0 is the best possible outcome, and anything higher represents an increasingly worse score
ETH Prediction Update:
Starting Monday 19th at 2 PM UTC, validators will start sending ETH prediction prompts, followed by BTC prompts 30 minutes later. This alternating schedule will continue at the same hourly cadence.
Every hour, we compute a moving average of prompt scores, using all prompts across both assets from the last 7 days, with more recent data weighted more heavily. The final smoothed score will therefore eventually be an equally weighted average of BTC and ETH (50/50) prompts (until Tuesday 27th, there will still be more scored BTC prompts present than Eth prompts in the past 7 days lookback window, courtesy of the BTC only period up to the 19th)
Reward weights are still calculated using the Softmax of the smoothed score, using the same Beta value as is used currently
Implications for Miners:
Miners who only predict one asset will be significantly disadvantaged. Predictions for the other asset will be considered missing, and hence be assigned the 90th percentile score, greatly damaging their reward weights.
Miners, please prepare accordingly.
v1.2.2
Upgrade Process
- Modify the pm2 config file
- change
--softmax.beta -0.0475
- change
- Commands:
git fetchgit checkout v1.2.2source bt_venv/bin/activatepip install -r requirements.txtalembic upgrade head
- Restart validator
pm2 start validator.local.config.js
What's Changed
Full Changelog: v1.2.1...v1.2.2
v1.2.1 Prompt score V3 (and backfill)
With prompt score v3 and backfill and avoid scores duplicate with if exists update else insert
- 2b043c4 insert or update score
- 4cdad0d backfill prompt score v3
- 334c114 prompt score v3
- 9be5b0e change crps calc
Use a new value for softmax.beta: -0.0475.
Full Changelog: v1.2.0...v1.2.1
v1.2.0 Refactor and prepare for multiple assets
Version just before prompt score v3, with the new forward algorithme (2 async tasks: one for prompt and one for scoring)
- ae49fcb refactor alchemy models
- 5b5d3e6 add pyth-listing.py
- 862e835 rename: remove _v2 for prompt score
- dfddaf9 no alert on testnet
- e4bc734 miner: handle different figma for assets
- 30cb779 add tokens of pyth oracle
- d694aa0 support for multiple simulations
- 9529fd3 fix wandb alert on finished run
- 7532dc5 fix #123: validate all available prompts
- e9cfe5a fix handler error KeyboardInterrupt
- 82d9bf1 fix lint with annotation
- d6b7a06 refactor score_time type
- b69e5a9 fix enable wandb
- d88849a remove num_concurrent_forwards param
- edf70aa refactor forward
- ee1fd55 get prediction request of any asset
- ff7d21d update bittensor lib
- 5712d93 refactor price data provider: token param
Full Changelog: v1.1.6...v1.2.0
v1.1.6
Upgrade process
- Commands:
git fetchgit checkout v1.1.6source bt_venv/bin/activate
- Restart validator
pm2 start validator.local.config.js
Full Changelog: v1.1.5...v1.1.6
v1.1.5
Upgrade process
- Commands:
git fetchgit checkout v1.1.5source bt_venv/bin/activate
- Restart validator
pm2 start validator.local.config.js
What's Changed
- update validator whitelist by @Thykof in #122
- Absolute price Crps calculation with gaps by @davidevdt in #96
- add flake8 and mypy linters by @Thykof in #119
- docs(mod-1516): add new website link and use github markdown for license link by @kieranroneill in #124
- Develop to main by @Thykof in #127
Full Changelog: v1.1.4...v1.1.5
v1.1.4
Synth Validator Upgrade process
- Modify the pm2 config file
- change
--ewma.half_life_days 3.5 - change
--ewma.cutoff_days 7 - change
--softmax.beta -0.005
- change
- Commands:
git fetchgit checkout v1.1.4source bt_venv/bin/activatepip install -r requirements.txtalembic upgrade head
- Restart validator
pm2 start validator.local.config.js
What's Changed
Full Changelog: v1.1.3...v1.1.4