Issue with candle making and/or resampling candles

[erocitna]

There seems to be an issue with timestamps and resampling ohlcv, which affects both the chart in the UI and calculation of data within strategy. Thanks for taking a look at it.

Here is my log with 5m setting and CandleTester strategy:

2020-04-30 11:46:08 - error: Loading notification class NoNotificationService failed. Loading fallback %s
2020-04-30 11:46:08 - verbose: Fetched ticker of BitMEX
2020-04-30 11:46:08 - info: Active candle makers: ["BITMEX USD_BTC"]
2020-04-30 11:46:08 - info: User specific configurations: none
2020-04-30 11:46:08 - info: Active candle batchers: ["BITMEX USD_BTC: 1 min","BITMEX USD_BTC: 5 min"]
2020-04-30 11:46:08 - verbose: Opening Websocket connection in BitMEX to wss://apiclient
2020-04-30 11:46:08 - verbose: Creating backup of 176 config files
2020-04-30 11:46:08 - info: Restored strategy state of 1-USD_BTC CandleTester from state-CandleTester.json
WebSocket [INFO]: Connected.
WebSocket [INFO]: Connected.
2020-04-30 11:46:08 - info: BitMEX WS Connection opened
2020-04-30 11:46:08 - verbose: BitMEX Client initialized, data is flowing.
2020-04-30 11:46:08 - info: Starting with empty order book snapshot. seqNr 0
2020-04-30 11:47:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 09:46:00: high 8882.00000000, low 8816.50000000, close 8840.00000000, volume 19690514.00000000, trades 5483
2020-04-30 11:47:08 - verbose: Fetched ticker of BitMEX
2020-04-30 11:48:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 09:47:00: high 8878.00000000, low 8840.00000000, close 8866.50000000, volume 9076955.00000000, trades 5096
2020-04-30 11:48:09 - verbose: Fetched ticker of BitMEX
(node:12763) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
2020-04-30 11:48:09 - verbose: Checked BitMEX margin collateral (for possible margin call): 1
2020-04-30 11:49:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 09:48:00: high 8886.50000000, low 8867.00000000, close 8872.00000000, volume 4240718.00000000, trades 4235
WebSocket [ERROR]: WebSocket closed with code 1006
WebSocket [INFO]: Retry in 1000 ms
WebSocket [INFO]: Reconnecting...
WebSocket [INFO]: Connected.
2020-04-30 11:49:10 - verbose: Fetched ticker of BitMEX
2020-04-30 11:50:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 09:49:00: high 8872.50000000, low 8859.50000000, close 8860.00000000, volume 2928570.00000000, trades 2760
2020-04-30 11:50:12 - verbose: Fetched ticker of BitMEX
2020-04-30 11:50:12 - verbose: Checked BitMEX margin collateral (for possible margin call): 1
2020-04-30 11:51:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 09:50:00: high 8859.50000000, low 8755.00000000, close 8792.00000000, volume 33073906.00000000, trades 8997
2020-04-30 11:51:00 - verbose: CandleTester 1 candle: 2020-04-30 11:46:00 CandleTester Market time: 2020-04-30 11:50:59, open 8881.50, close 8792.00, volume 69010663.00, trades 26571
2020-04-30 11:51:15 - verbose: Fetched ticker of BitMEX
2020-04-30 11:52:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 09:51:00: high 8814.50000000, low 8774.00000000, close 8774.50000000, volume 14366201.00000000, trades 7197
WebSocket [ERROR]: WebSocket closed with code 1006
WebSocket [INFO]: Retry in 1000 ms
WebSocket [INFO]: Reconnecting...
WebSocket [INFO]: Connected.
2020-04-30 11:52:14 - verbose: Checked BitMEX margin collateral (for possible margin call): 1
2020-04-30 11:52:19 - verbose: Fetched ticker of BitMEX
2020-04-30 11:53:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 09:52:00: high 8800.00000000, low 8755.00000000, close 8799.50000000, volume 19371439.00000000, trades 9856
2020-04-30 11:53:22 - verbose: Fetched ticker of BitMEX
2020-04-30 11:54:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 09:53:00: high 8809.50000000, low 8791.00000000, close 8809.50000000, volume 7722733.00000000, trades 4629
2020-04-30 11:54:16 - verbose: Checked BitMEX margin collateral (for possible margin call): 1
2020-04-30 11:54:24 - verbose: Fetched ticker of BitMEX
2020-04-30 11:55:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 09:54:00: high 8830.00000000, low 8808.50000000, close 8822.00000000, volume 10260625.00000000, trades 5927
WebSocket [ERROR]: WebSocket closed with code 1006
WebSocket [INFO]: Retry in 1000 ms
WebSocket [INFO]: Reconnecting...
WebSocket [INFO]: Connected.
2020-04-30 11:55:24 - verbose: Fetched ticker of BitMEX
2020-04-30 11:56:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 09:55:00: high 8822.00000000, low 8787.00000000, close 8787.00000000, volume 7302972.00000000, trades 4677
2020-04-30 11:56:00 - verbose: CandleTester 2 candle: 2020-04-30 11:51:00 CandleTester Market time: 2020-04-30 11:55:58, open 8792.50, close 8787.00, volume 59023970.00, trades 32286
2020-04-30 11:56:20 - verbose: Checked BitMEX margin collateral (for possible margin call): 1
2020-04-30 11:56:25 - verbose: Fetched ticker of BitMEX
2020-04-30 11:57:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 09:56:00: high 8791.00000000, low 8747.50000000, close 8763.00000000, volume 15338583.00000000, trades 4303
2020-04-30 11:57:25 - verbose: Fetched ticker of BitMEX
2020-04-30 11:58:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 09:57:00: high 8787.00000000, low 8750.00000000, close 8750.00000000, volume 7355696.00000000, trades 6453
WebSocket [ERROR]: WebSocket closed with code 1006
WebSocket [INFO]: Retry in 1000 ms
WebSocket [INFO]: Reconnecting...
WebSocket [INFO]: Connected.
2020-04-30 11:58:20 - verbose: Checked BitMEX margin collateral (for possible margin call): 1
2020-04-30 11:58:25 - verbose: Fetched ticker of BitMEX
2020-04-30 11:59:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 09:58:00: high 8765.00000000, low 8663.50000000, close 8693.50000000, volume 36610585.00000000, trades 11551
2020-04-30 11:59:25 - verbose: Fetched ticker of BitMEX
2020-04-30 12:00:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 09:59:00: high 8722.00000000, low 8668.00000000, close 8668.00000000, volume 18941875.00000000, trades 4006
2020-04-30 12:00:22 - verbose: Checked BitMEX margin collateral (for possible margin call): 1
2020-04-30 12:00:27 - verbose: Fetched ticker of BitMEX
2020-04-30 12:01:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 10:00:00: high 8668.50000000, low 8554.00000000, close 8625.50000000, volume 51234817.00000000, trades 6660
2020-04-30 12:01:00 - verbose: CandleTester 3 candle: 2020-04-30 11:56:00 CandleTester Market time: 2020-04-30 12:00:58, open 8787.00, close 8625.50, volume 129481556.00, trades 32973
WebSocket [ERROR]: WebSocket closed with code 1006
WebSocket [INFO]: Retry in 1000 ms
WebSocket [INFO]: Reconnecting...
WebSocket [INFO]: Connected.
2020-04-30 12:01:31 - verbose: Fetched ticker of BitMEX
2020-04-30 12:02:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 10:01:00: high 8644.00000000, low 8586.50000000, close 8644.00000000, volume 19721260.00000000, trades 6129
2020-04-30 12:02:23 - verbose: Checked BitMEX margin collateral (for possible margin call): 1
2020-04-30 12:02:33 - verbose: Fetched ticker of BitMEX
2020-04-30 12:03:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 10:02:00: high 8715.50000000, low 8643.50000000, close 8705.50000000, volume 22620195.00000000, trades 7059
2020-04-30 12:03:34 - verbose: Fetched ticker of BitMEX
2020-04-30 12:04:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 10:03:00: high 8744.00000000, low 8688.00000000, close 8706.00000000, volume 20101275.00000000, trades 3574
WebSocket [ERROR]: WebSocket closed with code 1006
WebSocket [INFO]: Retry in 1000 ms
WebSocket [INFO]: Reconnecting...
WebSocket [INFO]: Connected.
2020-04-30 12:04:26 - verbose: Checked BitMEX margin collateral (for possible margin call): 1
2020-04-30 12:04:35 - verbose: Fetched ticker of BitMEX
2020-04-30 12:05:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 10:04:00: high 8724.00000000, low 8695.00000000, close 8723.00000000, volume 8709278.00000000, trades 4841
2020-04-30 12:05:37 - verbose: Fetched ticker of BitMEX
2020-04-30 12:06:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 10:05:00: high 8757.50000000, low 8722.00000000, close 8754.50000000, volume 15073184.00000000, trades 7247
2020-04-30 12:06:00 - verbose: CandleTester 4 candle: 2020-04-30 12:01:00 CandleTester Market time: 2020-04-30 12:05:58, open 8625.50, close 8754.50, volume 86225192.00, trades 28850
2020-04-30 12:06:28 - verbose: Checked BitMEX margin collateral (for possible margin call): 1
2020-04-30 12:06:38 - verbose: Fetched ticker of BitMEX
2020-04-30 12:07:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 10:06:00: high 8757.50000000, low 8737.00000000, close 8740.00000000, volume 10147856.00000000, trades 6088
WebSocket [ERROR]: WebSocket closed with code 1006
WebSocket [INFO]: Retry in 1000 ms
WebSocket [INFO]: Reconnecting...
WebSocket [INFO]: Connected.
2020-04-30 12:07:41 - verbose: Fetched ticker of BitMEX
2020-04-30 12:08:00 - verbose: 1min BITMEX candle USD_BTC at 2020-04-30 10:07:00: high 8762.00000000, low 8736.50000000, close 8750.00000000, volume 5107978.00000000, trades 5532
2020-04-30 12:08:29 - verbose: Checked BitMEX margin collateral (for possible margin call): 1
2020-04-30 12:08:44 - verbose: Fetched ticker of BitMEX

[Ekliptor]

I will look into it. But no high priority as I am unfortunately very busy.

The issue is:

1. WolfBot computes candles itself by incoming trades
2. we have to add trades to the candle until it reaches the candle close time
3. so the candle closes at for example 14:33:59 but we wait until the next minute 14:34:00 - thus the log is a second too late
   (old versions of WolfBot closed candles earlier = too soon and thus had a chance of missing trades in the candle. But the time was correct then)

For people who want to investigate this before (pull requests welcome): The code is in calculateCandles() in CandleMaker.ts

edit: there is a config "updateIndicatorsOnTrade": true - which means the strategy reacts on the most recent candle on every price change. I recommend looking into this as a solution.