Ability to have longer history than just the last 24h

[elflo]

Hi there!

Thanks for goxtool, love it! Well-written code, plenty of callbacks to hook into, overall a very cool framework for bots!

I'd like to request a feature to be added which is the ability to pull a longer history than just the last 24h that MtGox sends by default.

Unfortunately, the problem in passing a specific time in the fetch url through the ?since= parameter is that MtGox only sends back a chunk of 100 trades since the requested timestamp.

Therefore, the request_history() function would have to loop the requests by updating its request parameter with the last timestamp received until the requested timeframe is filled, and slot_fullhistory() would maybe have to check there's no duplicates just in case.

A parameter could then be specified in the goxtool.ini file to override/specify the default history span, and a slot_history_complete() that fires when the history is fully downloaded and processed into candles would be the cherry on top :)

I don't know if i'm the only one needing this, but i can certainly think of multiple cases for needing more than just the last 24h history, for example calculating any kind of moving average would be a big one. I don't know if this is something that rates very high on your priority list, but in any case that functionality would be very helpful.

And again, thanks (a lot) for goxtool!

[v3nz3n]

This is something I have been thinking about. It could be implemented at module level (ie in a strategy) which fetches the price history, stores it locally in a file or db and then plots moving averages and other indicators via matplotlib. I have already implemented the plotting, but since you grasp the history logic for MtGox - if you could write the code to fetch the history from a strategy, I could integrate that with my indicator and price history plotting.

[elflo]

I too agree that being able to fetch history with a specific timeframe and duration directly from within the strategy itself and have it returned as ready-to-use candles would be ideal, but the way it is coded now, this would probably mean duplicating a bunch of code into the strategy files, unless you modify the core files themselves which i prefer to avoid.

If, on the other hand, the author decides to implement this functionality, it should not require a lot of changes to the existing code and would of course be much cleaner, so i'd like to wait a bit first, to see if this is something he wants to do relatively soon. Plus, he's a much better coder than me :)

[prof7bit]

Proper implementation of history is the next item on my todo list. The existing History() class was only a quick & dirty hack to be able to visualize the past 24 hours in goxtool, I just needed some data to plot the candlesticks, so I quickly hacked someting up to do this in one afternoon but when I wrote it I soon began to realize that this cannot be the ultimate solution. The only reason I did not yet implement something more powerful and more flexible yet was that none of the strategies I am using myself depends on price history.

I have now begun to implement a replacement that will be able to serve OHLCV data of many different timeframes, that will cache data on disk to not needlessly query the server more than once for the same old historic trades and I am also trying to keep in mind that people might want to process the data wth numpy/scipy or talib if needed. I'm currently developing and testing it as a separate strategy module (for now).

I have also implemented a better way for strategies to directly reference each other and call each others methods, so it would be possible to build any kind of indicator as a separate strategy module and then use it and access its data in any other strategy. I'm currently implementing the new history class as a separate strategy module to have a use case for this concept and see if its viable (but I might move essential parts of it directly into goxapi.py later to replace the old History() class)

[elflo]

That sounds great! One could build a whole library of ready-to-use indicators like that. I believe making the actual History() class a tad more flexible makes sense, if only to be able to display a wider history span than the default last 24h in the terminal. Or remove it from goxapi.py completely, and call the separate history module instead by passing a url and a few parameters. This way you could more easily add other exchanges later on should you decide to, as separate bitstampapi.py or btceapi.py for example?

[acriptis]

I have now begun to implement a replacement that will be able to serve OHLCV data of many different timeframes, >that will cache data on disk to not needlessly query the server more than once for the same old historic trades and I >am also trying to keep in mind that people might want to process the data wth numpy/scipy or talib if needed. I'm >currently developing and testing it as a separate strategy module (for now).

Thank you for your work! Now I use candles of different timeframes by instantiating different histories (with different timeframes) in the strategy file. Are you plannig to make it possible to use different candles (by timeframe) in the unique history instance?

[prof7bit]

Yes, ths will eventually replace the built in History() class (or make it obsolete and eventually I will remove the old one completely)