tradeGraphs will not render if more than two optimizing parameters are in a strategy

[QuantDevHacks]

Description

Ran a simple optimization of an MACD example using apply.paramset(.), with three paramsets: Fast MA, Slow MA, and Signal MA. When finished, I tried using tradeGraphs(.) on pairs of paramsets, but this results in an error saying Aggregation function missing: defaulting to length, and a plot showing a plane in the parameter space.

quantstrat version is 0.16.9.

Remark: I have verified that I can generate plots from tradeGraphs(.) in cases where there are exactly two paramsets in the model.

Expected behavior

Expected to be able to view plots of performance measures vs two paramsets chosen from the three that are used in the optimization.

Minimal, reproducible example

library(quantstrat)
symbol <- 'SPY'
currency("USD")
stock(symbol, currency="USD", multiplier=1)

initDate <- '2000-12-31'
startDate <- '2001-01-01'
endDate <- '2021-03-31'
initEq <- 1000000   # $1M
shs <- 500

Sys.setenv(TZ="UTC")
getSymbols(symbol, from=startDate, to=endDate, index.class="POSIXct", adjust=TRUE)
etfData <- get(symbol)

maType <- "EMA"  

stratName <- "macd.ema.opt"
portName <- "macd.ema.opt"
acctName <- "macd.ema.opt"
suppressWarnings(rm.strat(stratName)) # reset

# Distribution setup:
fastRange <- seq(10, 16, by=2)
slowRange <- seq(20, 32, by=4)
sigRange <- seq(5, 20, by=3)

initPortf(name=portName, symbols = symbol, initDate=initDate)
initAcct(name=acctName, portfolios=portName,
         initDate=initDate, initEq=initEq)
initOrders(portfolio=portName, initDate=initDate)
strategy(name = stratName, store=TRUE)

add.indicator(strategy = stratName, name = "MACD",
              arguments = list(x=quote(Cl(mktdata)), 
                               nFast = 0, nSlow = 0, nSig = 0),
              label='macd.osc')

add.signal(strategy = stratName, name="sigThreshold",
           arguments=list(column="signal.macd.osc",relationship="gt",threshold=0,cross=TRUE),
           label="signal.gt.zero")

add.signal(strategy = stratName, name="sigThreshold",
           arguments=list(column="signal.macd.osc",relationship="lt",threshold=0,cross=TRUE),
           label="signal.lt.zero")

add.rule(strategy = stratName, name='ruleSignal',
         arguments = list(sigcol="signal.gt.zero", sigval=TRUE, orderqty=shs,
                          ordertype='market',orderside='long'),
         type='enter',label='long_entry')   # ,storefun=FALSE)

add.rule(strategy = stratName, name='ruleSignal',
         arguments = list(sigcol="signal.lt.zero",sigval=TRUE,orderqty='all',
                          ordertype='market'),
         type='exit',label='long_exit')

add.distribution(strategy = stratName,
                 paramset.label = "macd.emas",
                 component.type = "indicator",
                 component.label = "macd.osc",
                 variable = list( nFast = fastRange),
                 label = "macd.fast")

add.distribution(strategy = stratName,
                 paramset.label = "macd.emas",
                 component.type = "indicator",
                 component.label = "macd.osc",
                 variable = list(nSlow = slowRange),
                 label = "macd.slow")

add.distribution(strategy = stratName,
                 paramset.label = "macd.emas",
                 component.type = "indicator",
                 component.label = "macd.osc",
                 variable = list(nSig = sigRange),
                 label = "macd.sig")

results <- apply.paramset(strategy.st = stratName, paramset.label = "macd.emas",
                            portfolio.st=portName, account.st=acctName, nsamples=0)

library(rgl)
library(reshape2)

tradeGraphs(stats = results$tradeStats, 
            free.params = c("macd.fast","macd.sig"),
            statistics = c("Profit.To.Max.Draw")) 

Session Info

Aggregation function missing: defaulting to length

[jaymon0703]

Hi @QuantDevHacks. So whats happening here, is reshape2::recast calls dcast() which calls cast(), both of which have fun.aggregate=NULL for the default argument value. When there is any duplicate in the data being melted and cast in one step, the cast function uses fun.aggregate <- length. If you used values of the free.params which did not overlap the function would correctly sum the statistic of interest.

My thinking at this point is to add a fun.aggregate=sum argument to our call to reshape2::recast. It works, so i will push the change shortly, and bump the version one minor increment.

EDIT - on second thoughts...fun.aggregate=sum might not make sense for a statistic such as "Profit.To.Max.Draw", so passing it to the tradeGraphs() function definition might be necessary...since in some cases sum() makes sense, and in others perhaps mean() or median() makes more sense...

[jaymon0703]

Ok tested my change locally...this is the output...

Would you mind testing @QuantDevHacks?

The code to install the version with the last commit is install_github("braverock/quantstrat", ref = "1e8dbbf").

The code i ran for the call to the newly defined tradeGraphs function is below...

tradeGraphs(stats = results$tradeStats, 
            free.params = c("macd.sig","macd.fast"),
            statistics = c("Profit.To.Max.Draw"),
            fun.aggregate=mean) 

[jaymon0703]

Apologies...i am too quick to type today...i think the optimal solution here is to not use recast() as we do not want any aggregating...this will require a bit more work...

[jaymon0703]

In the spirit of typing too fast, i think we do need to aggregate where there are duplicate column combinations...for example there are multiple observations in which macd.fast and macd.sig are equivalent...so how would we like to present that information...presumably mean or median makes sense.

[jaymon0703]

Ok @QuantDevHacks i have updated the default to use the mean as the aggregation function. This is necessary for scenarios in which the param search covers more than 2 params. I think the mean makes sense. Appreciate your thoughts, and testing the latest commit? You can use install_github("braverock/quantstrat", ref = "f73c3c7").

Using your original example, please note you can benefit from parallel computation for the param search if you add the below 2 lines:

library(doParallel)
registerDoParallel()

And because we set the default to mean there is no need to specify it in the call to tradeGraphs():

tradeGraphs(stats = results$tradeStats, 
            free.params = c("macd.sig","macd.fast"),
            statistics = c("Profit.To.Max.Draw"))

Appreciate your feedback.

EDIT: With your permission i may add your example as a demo...