gradient for list of variables

[YingzhenLi]

Is that possible to get gradient for a list of variables? e.g. instead of using
loss_func(w, b, x, y) then gradient = grad(loss_func, [0, 1]),
it might be preferred to use params = [w, b]
loss_func(params, x, y) then gradient = grad(loss_func, [0]).

I ask this because my task is to do MCMC on neural network weights. In the first case I need to specifically define loss_func() with lots of arguments.

As a reference, tensorflow support this simply by tf.gradients(loss, [w1, b1, w2, b2, ...]), and my understanding of pyTorch (which also uses autograd) seems to be that they also support this functionality.

I know in the original autograd package this is solved by lumping all the parameters into a big vector, then partitioning it into weight matrices and bias vectors. This doesn't sound a good solution, especially when using GPU (the original package has no GPU support).

[jermainewang]

This could be solved by adding a special ParamTuple structure. Let me see if I could quickly implement that.