Problems with custom gradient

[artem-s-shevchenko]

We have meet a problem when use checkpoints and custom gradients together. We have created custom gradient for operation tf.matrix_solve_ls for mode (fast=False), but if we include tensor MatrixSolveLs in the list of checkpointed tensors, the gradients function in memory_saving_gradient.py tries to use the default gradient and ends up with an error because the gradient is not defined for mode (fast=False). We are using tf 1.9. @yaroslavvb do you have any hints about how to make @tf.custom_gradient work with checkpointing?

[yaroslavvb]

So the monkey-patching logic calls tf.gradients right now. Are you saying that does not respect custom gradients?