Implement opt-in stabilization by computing common operations on log scale #177
ricardoV94
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Description
A naive way to obtain more stable computational graphs is to try to compute as much as possible on the log space and defer exponentiation until the last minute. Here is an example from investigating #90, with a naive implementation of the 2F1 function
For the values
(1, 1, 1, 0.5)
the first function returnsnan
at 100 steps, whereas the second can go up to 1 at least 1 millon steps.We probably don't want to rewrite such graphs by default (?), but would be nice to offer an opt-in rewrite for users.
Note: I don't mean we want to implement 2F1 like this (we don't). It was just used as an example of the type of stabilization rewrite that we could offer.
Beta Was this translation helpful? Give feedback.
All reactions