[Benchmark]: Exponential derivatives improvement

[andgoldschmidt]

Feature Description

Background

Often, we rely on automatic differentiation support of the matrix exponential. We currenly rely on expv from ExponentialAction.jl to accomplish this. This might be appropriate---especially because we usually have Jacobian-vector products---but it would be good to benchmark. In particular, see this issue thread and the following comparison.
JuliaDiff/ForwardDiff.jl#174 (comment)
In that thread, there is also a reference to a paper, which---see Thm 2---computes the Jacobian of the matrix exponential using the eigendecomposition. This has some similarity to hermitian_exp(...) in exponential_integrators.jl, and might be a practical way to implement the derivative.

Places where we use an exponential integrator include:

• src/integrators/exponential_integrators.jl
• src/rollouts.jl

See also:
https://docs.sciml.ai/ExponentialUtilities/stable/matrix_exponentials/

Suggested checklist

For integrators.jl

• Implement various SciML matrix exponential derivative via ForwardDiff.jl, following the reference issue comment.
• Implement the analytic matrix exponential derivative using the eigendecomposition. Refactor so that the eigendecomposition is only computed once.
• Copy/Paste code and duplicate integrators using the new SciML and Analytic solutions. Benchmark Jacobian on some named trajectory objects.

For rollouts.jl

• Implement the SciML matrix exponential derivative with a function signature that matches exp.
• Test that rollouts are still differentiable with ForwardDiff (there should be an existing test for this).

Importance

1 (lowest)

What does this feature affect?

• quantum system construction
• problem setup
• problem solution
• problem performance
• solution analysis
• plotting
• documentation
• tests
• other (please specify below)

Other information

No response