Conjugate gradient for SPEC

[zhucaoxiang]

The request for using the conjugate gradient (CG) method for minimization was raised recently. Here is what I posted at FOCUS repository about my limited experience of using CG. You can have a look and take it as a reference.

---

The current nonlinear conjugate gradient (CG) method was written by myself based on the strategy of Yuan & Dai. The line search subroutine is also written by myself based on Numerical Optimization. It is often observed un-converged line search.
In the develop branch, FOCUS is linked to another well-written library, CG-descent. If you want to use the old version CG, just compile with PFLAGS='-D oldcg'.

Here is a test using the rotating ellipse. Solid lines are the CG-descent, and dashed lines the old CG.

1. Convergence over iterations.
   

2. Convergence over wall-time.
   

3. Poincare plots from CG-descent
   

4. Poincare plots from old CG
   

The testing results show that the CG-descent is much faster in speed. My guess is that I didn't implement a very good line search subroutine. So I will encourage everyone to test the CG-descent and will merge it into master after some time.

git fetch
git checkout develop
make clean; make xfocus; make dfocus

---

[zhisong]

We can also explore other open source scientific libraries such as PETSc, GSL, NLOPT and so on, which contain lots of nonlinear solvers and optimization algorithms. Many of them come with sparse matrices and in-built parallelization.

We could let users choose at compilation between in-build SPEC nonlinear solvers that maybe slower(such as minpack), or external scientific libraries.