Repository navigation
Add opt-in exact spline calibration (v7.3.0) - #357
alecloudenback wants to merge 23 commits into
Conversation
A spline fit has one knot per quote, so `R(z) = pv(curve(z), q) - price = 0` is square. `fit(spline, quotes)` without a method, optimizer or solve_kwargs now solves it by Newton's method on the residual Jacobian (one ForwardDiff layer) with a backtracking line search, instead of minimizing the squared loss with Newton/LBFGS on nested-dual Hessians. It reprices every quote to rounding, or throws FitConvergenceError when no curve with the interpolation reprices them. An explicit Fit.Loss still minimizes that loss; an optimizer or solve_kwargs without a method still minimize the squared loss. A knot-curve refit whose knots are the quote maturities solves exactly like the spline fit; other grids keep minimizing. FX.Forwards with a spline or knot-curve foreign curve follow the foreign curve's rule. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nw2FiivhYrC16532c1mJQN
…dual scales Address the review of #357: - Public `Fit.Exact(; maxiters = 100)`: the square repricing solve, the default for spline fits and for refits whose knots are the quote maturities. A refit with `Fit.Exact()` needs one quote per knot. - `optimizer`/`solve_kwargs` configure a `Fit.Loss` fit, so a knot-curve fit takes them only with an explicit one (`Fit.Loss()` is the squared loss); without it they throw an ArgumentError instead of switching the objective. - Residuals are measured against each quote's size on the curve being fitted (the gross value of its fixed cashflows, or its price), not its value at the seed: `ZCBPrice(1e-12, 50.0)` repriced to a relative error of 4e-5 (exact fit) and 0.4 (bootstrap) and now reprices to rounding; `ZCBPrice(-1e-20, 50.0)` returned a curve from the exact fit and now throws. `Fit.Bootstrap()` uses the same rule. - The exact solve builds its ForwardDiff Jacobian configuration once: without one the chunk was picked at run time and the Newton loop was not inferred. Exact fits are 8-20% faster; the bootstrap is unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nw2FiivhYrC16532c1mJQN
The function barrier on the Jacobian configuration compiled the Newton loop for every chunk, so for each number of quotes, adding 0.13-0.21 s to first calls whose quote count the precompile workload does not have. Assert the Jacobian's type instead, so that the loop is inferred and compiled once, and keep the solve's vector arithmetic out of line, so that it is not inlined and re-optimized for each type of quotes. First calls are back to the previous head's (within 0.03 s, interleaved), warm fits keep their 7-15% gain, and the fitted knot rates are bit-identical. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nw2FiivhYrC16532c1mJQN
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nw2FiivhYrC16532c1mJQN
… scale) into feat/least-squares-fit Conflicts in src/fit.jl, resolved with #357's `Fit.Exact` in place of the internal `__ExactFit`: a refit without a method is `Fit.Exact()` when its knots are the quote maturities and `Fit.LeastSquares()` otherwise, and `optimizer`/`solve_kwargs` need an explicit `Fit.Loss`, as in #357. Drop the copy of `__fixed_cashflows` that #357 moved to src/fit_implicit.jl. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nw2FiivhYrC16532c1mJQN
…ares stops registering Address the review of #361: - Fit.LeastSquares(; weights, maxiters = 200) always minimizes, also with knots at the quote maturities: it reaches the exact fit's curve when one reprices the quotes, and the closest curve when none does, where Fit.Exact() throws. It no longer validates the weights and then runs the exact solve, so all-zero weights were ignored there. - It needs at least as many quotes with a positive weight as knots, so all-zero weights throw. - Weights are checked for dual numbers by element: a Vector{Real} of dual weights gave a zero derivative. - Convergence. Levenberg-Marquardt cannot take the gradient cosine below about sqrt(eps), where the sum of squares stops registering a decrease, and the fit accepted only a cosine of 1e-8 there, so fits stalled on that floor depending on rounding (MonotoneConvex on all three CI platforms). On a stall it now takes Gauss-Newton steps while they contract and then, when those contract slowly, Newton steps on the gradient with a Hessian by central differences of the analytic gradient (no second-order duals to compile). A fit is accepted when its last step is at most 1e-10 in knot rates. The exact-repricing stop uses #357's gross-value scale. - Monotone convex and PCHIP curves switch formulas, and a least-squares minimum can lie on a switch, where no smooth method converges. A fit that stops there throws a FitConvergenceError that says so (`Yield.__near_kink` on its remaining step). In a sweep of 20 random 80-bond indices, every linear, quadratic and cubic fit converges (75 of 75), and 13 and 12 of 25 monotone convex and PCHIP fits do; every other one throws the kink error, on Julia 1.13.1 and 1.14.0-alpha1. - The Jacobian configuration is built once per fit, with the Jacobian's type asserted, and the vector arithmetic is out of line, as in #357's exact solve. The 1,000-bond refit keeps its knot rates bit for bit, its first call, and its warm time. - The exact fit's errors point to Fit.LeastSquares() as the closest-curve alternative. - CI: SparseArrays joins the stdlibs in the Downgrade job's skip list, and the Fit.LeastSquares docstring is attached to its struct again. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nw2FiivhYrC16532c1mJQN
|
Thanks for the review. The changes are in 0f6c058 and 06b5a81; 8f6fc67 removes a stray file. CI is green on the head. Seed-relative accuracy. Fixed in the exact solve and in
The line search keeps the seed's sizes, so stopping still does not depend on notionals. Your cases:
These are in A public Solver settings preserve the objective. Curve quality and numerical ownership. Agreed on both. Repricing the quotes says nothing about the curve between them; that is the interpolation's job, which the docs leave to the user's choice of method. On ownership, the exact solve is one Newton loop with a line search (about 100 lines). It is covered by Also in 06b5a81: the Newton loop is now inferred (the Jacobian's chunk was picked at run time). Warm exact fits are 7–15% faster, first calls are unchanged, and knot rates are bit-identical. 🤖 Generated with Claude Code |
… scale) into perf/optim-direct Conflicts resolved with #357's exact solve and this branch's Optim names: `Fit.Exact`'s return codes are Symbols (`:MaxIters`, `:Stalled`, `:InternalLinearSolveFailed`), and the docstrings and tests that pass solver settings use `Optim.Options` keywords (`iterations`) with an explicit `Fit.Loss()`. The v8 migration guide notes that a knot-curve fit takes `optimizer` and `solve_kwargs` only with an explicit `Fit.Loss` (since v7.2). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nw2FiivhYrC16532c1mJQN
fit(spline, quotes; optimizer, solve_kwargs) and the same keywords on a knot-curve refit threw an ArgumentError asking for an explicit Fit.Loss. That broke v7.1 calls in a minor release. Without a method, the default is now Fit.Exact() when no solver settings are given and Fit.Loss() (the squared loss) when they are, which is the v7.1 fit for those calls. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nw2FiivhYrC16532c1mJQN
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nw2FiivhYrC16532c1mJQN
|
Thanks. On the keyword rule: agreed, it shouldn't break 7.x callers. 6b20579 drops the error. The method argument's default now depends on the keywords:
That is what every one of those calls did in v7.1, so 7.2.0 stands, with no deprecation path. The trade-off is the one you raised: adding solver settings to a square fit changes "reprice exactly or throw" into "closest curve". The Locally, the full suite passes on Julia 1.13.1 (14,018 tests), and so does a strict docs build. 🤖 Generated with Claude Code |
The loss fit minimizes locally from fixed starting rates, so "the closest curve" claimed more than it guarantees. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nw2FiivhYrC16532c1mJQN
- fit's signature no longer shows Fit.Loss as the universal default: it lists the methods, and states the defaults (Fit.Exact for spline fits and aligned refits, Fit.Loss otherwise, and Fit.Loss with solver settings). - Fit.Loss opens with what it minimizes and links to fit for the routing. - A failed exact solve is described as missing the repricing tolerance, which can reflect incompatible quotes or nonconvergence, not as proof that no curve reprices the quotes. - A loss fit may reprice every quote, to its optimizer's tolerance, rather than "converges to the same curve". Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nw2FiivhYrC16532c1mJQN
Add opt-in
Fit.Exact(; maxiters = 100)to solve spline repricing equations with the residual Jacobian. A spline gets one knot per quote maturity; an exact refit keeps its knot grid and requires one quote per knot. FX forwards route the method to the foreign curve. Fits throwFitConvergenceErrorwhen they cannot meet the repricing contract.Defaults remain
Fit.Loss(), including refits, FX and optimizer settings. Automatic method selection remains in the separate v8 PR #362; #359 retains the Optim migration.The cleanup removes the redundant method-omitted fit methods, sentinel keywords and FX varargs. Each Fit method owns its documented contract;
fitlinks to them. Exact forwards no unused optimizer placeholders. The exact solver lives infit_exact.jl, and trial construction belongs to Yield. Runic is verified in file mode with CI’s exact 1.13.0 version at every stack head.The quote-size hook is declared in Yield and implemented with all three cashflow traits in
src/quote_size.jl, loaded after projections. Yield no longer imports its parent or depends on the fit layer. Transducer-wrapped contract streams are classified through their underlying contract, so InterestRateSwap uses its gross payments; a regression checks a zero-price swap against its analytic gross value.One quote-size helper now scales exact fits, bootstrap and implied quotes. Built-in cashflow contracts use gross projected payments, including zero-price and curve-dependent instruments; opaque pricing-only custom contracts retain their price/value fallback for compatibility. A required positional quote value handles nonnegative cashflows without reprojecting them. This removes seed-value fallbacks and side-effecting curve thunks. Exact reuses each candidate curve to evaluate its residuals and quote sizes, while keeping the line-search scale fixed at the seed.
Bootstrap intentionally retains a fixed scale within each scalar root search and updates it between searches. Directly solving
f(z)/size(z)breaks bracketing for a valid 1,000-year ZCB priced at1e-12; a regression protects that case. Quote acceptance intentionally rejects invalid tiny/negative prices previously accepted by the old bootstrap check.Independent closed-form zero-coupon values and analytic quote-price Jacobians verify the method. Regressions cover zero-price gross sizes, projected floating cashflows, distant tiny prices, every interpolant and FX. The combined suite covers the implementation; the bootstrap/root-search/implied-quote suites pass 239 assertions on Julia 1.13 and 1.10, including two new pricing-only compatibility cases.
Release target: v7.3.0, in stack #369: #364 → #357 → #361 → #350 → #363.
Combined-stack validation: 16,595 passed, one existing
@test_brokenon Julia 1.13.1, including Aqua and extensions. Fourteen existing default/Loss calibration cases remain bit-identical to v7.2. Strict executed documentation, Runic and diff checks pass. The follow-up passes 551 targeted assertions on Julia 1.10.12: 465 calibration, 64 quote-shift and 22 independent financial checks. Runic 1.13.0 in file mode passes all 391 tracked Julia files across the five branch heads; #357 also passes its standalone 466-assertion suite..