Isolation Forest follow-up work

[csadorf]

Tracks deferred robustness, scalability, interoperability, API-parity, and cuml.accel work following #8226.

C++ API robustness

• Handle n_rows == 0 in the public C++ scoring path before launching a zero-block CUDA kernel, and use checked launch-dimension casts. Isolation Forest Implementation  #8226 (comment)

Sampling quality and scalability

• Avoid the linear duplicate checks used for row and feature sampling at large max_samples or max_features. Consider a fast path when max_samples == n_rows and a scalable sampling strategy. Isolation Forest Implementation  #8226 (comment)
• Select uniformly among nonconstant features when splitting a node, avoiding sensitivity to column order. Isolation Forest Implementation  #8226 (comment)

Python interoperability and persistence

• Support fitted IsolationForest conversion to scikit-learn #8479
• Use nvForest for inference, remove the native C++ model from the Python estimator, and preserve fitted-model behavior across pickle round trips. Remove native model from Isolation Forest estimators #8493
• Raise NotFittedError from unfitted estimator methods and remove the corresponding common-estimator-check xfail. Raise NotFittedError from unfitted IsolationForest methods #8546
• Remove the unsupported sample_weight and warm_start arguments. FIX Remove spurious IsolationForest arguments #8486
• Expose estimator_, estimators_, and estimators_features_ on the native cuml.ensemble.IsolationForest. These are currently available only after scikit-learn conversion and cuml.accel synchronization through Support fitted IsolationForest conversion to scikit-learn #8483.
• Retain or export each tree's sampled row indices and expose estimators_samples_ on native and converted estimators.
• [CI] IsolationForest as_sklearn tests fail against nightly scikit-learn #8548

cuml.accel support

• [FEA] Add cuml.accel support for IsolationForest #8468

Treelite usage

• Set task_type=kIsolationForest in the Treelite representation.
• Use the built-in exponential_standard_ratio postprocessor in Treelite/nvForest, and remove the transform s(x) = 2^(-E[h(x)] / c(n)) from the Python layer.
• Remove the local workaround for missing n_node_samples in exported isolation forest #8488
  • Populate data_count and data_count_present, then remove the local _invert_average_path_length and _recover_node_sample_counts workaround. Upstream support landed in Export more metadata from export_model() dmlc/treelite#684.

Follow-up to #8226.
Related to #6096.

[JulienAu]

I'd like to take two items from the "Python interoperability and persistence" section, complementary to the cuml.accel proxy discussed in #8468 (which @adityaanikam is taking):

1. Raise NotFittedError from unfitted estimator methods and remove the corresponding common-estimator-check xfail. Small and self-contained; I'll start here.
2. Support conversion of fitted models between cuML and scikit-learn, or document any representations that cannot be transferred. I'll first scope what the native model exposes (whether the fitted forest is reachable via Treelite in a form that can populate sklearn's estimators_), and report findings here before committing to the implementation path, since the outcome decides between "conversion" and "document what cannot be transferred".

If anyone is already on either of these, happy to step back.

[JulienAu]

Scoping findings for "Support conversion of fitted models between cuML and scikit-learn", as promised. Short version: cuML -> sklearn conversion is feasible and essentially exact; I prototyped it end to end against the 26.08.00a171 nightly.

How

Everything needed is already in the Treelite export (as_treelite()):

• Structure: split_feature_id, threshold, children, comparison_op (always <) map directly onto sklearn.tree._tree.Tree node arrays via __setstate__.
• Per-node sample counts (required for sklearn's path-length scoring): data_count is not populated in the export (all zeros), but each leaf_value is exactly depth + average_path_length(n_leaf_samples) with sklearn's _average_path_length convention. Since counts are integers, n_leaf_samples is recoverable by exact inversion, and internal counts by bottom-up sums. Validation: 0 inversion failures, and every root sum equals max_samples (10/10 trees).
• The fitted sklearn estimator additionally needs its private fit caches set: _average_path_length_per_tree, _decision_path_lengths, _max_features, _max_samples (plus the public estimators_features_, max_samples_, offset_, n_features_in_). Since tree nodes carry global feature ids, estimators_features_ = arange(n_features) is correct for every max_features.

Measured parity (cuML GPU model vs reconstructed sklearn model)

config	score_samples max abs diff	predict agreement
n_estimators=100, max_samples=256	1.7e-07	100%
max_features=0.5	1.3e-07	100%
max_samples="auto", contamination=0.05	1.4e-07	100%
bootstrap=True, max_samples=1000	1.2e-07	100%

The differences are float32-rounding level. The converted model also pickles and scores identically after a round trip, which would give check_estimators_pickle a path to pass through conversion (the pickle checkbox above).

Caveats worth recording

• Treelite splits are < while sklearn routes <= left; a sample exactly at a threshold routes differently. Thresholds are drawn from continuous ranges so this did not occur in any run, but it is not impossible with duplicated feature values.
• The apl inversion is a workaround: adjacent-count spacing shrinks like 2/n, so at very large max_samples (order 1e5) the tolerance needs care. The clean implementation would populate data_count in the C++ Treelite export and skip inversion entirely.
• The reverse direction (fitted sklearn -> cuML) is a different shape: the native C++ model has no importer, but treelite.sklearn.import_model handles IsolationForest, so the nvforest inference path already in the estimator could score an imported model. That connects directly to the "Investigate using nvforest for inference" item, so I would treat it as a follow-up rather than part of this change.

Question before I implement

Two viable shapes for _attrs_to_cpu / as_sklearn:

1. Pure-Python now: implement the reconstruction with apl inversion as above, no C++ change. Ships against the current export; the inversion is exact for realistic max_samples.
2. Export counts first: small C++ change to populate data_count in the Treelite export, then the Python side reads counts directly. Cleaner, touches the C++ layer.

I lean toward (2) as the robust path but can deliver (1) immediately; happy to follow whichever you prefer. Prototype script available if useful.

[csadorf]

@JulienAu Thanks for the detailed investigation. If you’re up for it, let’s move forward immediately with the pure-Python implementation. Please include validation around the APL inversion so that ambiguous results fail clearly rather than silently selecting a count.

We can follow up separately by populating data_count in the Treelite export, and then simplify the Python implementation once that is available.

[JulienAu]

Sounds good, thanks for the steer. I'll go with the pure-Python reconstruction and make the APL inversion fail loudly on ambiguous counts (raise rather than pick the nearest), so a borderline case surfaces instead of silently degrading. The data_count export can then simplify it later as you suggest.

I'll have the PR up on Sunday.

[csadorf]

@JulienAu I've created #8479 to track the sync work.

[betatim]

Do we want the "Support estimator_, estimators_, estimators_features_, and estimators_samples_ attributes." bullet point to cover cuml.accel only or should it also cover the cuml instance? I think having support in cuml proper is the intention.

[JulienAu]

cuml proper makes sense to me, and most of it is already within reach: estimators_, estimator_ and estimators_features_ can be lazy properties reusing the reconstruction as_sklearn already does (trees come out of the treelite bytes, and features are global so estimators_features_ is just an arange).
The exception is estimators_samples_: sklearn regenerates it from per tree seeds, and the C++ side does not record which rows each tree sampled, so that one needs a C++ change to export the indices.
For cuml.accel the sync path already gets these through _attrs_to_cpu.