Repository navigation
Answer "which node is above this set" from an index - #10
Conversation
intersect_permutation asks two questions of a strong module tree, once per set
in a family the size of the vertex set: which node sits just above the set, and
whether the set is exactly one of the tree's modules. parent_node answered the
first by walking down the tree and materializing the leaf set of every child it
passed, and `X in Ms` answered the second by scanning the whole list of modules
comparing vectors. Both are linear in the tree per question, so the pair cost
Θ(n) per set and Θ(n²) over the family, and together they were most of the
digraph path.
Both are the same lowest-common-ancestor question. The node wanted is the lowest
one whose leaves cover the set, a set's cover is fixed by its lowest and highest
leaf alone, and the set is a module exactly when that node's leaves are no more
numerous than the set. TreeIndex records where each leaf falls in the leaf
order, which range each node spans, and each node's parent, in one pass over the
tree; a query is then O(|S| + depth) and touches no leaf set at all.
The result dictionary is keyed by node numbers rather than by the nodes, which
hash their entire subtree.
The tail that nests the resulting sets into a tree is rebuilt too. It was
searching each parent for each child and filtering the child's leaves out of it,
O(|parent|·|child|) per pair and once per leaf of the child. The sets are
laminar, so sorting them by decreasing size makes the sets holding a leaf that
leaf's ancestors, outermost first, which gives every set its depth in one pass;
walking the tree from there is O(Σ|node|).
That walk has to take a node's own leaves before any of its children, which is
not cosmetic and cost a round of CI to learn. A strong module can fail to be one
of the laminar sets and still have to come out contiguous, and the only thing
holding such a module together is that the leaves belonging to a node directly
are not separated by that node's children. The old code did this incidentally,
appending each child after filtering its leaves out of the parent. Emitting
children where their leaves fall instead splits such modules: on the five vertex
digraph now in test/factorizing.jl the module {1,4} came out at positions 1 and
4.
intersect_permutation on digraphs built by substitution:
n before after
200 0.0142s 0.0007s
400 0.0628s 0.0014s
800 0.2464s 0.0027s
1600 0.9518s 0.0057s 167x
3200 - 0.0132s
~n^1.9 ~n^1.0
End to end that is 7.2510s to 2.6981s at n = 3200. intersect_permutation is now
0.5% of the digraph path rather than most of it; what dominates now is the two
symgraph_factorizing_permutation calls on the dense matrices G|G' and G&G',
which is quadratic because those matrices are.
Trees are unchanged: identical to master over 3000 random graphs and digraphs.
The permutations remain factorizing permutations: 900,000 random digraphs over
six seeds, every one modular, where the bug above shows up about once in 50,000
and the same sweep reports failures against the code without the fix; plus every
digraph on n ≤ 4 exhaustively. TreeIndex agrees with parent_node, and its
exactness flag with `X in Ms`, on all 49,612 (tree, subset) pairs over every
subset of every graph in a random sample.
test/factorizing.jl gains the counterexample and a seeded sweep sized to the
rate the bug occurs at rather than to what is quick.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013w7XdHqa7ynpy1nN1j19Qp
82812b1 to
2672a9c
Compare
|
Force-pushed: CI caught a real bug in the first version of this, and it is worth Walking the laminar family has to take a node's own leaves before any of its The counterexample, now a regression test: G = [0 0 1 0 0
0 0 0 0 0
0 1 0 0 0
0 0 1 0 0
0 1 1 0 0]
The verification I ran was too small to find this, which is the part I should Now:
|
Step 3 of the plan in #3:
intersect_permutation, which #9 left as the digraphpath's bottleneck.
The problem
intersect_permutationasks two questions of a strong module tree, once foreach set in a family the size of the vertex set:
parent_node, which walks down thetree materializing the leaf set of every child it passes;
X in Ms, a linear scanover the whole list of modules comparing vectors.
Both are linear in the tree per question, so the pair costs Θ(n) per set and
Θ(n²) over the family.
Both are the same LCA question
The node wanted is the lowest one whose leaves cover the set; a set's cover is
fixed by its lowest and highest leaf alone; and the set is a module exactly when
that node's leaves are no more numerous than the set. So
TreeIndexrecords,in one pass over the tree, where each leaf falls in the leaf order, which range
of that order each node spans, and each node's parent. A query is then
O(|S| + depth) and touches no leaf set at all.
The result dictionary is keyed by node numbers rather than by the nodes
themselves, which hash their entire subtree.
And the tail that builds the tree
That part searched each parent for each child and filtered the child's leaves
out of it — O(|parent|·|child|) per pair, paid once per leaf of the child. The
sets are laminar, so sorting them by decreasing size makes the sets holding a
given leaf that leaf's ancestors, outermost first, which gives every set its
depth in one pass. Walking the tree from there is O(Σ|node|).
Effect
intersect_permutationon digraphs built by substitution:End to end the digraph path goes 7.2510s → 2.6981s at n = 3200. Where its time
goes now:
G|G',G&G'symgraph_factorizing_permutationStrongModuleTreeintersect_permutationStrongModuleTree(H, p)sort_leaves!intersect_permutationis 0.5% of it, rather than most of it. What dominatesnow is the two factorizations, quadratic because
G .| G'andG .& G'aredense matrices — which is the same "linear in the input you were handed" story
as the symmetric path, since a dense matrix is n² words.
Verification
masterover 3000 random graphs and digraphs.TreeIndexagrees withparent_node, and its exactness flag agrees withX in Ms, on all 49,612 (tree, subset) pairs — every subset of every graph ina random sample.
uniformly random and substituted digraphs, dense and sparse.
🤖 Generated with Claude Code
https://claude.ai/code/session_013w7XdHqa7ynpy1nN1j19Qp