Skip to content

Conversation

@Unisay
Copy link
Contributor

@Unisay Unisay commented Nov 26, 2025

Summary

Change valueContains benchmark to use ValueTotalSize wrapper for both container and contained arguments, instead of ValueLogOuterSizeAddLogMaxInnerSize for the container.

This addresses Kenneth's observation that using total size for both arguments will:

  1. Straighten out the curved edge of the benchmark data
  2. Make the early-exit boundary at n₁ = n₂ visible as a straight line

Changes

Benchmark Wrapper Update

  • Changed wrapper from (ValueLogOuterSizeAddLogMaxInnerSize, ValueTotalSize) to (ValueTotalSize, ValueTotalSize)
  • Updated denotation signature in Builtins.hs to match
  • Updated golden signature test file

Cost Model Regeneration

  • Regenerated benchmark data in benching-conway.csv
  • Updated cost model parameters in all JSON files (A, B, C)
  • New model: multiplied_sizes with updated coefficients

Conformance Tests

  • Updated budget expectations for valueContains tests
  • Updated budget expectations for array builtin tests affected by rebase

Technical Context

The valueContains builtin uses Map.isSubmapOfBy with a splitLookup-based divide-and-conquer algorithm. By measuring both arguments using ValueTotalSize (total entry count), the cost model better reflects the actual complexity bounds where early exit occurs when contained size exceeds container size.

Test plan

  • Build compiles with --ghc-options=-Werror
  • Conformance tests updated with new budget expectations
  • Benchmark still generates ~1000 test cases

@Unisay Unisay marked this pull request as draft November 26, 2025 11:46
@Unisay Unisay self-assigned this Nov 26, 2025
@Unisay Unisay added the No Changelog Required Add this to skip the Changelog Check label Nov 26, 2025
@Unisay
Copy link
Contributor Author

Unisay commented Nov 26, 2025

newplot newplot1 newplot2

@Unisay Unisay marked this pull request as ready for review December 1, 2025 14:25
@Unisay Unisay force-pushed the yura/fix-valuecontains-benchmark-comment branch from cf15ee5 to bed890f Compare December 1, 2025 14:32
@Unisay Unisay requested review from ana-pantilie and kwxm December 1, 2025 14:33
Copy link
Contributor

@kwxm kwxm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest using the ValueTotalSize wrapper here. I think that'll straighten out the curved edge of the benchmarking data and will make it easier to see what's going on. If you're trying to check whether something wih a nodes is contained in something with b nodes then you can return immediately if a>b, which is everything on one side of the straight line given by a=b. Also the complexity bounds depend on the total number of nodes.

It might also be worth trying a size measure that adds the number of nodes in the outer map to the total of the numbers of nodes in the inner map since that kind of measures the total number of nodes in the value (see my remark about pretending it's one big tree). I think that in many cases (depending on the benchmark inputs) the number of outer nodes will be a lot smaller than the total number of inner nodes, so maybe adding on the number of outer nodes won't make too much difference.

Change valueContains benchmark to use ValueTotalSize wrapper for both container and contained arguments, instead of ValueLogOuterSizeAddLogMaxInnerSize for the container. This addresses Kenneth's observation that using total size for both will straighten out the curved edge of the benchmark data and make the early-exit boundary at n₁ = n₂ visible as a straight line.
Regenerate benchmark data and cost model parameters for valueContains using the new ValueTotalSize wrapper for both arguments. The new model uses multiplied_sizes with updated coefficients.
Update expected budget values for valueContains and array builtins conformance tests to match the new cost model parameters.
@Unisay Unisay force-pushed the yura/fix-valuecontains-benchmark-comment branch from 280bc38 to 94bc1fd Compare December 3, 2025 15:16
@Unisay Unisay changed the title fix(benchmark): update valueContains comment for isSubmapOfBy algorithm refactor(benchmark): use ValueTotalSize for both valueContains args Dec 3, 2025
@Unisay Unisay requested a review from kwxm December 3, 2025 15:18
@zliu41 zliu41 requested a review from a team December 3, 2025 15:34
Copy link
Contributor

@kwxm kwxm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • I think the model should probably be linear_in_x_and_y instead of multiplied_sizes; try fitting models of both types to some benchmark results and see what happens.

  • We probably don't need so many benchmark inputs (especially for smaller inputs: I think there are a lot of results squashed together near the origin and it's hard to see what's going on there), and maybe they could be a bit more evenly spaced: see this comment.. Also, having fewer inputs would speed up the turnaround on costing experiments. But now that I've said that, you can have lots of values with very different structures that have the same size, so I'm not so sure.

@zliu41
Copy link
Member

zliu41 commented Dec 4, 2025

I think the model should probably be linear_in_x_and_y instead of multiplied_sizes

linear_in_x_and_y when x >= y, and constant otherwise.

@Unisay
Copy link
Contributor Author

Unisay commented Dec 5, 2025

Closing this PR in favor of #7476

@Unisay Unisay closed this Dec 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

No Changelog Required Add this to skip the Changelog Check

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants