Add continuous interpoint constraints #625

AVHopp · 2025-08-19T08:41:30Z

This PR adds continuous interpoint constraints, and is the re-visited version of #345.

It is for now in draft state as I am currently re-visiting the old version of this branch and investigate what we can use.

I am also explicitly tagging @StefanPSchmid here as he was interested in this feature and planned to potentially implement a variant for discrete spaces.

Fork for example and userguide:
https://avhopp.github.io/baybe_dev/latest/examples/Constraints_Continuous/interpoint.html and https://avhopp.github.io/baybe_dev/latest/userguide/constraints.html#continuous-constraints

Copilot

Pull Request Overview

This PR adds support for continuous interpoint constraints to BayBE, enabling constraints that apply across all points in a batch rather than to individual points. An interpoint constraint like x_1 + x_2 <= 1 enforces that the sum of all x_1 values plus the sum of all x_2 values in the entire batch must not exceed 1.

Added is_interpoint flag to ContinuousLinearConstraint with corresponding logic changes
Implemented interpoint constraint handling in sampling and optimization methods
Added comprehensive test coverage for various interpoint constraint scenarios

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
baybe/constraints/continuous.py	Added `is_interpoint` field and updated `to_botorch` method to handle interpoint constraint index calculation
baybe/searchspace/continuous.py	Added interpoint constraint detection properties and updated sampling logic to handle batch-aware constraint enforcement
baybe/recommenders/pure/bayesian/botorch.py	Updated recommender to disable sequential optimization and pass batch size for interpoint constraints
tests/constraints/test_constraints_continuous.py	Added comprehensive test cases for interpoint equality/inequality constraints with tolerance validation
tests/conftest.py	Added test constraint definitions for interpoint scenarios
tests/hypothesis_strategies/constraints.py	Updated constraint generation strategy to include interpoint flag
examples/Constraints_Continuous/interpoint.py	Added example demonstrating interpoint constraint usage
docs/userguide/constraints.md	Added documentation explaining interpoint constraints with limitations
CHANGELOG.md	Added feature entry

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

tests/constraints/test_constraints_continuous.py

Scienfitz

Prädikat almost gucci

baybe/constraints/continuous.py

baybe/recommenders/pure/bayesian/botorch.py

baybe/searchspace/continuous.py

docs/userguide/constraints.md

baybe/constraints/continuous.py

baybe/searchspace/continuous.py

baybe/recommenders/pure/bayesian/botorch.py

AdrianSosic

Hi @AVHopp. Here the first batch of comments. Haven't gone through everything yet, so more will follow 🙃

AdrianSosic · 2025-09-09T19:21:51Z

baybe/constraints/continuous.py

+    interpoint constraints do so **across** the points of the batch. That is, an
+    interpoint constraint of the form ``x + y <= 1`` technically encodes
+    ``sum(x_i) + sum(y_i) <= 1`` for all points ``i`` in the batch.


I know what you are trying to say here, but the it's written is really confusing. Because what you write is it encodes "some inequality" and this encoding is done for all points i --> it sort of suggests that there are as many inequalities as there are points i.

I'd rather rephrase it to something like "encodes sum_i(x_i) + sum_i(y_i) <= 1, where x and y are ..." or similar. You know what I mean?

Not entirely sure - but I will try to see if I can simply LaTeX-ify this which should make clear what exactly is meant.

to clarify: if I take your statement word by word and "expand it", I'd get

sum(x_1) + sum(y_1) <= 1

sum(x_2) + sum(y_2) <= 1

...

Since you write that this constraint holds for all i

And since it has never really defined what x_i really is, it could be the case that a reasonable sum operation exists for x_i itself (since the sum also has no index)

I have changed the docstring to a rather minimal example. I guess that people that are interested in this feature would rather have a look at examples/user guide anyway, so I would try to keep the docstring here rather short. Please resolve if you agree or propose a very specific alternative formulation otherwise.

baybe/constraints/continuous.py

AdrianSosic · 2025-09-09T19:32:11Z

baybe/constraints/continuous.py


        from baybe.utils.torch import DTypeFloatTorch

+        # NOTE: The interpoint constraint case requires indices to be a 2-d tensor.


I have some problems with this comment:

It points to a page whose content is going to change (i.e. pointing to latest)

The docstring of optimize_acqf is super long, so it's not immediately clear where to look

--> better create a permalink to the exact line in the botorch github repo and point to that

Finally, when you say "adjustements", it sounds like the non-interpoint case is the default and in order to turn into interpoint indices, you slightly modify the tensor. But these things are really just completely different in nature, which is also reflected by the fact that your code uses two completely independent construction paths

I have moved and changed the comment in 27ee1c9 is it better now?

baybe/constraints/continuous.py

baybe/recommenders/pure/bayesian/botorch.py

baybe/searchspace/continuous.py

baybe/searchspace/core.py

baybe/searchspace/discrete.py

docs/userguide/constraints.md

examples/Constraints_Continuous/interpoint.py

AdrianSosic · 2025-09-10T10:00:41Z

baybe/searchspace/continuous.py

+
+            for c in self.constraints_lin_eq + self.constraints_lin_ineq:
+                if not c.is_interpoint:
+                    param_indices, coefficients, rhs = c.to_botorch(self.parameters)


why has the logic of the intrapoint case changed?

Huh, that might be a left-over. Will investigate.

Ahh, I see: This is relevant for the case of having both inter- and intrapoint constraints: If we do not have any interpoint constraints, we can (and actually will) simply use the old logic, see the if part. If we have both, we however need to manipulate the non-interpoint constraints by copying it for each "row" that we will have in the interpoint constraint. Does this make sense?

Is that a requirement from botorch? I'm just surprised that they then also expect a different format for the "regular" constraints, just because we additionally pass batch constraints. Have you checked that this is really needed? What happens if you pass the intrapoint constraints as 1d versions together with interpoint constraints in 2d?

Note that we are in the place of the polytope sampling here. This requires us to transform/flatten the space in order to be able to sample using interpoint constraints - see here: meta-pytorch/botorch#2468
That is, you cannot hand over interpoint constraints in 2D notation as they are not guaranteed to work - see the discussion resp. the following part of it:

And consequently, as we literally have to reshape the whole space, we also need to reshape the intrapoint (in-)equalities by adjusting their indices.

AdrianSosic · 2025-09-10T10:02:41Z

baybe/searchspace/continuous.py

+                equality_constraints=eq_constraints,
+                inequality_constraints=ineq_constraints,
+            )
+            points = points.reshape(batch_size, points.shape[-1] // batch_size)


why is this a // here?

Because torch explicitly requires int when reshaping:

Ok, I get this part, but then your approach is suboptimal. Because by using //, you are silently dropping potential error cases that could happen if the result of your computation is non-integer!
Just imagine that points.shape[-1] is 5 but batch_size is 2 --> silent bug.

And if you really didn't care about having an explicit specification of the remainder dimension, you could have just used -1, which auto-infers the size, which would have been much simpler. So your version is neither simple, nor logic-safe.

So I'd recommend to use a strict conversion to int and then pass that instead.

Changed this in 0824606 to make it more robust, please resolve if happy :)

see new thread

baybe/recommenders/pure/bayesian/botorch.py

AdrianSosic · 2025-09-11T08:50:23Z

baybe/constraints/continuous.py


+        if batch_size is None and self.is_interpoint:
+            raise RuntimeError(
+                "No ``batch_size`` set but using interpoint constraints."


missing white space at the end --> incorrectly rendered
But I think you can actually just remove the second line – it's an obvious statement (you could add this to "any" error message in our codebase) since error are "always" unintended.

(there are only a few exceptions when such a "useless" error message makes sense, namely when the error is needed to block a certain execution path for type safety that would otherwise be left open, but this is not the case here)

Will remove it. My intention was that this error is something that a user should never be able to see at all - so if this error shows up, it demonstrates that something has gone wrong * internally* - in contrast to stuff like "User is stupid because of miosconfiguration"

Fixed in cae7650

Based on your other comment, I realized that those should then rather be asserts. Will change here, please resolve this comment if you agree with "yes, those should be asserts" - see 84ccdb0

baybe/constraints/continuous.py

AdrianSosic · 2025-09-11T09:00:09Z

baybe/constraints/continuous.py

            torch.tensor(param_indices),
            torch.tensor(
-                [self._multiplier * c for c in self.coefficients], dtype=DTypeFloatTorch
+                [self._multiplier * c for c in coefficients], dtype=DTypeFloatTorch


you're doing unnecessary conversions between lists and tensor here + unnecessary list comprehensions. simply turn the type of coefficients to Tensor at the very top and keep everything in torch. make everything shorter, easier to read, and more efficient

Changed in 58d0a42 - thought about maybe even just having something in the style of coefficients = torch.Tensor(...) is not self:is_interpoint else torch.Tensor(...).repeat ..... but I had the feeling that this actually made readability worse. Let me know what you think :)

baybe/searchspace/continuous.py

Co-authored-by: AdrianSosic <[email protected]>

This reverts commit f458310.

AVHopp · 2025-09-23T13:51:08Z

@AdrianSosic @Scienfitz please see here for the current versions of user guide and example:
https://avhopp.github.io/baybe_dev/latest/examples/Constraints_Continuous/interpoint.html and https://avhopp.github.io/baybe_dev/latest/userguide/constraints.html#continuous-constraints

docs/userguide/constraints.md

Scienfitz · 2025-09-25T14:33:20Z

docs/userguide/constraints.md

+2. **Interpoint constraints:** These constraints model the relationship of different experiments
+across the whole batch (*inter-*) of exeriments (*-points*).
+
+#### Intrapoint constraints


another suggestion you might consider: I personally find it a bit hard to read Itrapoint different form Interpoint. So Why not sue hyphens (or dreadful emdashes if thats correct in this case):

Inter-Point Constraints
Intra-Point Constraints

Idea was to have it consistent with how it is called in the code. I am however also open to have it different in the texts, but would summon @AdrianSosic for his opinion.

Scienfitz · 2025-09-25T14:34:27Z

docs/userguide/constraints.md

+interpoint constraints models constraints **across** the points of the batch.
+That is, an interpoint constraint of the form $x + y <= 1$ encodes
+$$
+\sum_{i\text{ in batch}}x_i + \sum_{i\text{ in batch}}y_i <= 1


doesnt proplery render:

You might also change the index to b to distinguish it from the i used for itra-point constraints

Maybe make this a "large" equation instead of an inline one? This would also make the "in batch" appear below the sigma and thus look nicer

docs/userguide/constraints.md

examples/Constraints_Continuous/interpoint.py

Scienfitz · 2025-09-25T14:55:05Z

examples/Constraints_Continuous/interpoint.py

+results_df = pd.DataFrame(results_log)
+
+# fmt: off
+fig, axs = plt.subplots(1, 2, figsize=(10, 4));


would there be any merit to add a second row of results showing how it would look like unconstrained? Ie we would nicely see the violations off of the required and limit lines

Will play around with that and see

Thought about this, and I do not think that we should add this. Reasons:

The example is about demonstrating that the feature works, not that you will not have this behavior if you do not use the feature

It would add quite some boilerplate code

If we would want to add a more detailed comparison, I would prefer to then re-introduce a true chemical model as we could then say "Look, without the constraint it just uses a lot of solvent/wastes catalyst" (or similar) which cannot be guaranteed if we use random stuff

Scienfitz · 2025-09-25T14:55:43Z

examples/Constraints_Continuous/interpoint.py

+    label="Total",
+)
+plt.axhline(y=60, color="red", linestyle="--", label="Required")
+plt.title("Solvent: Individual + Total")


Individual + Total is kind of obsolete and already part of the caption, perhaps better Solvent (Constrained)?

Addressed in 43e3257 new version is already up in the fork, please resolve if happy.

last minor plot things

make the red dashed line not write over existing lines, ie put it at zorder=0

make the y axis start at 0, otherwise its very deceiving and it looks like the Exp lines do not add up to the Budget

examples/Constraints_Continuous/interpoint.py

AVHopp self-assigned this Aug 19, 2025

This comment was marked as outdated.

Sign in to view

AVHopp force-pushed the feature/continuous_interpoint_constraints branch from de882d4 to 6a2086b Compare August 25, 2025 07:54

AVHopp requested review from Copilot and removed request for Copilot August 25, 2025 07:54

AVHopp marked this pull request as ready for review August 25, 2025 08:24

AVHopp requested review from Scienfitz and AdrianSosic as code owners August 25, 2025 08:24

This comment was marked as outdated.

Sign in to view

Scienfitz requested a review from Copilot August 29, 2025 10:16

Copilot AI reviewed Aug 29, 2025

View reviewed changes

tests/constraints/test_constraints_continuous.py Outdated Show resolved Hide resolved

Scienfitz requested changes Aug 29, 2025

View reviewed changes

AVHopp force-pushed the feature/continuous_interpoint_constraints branch from 92fa396 to c616413 Compare August 29, 2025 13:27

AVHopp force-pushed the feature/continuous_interpoint_constraints branch from efdbde0 to fe7b512 Compare September 8, 2025 06:51

Scienfitz reviewed Sep 9, 2025

View reviewed changes

baybe/recommenders/pure/bayesian/botorch.py Outdated Show resolved Hide resolved

baybe/recommenders/pure/bayesian/botorch.py Show resolved Hide resolved

AdrianSosic reviewed Sep 9, 2025

View reviewed changes

AdrianSosic reviewed Sep 10, 2025

View reviewed changes

AVHopp force-pushed the feature/continuous_interpoint_constraints branch 2 times, most recently from dfc2449 to e79d582 Compare September 10, 2025 14:11

AdrianSosic reviewed Sep 11, 2025

View reviewed changes

AVHopp force-pushed the feature/continuous_interpoint_constraints branch 2 times, most recently from b819abb to 74d546e Compare September 12, 2025 11:44

AVHopp added 7 commits September 23, 2025 10:12

Add is_interpoint field to ContinuousLinearConstraint

7157e3e

Add is_interpoint field to serialization test

9f90b3d

Extend to_botorch function for interpoint constraints

d4773b6

Add batch_size argument to to_botorch calls

c3d6196

Add properties regarding constrainedness of continuous subspaces

3c95db4

Raise error when using interpoint constraints in hybrid spaces

a64580d

Extend _sample_from_polytope function

1be45d8

AVHopp and others added 13 commits September 23, 2025 10:13

Delete interpoint example for now as it will be redone anyway

2abeadd

Change wording of ImcompatibilityError

5450c1c

Make reshaping more robust

76d8300

Improve comment on index shape

bb54733

Use more meanigful batch sizes in tests

f797c11

Use .name when raising the IncompatibilityError

28f5ba7

Remove unnecessary part of error message

416ee03

Remove unnecessary comment

7512b3c

Co-authored-by: AdrianSosic <[email protected]>

Use Tensors for coefficients instead of lists

a38e369

Improve reshaping operation

1c7b78e

Co-authored-by: AdrianSosic <[email protected]>

Replace raised errors by asserts

4042d6d

Revert "Delete interpoint example for now as it will be redone anyway"

3e415df

This reverts commit f458310.

Redo example

a56d018

AVHopp force-pushed the feature/continuous_interpoint_constraints branch from d051cad to 23cf823 Compare September 23, 2025 08:13

AVHopp added 2 commits September 23, 2025 10:48

Rewrite example

0793216

Rewrite userguide

0e8682e

AVHopp force-pushed the feature/continuous_interpoint_constraints branch from 23cf823 to 0e8682e Compare September 23, 2025 08:49

Fix broken links

2c5dc47

AVHopp force-pushed the feature/continuous_interpoint_constraints branch from 3922e6a to 2c5dc47 Compare September 23, 2025 09:47

Fix formatting

a48f56b

Scienfitz reviewed Sep 25, 2025

View reviewed changes

Fix minor typos and formatting

03280fb

AVHopp force-pushed the feature/continuous_interpoint_constraints branch from a570f35 to 03280fb Compare September 26, 2025 08:35

AVHopp added 5 commits September 26, 2025 10:55

Replace chemical model by add_fake_measurements

9f1501c

Remove all_constraints

7af6573

Change role of catalyst and solvent

8e2b944

Fix bounds for catalyst

99adaa4

Improve visualization

1e7ff6d

AVHopp force-pushed the feature/continuous_interpoint_constraints branch from 43e3257 to 1e7ff6d Compare September 26, 2025 10:03


		from baybe.utils.torch import DTypeFloatTorch

		# NOTE: The interpoint constraint case requires indices to be a 2-d tensor.

Add continuous interpoint constraints #625

Are you sure you want to change the base?

Add continuous interpoint constraints #625

Uh oh!

Conversation

AVHopp commented Aug 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Scienfitz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AdrianSosic left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

AVHopp commented Aug 19, 2025 •

edited

Loading

AVHopp Sep 12, 2025 •

edited

Loading