make device factory decorator #597

stan-dot · 2024-06-05T15:56:26Z

Fixes #483

Instructions to reviewer on how to test:

Do thing run i22 beamline
Confirm lazy loading is possible

Checks for reviewer

Would the PR title make sense to a scientist on a set of release notes
If a new device has been added does it follow the standards
If changing the API for a pre-existing device, ensure that any beamlines using this device have updated their Bluesky plans accordingly
Have the connection tests for the relevant beamline(s) been run via dodal connect ${BEAMLINE}

coretl

No idea if the rest of the code is right until some tests are written...

src/dodal/beamlines/i22.py

coretl · 2024-06-07T16:07:52Z

To clarify, the code in the original ticket is an idea, but has never been run or tests. It is expected that it is wrong in some way, and writing tests to exercise it will reveal how it is wrong...

docs/developer/explanations/decisions/0003-added-make-devices-decorator.rst

stan-dot · 2024-06-20T14:14:27Z

do we want to add 'skip device' to the factory too?

we could have just one decorator

coretl · 2024-06-21T09:53:44Z

do we want to add 'skip device' to the factory too?

we could have just one decorator

This is a question for @callumforrester and @DominicOram. I don't know what a skipped device actually is? If we have a decorator here, then maybe we could "skip" it by omitting the decorator?

DominicOram · 2024-06-21T10:14:38Z

skip was designed to let you provide logic on when to not instantiate a device. This is useful for:

Running the i03 devices against s03, where not all the devices are simulated
Running the i03 devices from a workstation, where we do not have PVA access to the beamline

stan-dot · 2024-06-21T10:14:41Z

it's an architectural question. We can have all the logic that belongs to 'making device class instances' inside a bunch of nested decorators that need to be in right order. OR we can have 1 factory class that accepts a dictionary of features (lazy, mock, skip, directory provicder, post-create script) in one place.

coretl · 2024-06-21T10:38:35Z

it's an architectural question. We can have all the logic that belongs to 'making device class instances' inside a bunch of nested decorators that need to be in right order. OR we can have 1 factory class that accepts a dictionary of features (lazy, mock, skip, directory provicder, post-create script) in one place.

I have a weak preference for a single decorator, but defer to @callumforrester and @DominicOram for a decision

DominicOram · 2024-06-21T11:15:19Z

Single decorator works for me.

stan-dot · 2024-06-21T16:20:21Z

ok I'll prepare a proof of concept for a single decorator device_factory with some (hopefully) sensible default args so that we can put the least props every time.

stan-dot · 2024-06-25T18:02:56Z

@coretl please checkout the stuff in here

stan-dot · 2024-06-26T07:31:36Z

device_factory = cast(Callable[..., T], make_fake_device(device_factory))

beamline_utils.py 180

but that is inside ophyd, not sure how it works for ophyd-async

stan-dot · 2024-08-23T08:47:12Z

todos:

help(device) make it comprehensive
docs on top of i22.py
consider the arguments visibility question

stan-dot · 2024-08-23T08:55:44Z

@callumforrester from the blueapi perspective is the feature set enough (nonwithstanding the documentation and testing part, just the API exposed to blueapi)?

Co-authored-by: Tom C (DLS) <[email protected]>

callumforrester

@stan-dot, apologies for not looking at this for a while. See DiamondLightSource/blueapi#440 for blueapi's requirements.

Does this API allow me to ask dodal for a list of all device factory functions for a particular beamline?

Having collected them all, how do I connect them in parallel? Or is that for a later PR?

Tagging @coretl in case he can help.

callumforrester · 2024-08-27T12:22:20Z

src/dodal/common/beamlines/instantiation_behaviour.py

+CONTROLLERS: dict[str, DeviceInitializationController] = {}
+
+
+def instantiation_behaviour(


Should: Apologies if there's already been a discussion around it, but can we change this name? I'm concerned because _behaviour is a redundant suffix (see point 8 here) and because it didn't make sense to me until I read a function it was decorating. Some alternatives:

device_instantiation

device_factory

device_configuration

device_singleton

device_instantiation is bad for a function name here, as device_instantiation is an event that happens whenever we call the waxs() (where waxs = some_device), so a function with that name is confusing
factory is technically true but it's an oop statement fitting for a class, but here it is a decorator
device_configuration is underspecified - this is specifically device_iniitalization_configuration
singleton is inaccurate as it does not prevent by itself more than 1 instance afaik

add_instantiation_behavior_to_device would be my suggestion.
Zen of Python says:

Explicit is better than implicit.

Sparse is better than dense.

Readability counts.

callumforrester · 2024-08-27T12:23:16Z

src/dodal/beamlines/i22.py

-            description="AVT Mako G-507B",
-            distance=(-1.0, "m"),
-        ),
+        metadata_holder=metadata_holder,
        directory_provider=get_directory_provider(),
    )


 @skip_device()


Could: Is @instantiation_behaviour now rich enough that we can replace calls to @skip_device with it?

yes, that is the premise of the function signature, not 100% sure about the implementation details still

callumforrester · 2024-08-27T12:24:07Z

src/dodal/common/beamlines/instantiation_behaviour.py

+
+    def __call__(
+        self,
+        connect=False,


Suggested change

connect=False,

connect: bool = False,

callumforrester · 2024-08-27T12:24:41Z

src/dodal/common/beamlines/instantiation_behaviour.py

+        CONTROLLERS[self.factory.__name__] = self
+        self.device = None
+
+    def see_if_device_is_in_cache(self) -> AnyDevice | None:


Should: I would prefer more conventional terminology here:

Suggested change

def see_if_device_is_in_cache(self) -> AnyDevice | None:

def get_device_if_cached(self) -> AnyDevice | None:

will implement suggestions tomorrow

callumforrester · 2024-08-27T12:25:56Z

src/dodal/common/beamlines/instantiation_behaviour.py

+beamline_prefix = "example:"
+
+
+@instantiation_behaviour(
+    default_use_mock_at_connection=True, default_timeout_for_connect=10
+)
+def new_detector_xyz():
+    """Create an XYZ detector with specific settings."""
+    return XYZDetector(name="det1", prefix=f"{beamline_prefix}xyz:")
+
+
+@instantiation_behaviour(
+    eager=True, default_use_mock_at_connection=True, default_timeout_for_connect=10
+)
+def detector_xyz_variant():
+    """Create an XYZ detector with specific settings."""
+    return XYZDetector(name="det2-variant", prefix=f"{beamline_prefix}xyz:")


Could: Could the example module be in its own file? I'd find that more readable.

sure! any path and name suggestions @callumforrester ?

stan-dot requested a review from callumforrester June 5, 2024 15:56

stan-dot self-assigned this Jun 5, 2024

stan-dot linked an issue Jun 5, 2024 that may be closed by this pull request

Make device_factory decorator to ease making and connecting ophyd-async Devices #483

Open

stan-dot requested a review from coretl June 5, 2024 16:22

stan-dot added documentation Improvements or additions to documentation enhancement New feature or request python Pull requests that update Python code labels Jun 5, 2024

stan-dot force-pushed the 483-make-device_factory-decorator-to-ease-making-and-connecting-ophyd-async-devices branch from 94124b4 to 27c1b79 Compare June 7, 2024 08:12

coretl reviewed Jun 7, 2024

View reviewed changes

src/dodal/beamlines/i22.py Outdated Show resolved Hide resolved

callumforrester reviewed Jun 10, 2024

View reviewed changes

docs/developer/explanations/decisions/0003-added-make-devices-decorator.rst Outdated Show resolved Hide resolved

stan-dot changed the title ~~Initial draft changes~~ make device factory decorator Jun 10, 2024

stan-dot force-pushed the 483-make-device_factory-decorator-to-ease-making-and-connecting-ophyd-async-devices branch 3 times, most recently from f2ad915 to fb5e142 Compare June 12, 2024 14:29

stan-dot force-pushed the 483-make-device_factory-decorator-to-ease-making-and-connecting-ophyd-async-devices branch 4 times, most recently from 26368ca to 6cb1db6 Compare June 20, 2024 14:14

stan-dot force-pushed the 483-make-device_factory-decorator-to-ease-making-and-connecting-ophyd-async-devices branch from 6cb1db6 to 49e3bd2 Compare June 25, 2024 15:27

stan-dot force-pushed the 483-make-device_factory-decorator-to-ease-making-and-connecting-ophyd-async-devices branch from 0ecf386 to 656f918 Compare June 27, 2024 08:13

DiamondJoseph mentioned this pull request Aug 22, 2024

Define standard baseline list to prevent re-definition per-plan #761

Open

stan-dot force-pushed the 483-make-device_factory-decorator-to-ease-making-and-connecting-ophyd-async-devices branch from e436ef6 to f5a3fcf Compare August 23, 2024 08:25

stan-dot and others added 24 commits August 27, 2024 11:25

example beamline with implementation

647f8e2

small cleanup

017da4e

added controller demonstration

904dbac

fix wrong extension

aeb4cee

Update src/dodal/beamlines/example_beamline.py

c1a2914

Co-authored-by: Tom C (DLS) <[email protected]>

Update src/dodal/beamlines/example_beamline.py

552255d

Co-authored-by: Tom C (DLS) <[email protected]>

Update src/dodal/beamlines/example_beamline.py

f921b5c

Co-authored-by: Tom C (DLS) <[email protected]>

Update src/dodal/beamlines/example_beamline.py

324aa32

Co-authored-by: Tom C (DLS) <[email protected]>

fix example beamline into a dataclass

21aca7b

new-i22 as an example

df4af23

respond to feedback

6d7d03f

respond to comments

b7434cf

initial test

6f85265

comment out some tests to check coverage

a285561

lint

28d9c2b

use factory name for active devices dict key

d693434

this cannot be active devices, globals harder to mock

82acab8

amend the test

319af5a

added a red test for caching by name

b05e0f9

fix the tests and add the support for a name flag

95e135c

add the API for blueapi

d9e2729

remove todos

55a9b6e

unused utils deleted for coverage

27fb223

Just one i22 now

3f9227f

stan-dot force-pushed the 483-make-device_factory-decorator-to-ease-making-and-connecting-ophyd-async-devices branch from 16bfb79 to 3f9227f Compare August 27, 2024 10:25

callumforrester requested changes Aug 27, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

make device factory decorator #597

make device factory decorator #597

stan-dot commented Jun 5, 2024

coretl left a comment

coretl commented Jun 7, 2024

stan-dot commented Jun 20, 2024 •

edited

Loading

coretl commented Jun 21, 2024

DominicOram commented Jun 21, 2024

stan-dot commented Jun 21, 2024

coretl commented Jun 21, 2024

DominicOram commented Jun 21, 2024

stan-dot commented Jun 21, 2024

stan-dot commented Jun 25, 2024

stan-dot commented Jun 26, 2024 •

edited

Loading

stan-dot commented Aug 23, 2024

stan-dot commented Aug 23, 2024

callumforrester left a comment

callumforrester Aug 27, 2024

stan-dot Aug 27, 2024

callumforrester Aug 27, 2024

stan-dot Aug 27, 2024

callumforrester Aug 27, 2024

callumforrester Aug 27, 2024

stan-dot Aug 27, 2024

callumforrester Aug 27, 2024

stan-dot Aug 27, 2024 •

edited

Loading

		CONTROLLERS: dict[str, DeviceInitializationController] = {}


		def instantiation_behaviour(

	def see_if_device_is_in_cache(self) -> AnyDevice \| None:
	def get_device_if_cached(self) -> AnyDevice \| None:

make device factory decorator #597

Are you sure you want to change the base?

make device factory decorator #597

Conversation

stan-dot commented Jun 5, 2024

Instructions to reviewer on how to test:

Checks for reviewer

coretl left a comment

Choose a reason for hiding this comment

coretl commented Jun 7, 2024

stan-dot commented Jun 20, 2024 • edited Loading

coretl commented Jun 21, 2024

DominicOram commented Jun 21, 2024

stan-dot commented Jun 21, 2024

coretl commented Jun 21, 2024

DominicOram commented Jun 21, 2024

stan-dot commented Jun 21, 2024

stan-dot commented Jun 25, 2024

stan-dot commented Jun 26, 2024 • edited Loading

stan-dot commented Aug 23, 2024

stan-dot commented Aug 23, 2024

callumforrester left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stan-dot Aug 27, 2024 • edited Loading

Choose a reason for hiding this comment

stan-dot commented Jun 20, 2024 •

edited

Loading

stan-dot commented Jun 26, 2024 •

edited

Loading

stan-dot Aug 27, 2024 •

edited

Loading