Draft: Revamped Plugin system #21

AKuederle · 2023-04-27T11:31:37Z

This PR implements a new way of how data can be loaded via Plugins.
Instead of creating a loader for a specific file type, this PR allows to create plugins that can generate their own list of data to be loaded.

This is create, if your data is either not linked to a specific file and requires complicated loading logic with multiple files involved, or when you simply want to make it easy for users to just get a specific selection of available datafiles.

@katharina-j-fau and I already used this version to make labling predefined datasets much easier and it also allows for straight forward integration with tpcp-datasets.

The big issue is, that this change to the plugin system is not backwards compatible in the following ways:

All plugins are now class instances and not classes
BaseImporter is renamed to BaseFileImporter
On all plugins the static name is not a get_name method

As I was doing work on the GUI anayway, also intrduced a couple further changes:

(internal) loading of UI files should have less duplicated code
I added Q and W as global shortcuts to shift the viewport left or right by 30% of the current X axis
Fixed some bugs with regards to loading of data

As this is backwards compatible, the biggest issue is, that we need to update all the documentation, which I have not done and don't have a lot of time for at the moment. @katharina-j-fau Would you be interested to help here?

@MalteOlle What do you think of the changes in general? Could you have a look?

Remaining TODOS:

Update documentation with regards to changes in the plugin system
Update docstrings of the gloabl data object
Potentially update load data window, see Draft: Revamped Plugin system #21 (comment)

MalteOlle · 2023-04-27T16:09:11Z

I took some time to go over the code and did not find anything suspicious. However, I did not really understand what's going on, let's talk "offline".

MalteOlle · 2023-04-27T16:15:27Z

I took some time to go over the code and did not find anything suspicious. However, I did not really understand what's going on, let's talk "offline".

Well, looking at the working example it is a bit clearer to me, I just wonder whether we should incorporate the "load from plugin" into the usual loading data dialog. But maybe there are some good reasons not to do it.

MalteOlle · 2023-04-27T15:51:40Z

mad_gui/plugins/base.py

-            This could for example be the main window of the GUI, in case the plugin wants to access something there.
-            However, this is unlikely to be necessary since plugins receive either the single
-            :class:`mad_gui.models.GlobalData` or its attribute `PlotData`, which should be sufficient to do everything.
+    def _configure(self, parent=None, **_) -> Self:


I don't yet get what this can actually be used for

mad_gui/plugins/base.py

MalteOlle · 2023-04-27T16:02:18Z

mad_gui/models/global_data.py

@@ -43,6 +43,9 @@ class GlobalData(BaseStateModel):

    """

+    active_data_loader = Property(None)
+    data_index = Property(None, dtype=int)
+    data_label = Property("", dtype=str)


I did not get what this is - add to docstring?

MalteOlle · 2023-05-03T19:57:53Z

From the beginning I struggled a bit with now having two buttons for importers. Today we discussed that dynamically changing the already existing LoadDataDialog is not as easy, because it is coded in the .ui files and would require too much work. However, what about we make use of the PluginSelectionDialog when clicking on "Load data" in the app and then depending on whether it is a file loader or a dataset loader we spawn either the LoadDataDialog, which is now the FileLoaderDialog, or if it is dataset loader spawns the FromPluginLoaderDialog. What do you think @AKuederle ?

The reason why I am suggesting this is that then we keep only one "load data" button and thus I hope that it may be less confusing.

MalteOlle

two naming suggestions

MalteOlle · 2023-05-03T20:00:32Z

mad_gui/plugins/base.py

+    sampling_rate_hz: float
+
+
+class BaseDataImporter(BasePlugin):


Suggested change

class BaseDataImporter(BasePlugin):

class BaseDatasetImporter(BasePlugin):

MalteOlle · 2023-05-03T20:02:37Z

mad_gui/components/dialogs/plugin_selection/load_data_dialog.py

        """Run this dialog and return the data, that was selected by the user."""
        if self.exec_():
            return self.final_data_, self.loader_
        return None, None
+
+
+class FromPluginLoaderDialog(QDialog):


Suggested change

class FromPluginLoaderDialog(QDialog):

class FromDatasetLoaderDialog(QDialog):

or even remove the From. The reason why I would suggest this is, that also the FileLoaderDialog actually also makes use of plugins and therefore, I think FromPluginLoaderDialog is not pinpointed

remove outdated option of pyinstaller

MalteOlle · 2023-05-03T20:13:47Z

buildin an executable works locally, however the executable cannot start because of

  File "mad_gui\components\dialogs\plugin_selection\plugin_selection_dialog.py", line 14, in <module>
  File "mad_gui\utils\helper.py", line 90, in load_window_from_file
  File "mad_gui\utils\helper.py", line 60, in resource_path
  File "mad_gui\utils\helper.py", line 76, in _get_resource_path
TypeError: descriptor 'replace' for 'str' objects doesn't apply to a 'WindowsPath' object

I'll try and see if I can fix that within the next days

MalteOlle · 2023-06-03T14:52:37Z

buildin an executable works locally, however the executable cannot start because of

  File "mad_gui\components\dialogs\plugin_selection\plugin_selection_dialog.py", line 14, in <module>
  File "mad_gui\utils\helper.py", line 90, in load_window_from_file
  File "mad_gui\utils\helper.py", line 60, in resource_path
  File "mad_gui\utils\helper.py", line 76, in _get_resource_path
TypeError: descriptor 'replace' for 'str' objects doesn't apply to a 'WindowsPath' object

I'll try and see if I can fix that within the next days

Fixed this and also all other builds. Now pipelines for building standalone on windows, macOS, and Ubuntu work again.

AKuederle added 12 commits April 21, 2023 15:19

Renamed a couple of things to make room for other types of loaders

9f5ee7a

Started refactoring to make plugins class instances

0fb016f

Removed pandas deprecation calls

0602746

Implemented general structure for plugin loaders

b230873

refactor and lint

c4e526a

Fixed error when popup was closed without data selection

4462148

Fixed some small bugs and added new keycombo to shift plot

1a38bf9

Some small fixes

22130fe

Save last selected loader plugin

a9e0b54

Fully implemented plugin loading with forward backwards buttons

4a2700c

Remove debug print

7b4a3fd

Update

2858dbd

AKuederle requested a review from MalteOlle April 27, 2023 11:31

AKuederle mentioned this pull request Apr 27, 2023

[FEATURE] Document Keyboard shortcuts #22

Open

MalteOlle reviewed May 3, 2023

View reviewed changes

Update build_windows.yml

af9f6d8

remove outdated option of pyinstaller

MalteOlle added 9 commits May 4, 2023 14:56

Update build_windows.yml

7e8368e

Update build_windows.yml

cf1c001

Update build_windows.yml

41dc15a

Add files via upload

80132bb

Update pyproject.toml

1e70613

Update build_windows.yml

26b0e7f

Try to fix windows standalone

f848eab

Update build_windows.yml

6587bbc

Update build_mac.yml

5facc3e

MalteOlle added 29 commits June 2, 2023 22:35

Update build_ubuntu.yml

eddb218

Update build_mac.yml

d8a7746

try to fix gstreamer installation

828fe92

Update build_ubuntu.yml

d6a402e

Update pyinstaller.spec

cfaa32f

Update build_mac.yml

8b0b74f

Update dodo.py

9c9ddee

Update dodo.py

52d7569

Update dodo.py

63f53d4

Update build_mac.yml

5f11eb4

Update build_mac.yml

23f7813

Update dodo.py

bdfb0a3

Update dodo.py

e68881d

Update dodo.py

da97a0f

Update dodo.py

c93b02f

Update dodo.py

8c99920

Update dodo.py

ed42b74

Update dodo.py

9f0fd91

Update dodo.py

f305e9f

Update pyinstaller.spec

5d82710

Update pyinstaller.spec

e5ea8aa

Update build_mac.yml

65ab59b

Update dodo.py

4dbb1b4

Update dodo.py

811b182

Update dodo.py

5bbe3cf

Update dodo.py

63e1b98

Update build_mac.yml

8b25708

Update build_mac.yml

403e55d

Update dodo.py

704475f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Draft: Revamped Plugin system #21

Draft: Revamped Plugin system #21

AKuederle commented Apr 27, 2023 •

edited by MalteOlle

Loading

MalteOlle commented Apr 27, 2023

MalteOlle commented Apr 27, 2023

MalteOlle Apr 27, 2023

MalteOlle Apr 27, 2023

MalteOlle commented May 3, 2023

MalteOlle left a comment

MalteOlle May 3, 2023

MalteOlle May 3, 2023

MalteOlle commented May 3, 2023

MalteOlle commented Jun 3, 2023

	class BaseDataImporter(BasePlugin):
	class BaseDatasetImporter(BasePlugin):

	class FromPluginLoaderDialog(QDialog):
	class FromDatasetLoaderDialog(QDialog):

Draft: Revamped Plugin system #21

Are you sure you want to change the base?

Draft: Revamped Plugin system #21

Conversation

AKuederle commented Apr 27, 2023 • edited by MalteOlle Loading

MalteOlle commented Apr 27, 2023

MalteOlle commented Apr 27, 2023

MalteOlle Apr 27, 2023

Choose a reason for hiding this comment

MalteOlle Apr 27, 2023

Choose a reason for hiding this comment

MalteOlle commented May 3, 2023

MalteOlle left a comment

Choose a reason for hiding this comment

MalteOlle May 3, 2023

Choose a reason for hiding this comment

MalteOlle May 3, 2023

Choose a reason for hiding this comment

MalteOlle commented May 3, 2023

MalteOlle commented Jun 3, 2023

AKuederle commented Apr 27, 2023 •

edited by MalteOlle

Loading