mpy-cross with package interdependencies

[pn2new]

I am developing an application for an ESP32 target and ran into the following problem:

Usually, I compile packages using mpy-cross before uploading them to the target. Now, I did this with a package that loaded another one. Specifically, it was this file:
https://github.com/miguelgrinberg/microdot/blob/main/src/microdot/utemplate.py. In the first code line, it loads another package:

from utemplate import recompile    # <-- other package is loaded here

_loader = None


class Template:
    """A template object.

    :param template: The filename of the template to render, relative to the
                     configured template directory.
    """
    @classmethod
    def initialize(cls, template_dir='templates',
                   loader_class=recompile.Loader):    # <-- error occurs here

   (...)

When loading this library from my application, I always got an error message saying that 'recompile' was unknown in line 14 (which is the line with loader_class=recompile.Loader). It took me ages to find out that everything worked fine when I used the .py files instead of the .mpy files.

So it seems that during mpy-cross the code from the import was not included.

My question now is:

• is there a way to use mpy-cross in this situation? Did I possibly miss an option to 'include' the content of the package that is loaded ?

[andrewleech]

This confused me a bit when I first looked at it, a file called utemplate importing something from utemplate looked like a circular dependency bug.
However on closer inspection, the link ed file will be imported as microdot.utemplate so its first line is importing a completely separate library. A search of the repo found this: https://github.com/miguelgrinberg/microdot/tree/d7459f23b237f6f6f7753f051333d6095abb61db/libs/common/utemplate

I think it all should work fine after using mpy-cross, however now you'll need to ensure you also compile / copy / install that package as well.

[pn2new]

I have both packages compiled and installed, that is not the problem. Note that everything works when I have the .py files instead.

I am not sure how dependencies are compiled into the mpy file, but It seems that there they are either missing or these structures not properly represented because mpy-cross did not consider the "other" file.

[andrewleech]

mpy_cross doesn't compile dependencies, it just compiles each line into the matching bytecode for the command. This is done exactly the same way each file is compiled from py when running live on the device.
The problem is possibly related to the folder / package structure of the mpy files on the device. After you run mpy-cross on the microdot files, how do you copy them onto the device? Are they still in a microdot folder, such that your main application does import microdot or from microdot import utemplate etc?

[bulletmark]

@pn2new I also always compile all my python source files to .mpy files and only copy those to the device. Therefore I don't want any compilation done on the device, even for templates. So I created mutemplate as an alternative to utemplate, which you may want to consider.