Can't create frame

[DominoLeChat]

Hi !
First of all, thank you for your work !
I installed FrameOS using Docker Compose on an arm64 server running Ubuntu 24.04, and I get an error in the logs when the container starts. I can still connect on the web ui, but I can't create a new frame. I tried with Docker Run and I even tried building the image locally, but the same error appears. Here's the log :

INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
 * Restarting with stat
INFO  [alembic.runtime.migration] Running upgrade 7f2a8719a009 -> f8db11069084, frame access
Traceback (most recent call last):
  File "/app/backend/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1965, in _exec_single_context
    self.dialect.do_execute(
  File "/app/backend/.venv/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 921, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: duplicate column name: frame_access

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/backend/.venv/bin/flask", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/app/backend/.venv/lib/python3.11/site-packages/flask/cli.py", line 1063, in main
    cli.main()
  File "/app/backend/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/app/backend/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/.venv/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/.venv/lib/python3.11/site-packages/flask/cli.py", line 357, in decorator
    return __ctx.invoke(f, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/.venv/lib/python3.11/site-packages/flask_migrate/cli.py", line 150, in upgrade
    _upgrade(directory, revision, sql, tag, x_arg)
  File "/app/backend/.venv/lib/python3.11/site-packages/flask_migrate/__init__.py", line 111, in wrapped
    f(*args, **kwargs)
  File "/app/backend/.venv/lib/python3.11/site-packages/flask_migrate/__init__.py", line 200, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/app/backend/.venv/lib/python3.11/site-packages/alembic/command.py", line 385, in upgrade
    script.run_env()
  File "/app/backend/.venv/lib/python3.11/site-packages/alembic/script/base.py", line 578, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/app/backend/.venv/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
    module = load_module_py(module_id, path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/.venv/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 109, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/app/backend/migrations/env.py", line 110, in <module>
    run_migrations_online()
  File "/app/backend/migrations/env.py", line 104, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/app/backend/.venv/lib/python3.11/site-packages/alembic/runtime/environment.py", line 927, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/app/backend/.venv/lib/python3.11/site-packages/alembic/runtime/migration.py", line 626, in run_migrations
    step.migration_fn(**kw)
  File "/app/backend/migrations/versions/f8db11069084_frame_access.py", line 22, in upgrade
    with op.batch_alter_table('frame', schema=None) as batch_op:
  File "/usr/local/lib/python3.11/contextlib.py", line 144, in __exit__
    next(self.gen)
  File "/app/backend/.venv/lib/python3.11/site-packages/alembic/operations/base.py", line 377, in batch_alter_table
    impl.flush()
  File "/app/backend/.venv/lib/python3.11/site-packages/alembic/operations/batch.py", line 113, in flush
    fn(*arg, **kw)
  File "/app/backend/.venv/lib/python3.11/site-packages/alembic/ddl/impl.py", line 322, in add_column
    self._exec(base.AddColumn(table_name, column, schema=schema))
  File "/app/backend/.venv/lib/python3.11/site-packages/alembic/ddl/impl.py", line 193, in _exec
    return conn.execute(  # type: ignore[call-overload]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1412, in execute
    return meth(
           ^^^^^
  File "/app/backend/.venv/lib/python3.11/site-packages/sqlalchemy/sql/ddl.py", line 181, in _execute_on_connection
    return connection._execute_ddl(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1524, in _execute_ddl
    ret = self._execute_context(
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1844, in _execute_context
    return self._exec_single_context(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1984, in _exec_single_context
    self._handle_dbapi_exception(
  File "/app/backend/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2339, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/app/backend/.venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1965, in _exec_single_context
    self.dialect.do_execute(
  File "/app/backend/.venv/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 921, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) duplicate column name: frame_access
[SQL: ALTER TABLE frame ADD COLUMN frame_access VARCHAR(50)]
(Background on this error at: https://sqlalche.me/e/20/e3q8)

[mariusandra]

Hey, thanks for trying FrameOS! Curious to hear what's your use case and/or if you have any feedback for the system as it is.

Regarding this issue, it seems like I had a broken migration. This should be fixed now, and you can try it out when the new docker image lands.

[DominoLeChat]

The new Docker image appears to be working, thank you very much !

I built two little picture frames, each one containing a 7.5inch 640x384 3 colors (black, white and red) e-ink screen, one using a ESP32 running ESPHome to show the music currently playing on my speakers, and the second one using a Raspberry Pi Zero 2W to show a random image every day. I was running Inkycal on the Raspberry Pi previously, but the software doesn't support generating images via AI, and doesn't support landscape mode, only portrait.

I'm however running into another issue, the scenes refuses to deploy on the Raspberry Pi with the error "Low memory detected".
I tried resizing the swap file to 2Gb and expanding the SD Card partitions, but it still refuses to deploy...

2024-05-09 15:22:03
Deploying frame salon with build id qkipcgnalglt
2024-05-09 15:22:03
Connecting via SSH to clement@192.168.0.51 (password)
2024-05-09 15:22:03
Connected via SSH to clement@192.168.0.51
2024-05-09 15:22:03
- Getting target architecture
2024-05-09 15:22:03
> uname -m
2024-05-09 15:22:04
aarch64
2024-05-09 15:22:04
> free -m
2024-05-09 15:22:04
total used free shared buff/cache available
2024-05-09 15:22:04
Mem: 417 125 216 0 124 291
2024-05-09 15:22:04
Swap: 99 0 99
2024-05-09 15:22:04
- Copying build folders
2024-05-09 15:22:04
- Applying local modifications
2024-05-09 15:22:04
- Creating build archive
2024-05-09 15:22:04
- No cross compilation. Generating source code for compilation on frame.
2024-05-09 15:22:04
$ cd /tmp/tmpd5knmx6e/frameos && nimble assets -y && nimble setup && nim compile --os:linux --cpu:arm64 --compileOnly --genScript --nimcache:/tmp/tmpd5knmx6e/build_qkipcgnalglt src/frameos.nim 2>&1
2024-05-09 15:22:05
Downloading https://github.com/wltsmrz/nim_zstd using git
2024-05-09 15:22:07
Installing zstd@0.9.0
2024-05-09 15:22:07
Success: zstd installed successfully.
2024-05-09 15:22:08
Downloading https://github.com/xmonader/nimassets using git
2024-05-09 15:22:10
Installing nimassets@0.2.4
2024-05-09 15:22:10
Building nimassets/nimassets using c backend
2024-05-09 15:22:47
Success: nimassets installed successfully.
2024-05-09 15:22:56
Executing task assets in /tmp/tmpd5knmx6e/frameos/frameos.nimble
2024-05-09 15:22:57
assets/web/index.html ... ok
2024-05-09 15:22:57
assets/web/control.html ... ok
2024-05-09 15:22:57
assets/fonts/Ubuntu-Regular_1.ttf ... ok
2024-05-09 15:22:57
Info: "nimble.paths" is updated. 
2024-05-09 15:22:57
Info: "config.nims" is already set up.
2024-05-09 15:22:58
Hint: used config file '/opt/nim/config/nim.cfg' [Conf]
2024-05-09 15:22:58
Hint: used config file '/opt/nim/config/config.nims' [Conf]
2024-05-09 15:22:58
Hint: used config file '/tmp/tmpd5knmx6e/frameos/nim.cfg' [Conf]
2024-05-09 15:22:58
Hint: used config file '/tmp/tmpd5knmx6e/frameos/config.nims' [Conf]
2024-05-09 15:23:02
...................................................................................................................................................................................................................................................
2024-05-09 15:23:02
/tmp/tmpd5knmx6e/frameos/src/lib/httpclient.nim(345, 35) Hint: 'code' cannot raise 'OverflowDefect' [XCannotRaiseY]
2024-05-09 15:23:03
/tmp/tmpd5knmx6e/frameos/src/lib/httpclient.nim(345, 35) Hint: 'code' cannot raise 'OverflowDefect' [XCannotRaiseY]
2024-05-09 15:23:03
/opt/nim/lib/pure/asyncmacro.nim(361, 14) Hint: 'await' is declared but not used [XDeclaredButNotUsed]
2024-05-09 15:23:04
...........................
2024-05-09 15:23:04
/root/.nimble/pkgs2/jester-0.6.0-a5120cfff525e23be29bb2ff47b6ba2f4b2674a3/jester.nim(1346, 9) Hint: Asynchronous route: myrouter. [User]
2024-05-09 15:23:04
/root/.nimble/pkgs2/jester-0.6.0-a5120cfff525e23be29bb2ff47b6ba2f4b2674a3/jester.nim(866, 9) Hint: 'settings' is declared but not used [XDeclaredButNotUsed]
2024-05-09 15:23:04
/tmp/tmpd5knmx6e/frameos/src/frameos/frameos.nim(1, 36) Warning: imported and not used: 'chroma' [UnusedImport]
2024-05-09 15:23:05
Hint: mm: orc; threads: on; opt: speed; options: -d:release
2024-05-09 15:23:05
127329 lines; 7.258s; 397.184MiB peakmem; proj: /tmp/tmpd5knmx6e/frameos/src/frameos.nim; out: /tmp/tmpd5knmx6e/build_qkipcgnalglt/frameos.json [SuccessX]
2024-05-09 15:23:05
�[1m/tmp/nimble_47/githubcom_xmonadernimassets_0.2.4_d06724dd7b80fb470542ab932f3a94af78fe2eb1/src/nimassets.nim(3, 8) �[0m�[33mWarning: �[0muse the nimble packages `malebolgia`, `taskpools` or `weave` instead; threadpool is deprecated�[36m [Deprecated]�[0m�[0m
2024-05-09 15:23:06
- Low memory detected, stopping FrameOS for compilation
2024-05-09 15:23:06
> sudo service frameos stop
2024-05-09 15:23:10
Failed to stop frameos.service: Unit frameos.service not loaded.
2024-05-09 15:23:10
Command exited with status 5
2024-05-09 15:23:10
SSH connection closed

[mariusandra]

Looks nice! If you get something running, feel free to share in this thread as well :).

Currently compilation takes place in two steps: 1) compile nim -> c, 2) compile c -> binary. The first step is performed on the controller host, the second on the raspberry. It seems like the first stage on the host controller is erroring out. Did you increase memory on the docker host or the device?

Coincidentally I've been exploring memory issues these last days, as my two Raspberry Pi Zero W2-s sometimes crash and need a hard reset. I patched a hole with open file descriptors earlier, but there's still something up with the logger, which will take another few days to patch.

I've noticed that when I run other stuff on the raspberry at the same time (e.g. netdata for monitoring), compilation often fails... however it has always succeeded when I have 400+MB available on the raspberry, with swap disabled. I have however not measured how much RAM is needed on the host.

[DominoLeChat]

Thanks ! I will publish the final result in the thread !
I tried running the controller part on two different servers, one with 8Gb of RAM and another with 16Gb of RAM, and I still get the error "Low memory detected, stopping FrameOS for compilation". I also tried reformatting the SD Card and reinstalling the OS (Raspberry Pi OS Lite 64-bit), on a 64Gb micro SD, but still the same result.
I see that the container is using around 600mb of RAM, and the log says "No cross compilation. Generating source code for compilation on frame". Would it be possible maybe to force compilation on the host rather than on the raspberry ?

Here's the log on the second machine :

2024-05-09 16:53:24
�[1m/tmp/nimble_42/githubcom_xmonadernimassets_0.2.4_d06724dd7b80fb470542ab932f3a94af78fe2eb1/src/nimassets.nim(3, 8) �[0m�[33mWarning: �[0muse the nimble packages `malebolgia`, `taskpools` or `weave` instead; threadpool is deprecated�[36m [Deprecated]�[0m�[0m
2024-05-09 16:53:25
- Low memory detected, stopping FrameOS for compilation
2024-05-09 16:53:25
> sudo service frameos stop
2024-05-09 16:53:28
Failed to stop frameos.service: Unit frameos.service not loaded.
2024-05-09 16:53:28
Command exited with status 5
2024-05-09 16:53:28
SSH connection closed
2024-05-09 16:53:56
Deploying frame salon with build id uomxhwbrsmgs
2024-05-09 16:53:56
Connecting via SSH to clement@192.168.0.51 (password)
2024-05-09 16:53:56
Connected via SSH to clement@192.168.0.51
2024-05-09 16:53:56
- Getting target architecture
2024-05-09 16:53:56
> uname -m
2024-05-09 16:53:57
aarch64
2024-05-09 16:53:57
> free -m
2024-05-09 16:53:57
total used free shared buff/cache available
2024-05-09 16:53:57
Mem: 417 125 223 0 118 291
2024-05-09 16:53:57
Swap: 2047 0 2047
2024-05-09 16:53:57
- Copying build folders
2024-05-09 16:53:57
- Applying local modifications
2024-05-09 16:53:57
- Creating build archive
2024-05-09 16:53:57
- No cross compilation. Generating source code for compilation on frame.
2024-05-09 16:53:57
$ cd /tmp/tmp6ipneiga/frameos && nimble assets -y && nimble setup && nim compile --os:linux --cpu:arm64 --compileOnly --genScript --nimcache:/tmp/tmp6ipneiga/build_uomxhwbrsmgs src/frameos.nim 2>&1
2024-05-09 16:53:58
Executing task assets in /tmp/tmp6ipneiga/frameos/frameos.nimble
2024-05-09 16:53:59
assets/web/index.html ... ok
2024-05-09 16:53:59
assets/web/control.html ... ok
2024-05-09 16:53:59
assets/fonts/Ubuntu-Regular_1.ttf ... ok
2024-05-09 16:53:59
Info: "nimble.paths" is updated.
2024-05-09 16:53:59
Info: "config.nims" is already set up.
2024-05-09 16:53:59
Hint: used config file '/opt/nim/config/nim.cfg' [Conf]
2024-05-09 16:53:59
Hint: used config file '/opt/nim/config/config.nims' [Conf]
2024-05-09 16:53:59
Hint: used config file '/tmp/tmp6ipneiga/frameos/nim.cfg' [Conf]
2024-05-09 16:53:59
Hint: used config file '/tmp/tmp6ipneiga/frameos/config.nims' [Conf] 
2024-05-09 16:54:04
...................................................................................................................................................................................................................................................
2024-05-09 16:54:04
/tmp/tmp6ipneiga/frameos/src/lib/httpclient.nim(345, 35) Hint: 'code' cannot raise 'OverflowDefect' [XCannotRaiseY]
2024-05-09 16:54:04
/tmp/tmp6ipneiga/frameos/src/lib/httpclient.nim(345, 35) Hint: 'code' cannot raise 'OverflowDefect' [XCannotRaiseY]
2024-05-09 16:54:04
/opt/nim/lib/pure/asyncmacro.nim(361, 14) Hint: 'await' is declared but not used [XDeclaredButNotUsed]
2024-05-09 16:54:05
...........................
2024-05-09 16:54:05
/root/.nimble/pkgs2/jester-0.6.0-a5120cfff525e23be29bb2ff47b6ba2f4b2674a3/jester.nim(1346, 9) Hint: Asynchronous route: myrouter. [User]
2024-05-09 16:54:05
/root/.nimble/pkgs2/jester-0.6.0-a5120cfff525e23be29bb2ff47b6ba2f4b2674a3/jester.nim(866, 9) Hint: 'settings' is declared but not used [XDeclaredButNotUsed]
2024-05-09 16:54:05
/tmp/tmp6ipneiga/frameos/src/frameos/frameos.nim(1, 36) Warning: imported and not used: 'chroma' [UnusedImport]
2024-05-09 16:54:06
Hint: mm: orc; threads: on; opt: speed; options: -d:release
2024-05-09 16:54:06
127329 lines; 6.368s; 397.184MiB peakmem; proj: /tmp/tmp6ipneiga/frameos/src/frameos.nim; out: /tmp/tmp6ipneiga/build_uomxhwbrsmgs/frameos.json [SuccessX]
2024-05-09 16:54:06
- Low memory detected, stopping FrameOS for compilation
2024-05-09 16:54:06
> sudo service frameos stop
2024-05-09 16:54:10
Failed to stop frameos.service: Unit frameos.service not loaded.
2024-05-09 16:54:10
Command exited with status 5
2024-05-09 16:54:10
SSH connection closed

[mariusandra]

Oh... I'm mistaken here. I read the log wrong. The first compilation succeeded, yet we didn't even get to the second stage. The bug was this: 795c5da

The new docker image should be out shortly

[mariusandra]

Would it be possible maybe to force compilation on the host rather than on the raspberry ?

I'd love that, but getting the cross compilation toolchain setup isn't trivial. I did get it working once as a proof of concept, but didn't "productize" it yet. I guess stay tuned... 😅

Compiling on a Pi Zero2 is really slow though, so I'm soon past the point of frustration where I'll work on it :).

[DominoLeChat]

Sorry for the late reply, the new docker image works perfectly ! Thank you so much !
I also have another problem, I don't know if I have to create a new issue on Github...
My display (Waveshare 7.5inch e-Paper HAT (B) V1 640x384 black/white/red) is not on the list. There is a Waveshare 7.5" 640x384 option that works, but only in black and white (no red) and a Waveshare 7.5" (BC) 640x384 black/white/yellow that doesn't work (faded image and no red). I think the driver for my display is this one : https://github.com/waveshareteam/e-Paper/blob/master/RaspberryPi_JetsonNano/python/lib/waveshare_epd/epd7in5bc.py. In this python code from Waveshare, the color red seems to be supported.

[mariusandra]

Your display should indeed be the "Waveshare 7.5" (BC) 640x384 black/white/yellow" version. It seems I made a slight mistake in setting them up --> there are two displays, B (red) and C (yellow), both using the same driver. I need to break them up since now it'll generate the red image, but dither it as if it was yellow, likely resulting in odd colors.

However... you report this driver also doesn't work... 🤔. Can you check if it works with the WaveShare C examples, as that's what FrameOS's waveshare drivers are based on. If it works, then FrameOS is probably sending data wrong and we need to look closer. If it doesn't, can you check if it works with waveshare's python examples. The inkycal driver seems to be a direct copy of the waveshare driver, so it probably should. If the Python version works and the C version doesn't, we need to figure out what's different between the two...

[mariusandra]

I just split up the drivers into separate B and C variants. Once that's merged and a new docker image is available, you'll need to select a new driver entry from the list. However I fear it'll be equally broken. The shown dithered image will likely show better, but if nothing appeared on your screen before, that's unlikely to change.

[mariusandra]

Hey @DominoLeChat , were you able to get it working in the end? Did the waveshare C example work?

[DominoLeChat]

Hi,
So I just tried the "Waveshare 7.5" (B) 640x384 black/white/red" and it works !
There are sometimes white lines across the screen, but I think it's a problem with my display.