failing to connect to camara after working great for a week

[eckerse]

It was working without issues for a week or so. Then it stopped to work. From the logs it is not able to access the camera and restarts getting new token every few seconds in an infinite loop

cryze_api-1          | info: Cryze.API.PythonExtensions.PythonLogger[0]
cryze_api-1          |       2025-01-29 15:02:15,625 - wyze_sdk.service.base - INFO - requesting POST to https://wyze-mars-service.wyzecam.com/plugin/mars/v2/regist_gw_user/GW_GC1_***
cryze_android_app-1  | 01-29 15:02:12.358 I/chatty  ( 1274): uid=10116(com.github.xerootg.cryze) Thread-24 identical 18 lines
cryze_android_app-1  | 01-29 15:02:12.358 W/IoTVideo-P2P( 1274): iv_gute_frm_rc5_decrypt opt_encrypt=2 frm->opt_resp=0 chkval fail...
cryze_api-1          | Failed to cast req_args to Dictionary`2: cannot convert object to target type
cryze_api-1          | Failed to cast headers to Dictionary`2: cannot convert object to target type
cryze_api-1          | info: Cryze.API.Services.WyzeSdkService[0]
cryze_api-1          |       Got camera token{'code': 1, 'current': 1, 'data': {'accessId': '-9222942308196768942', 'accessToken': '***longtoken***', 'expireTime': 1738767737}, 'hash': '1', 'instance_id': None, 'isShowToast': 0, 'message': 'Success', 'toastMsg': None, 'total': 1, 'version': 1}
cryze_android_app-1  | 01-29 15:02:16.522 D/CryzeHttpClient( 1274): getCameraCredential: Body: {"accessId":-9222942308196768942,"accessToken":"***longtoken***"}
cryze_api-1          | info: Microsoft.AspNetCore.Mvc.Infrastructure.SystemTextJsonResultExecutor[1]
cryze_api-1          |       Executing JsonResult, writing value of type 'Cryze.API.Services.AccessCredential'.
cryze_android_app-1  | 01-29 15:02:16.556 I/NetConfig( 1274): subscribeDevice tid = GW_GC1_*** token = ***longtoken***
cryze_api-1          | info: Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker[105]
cryze_android_app-1  | 01-29 15:02:16.557 W/IoTVideo-P2P( 1274): iv_subscribe_dev token_len=244
cryze_api-1          |       Executed action Cryze.API.Controllers.CameraController.GetCameraToken (cryze_api) in 2465.8412ms
cryze_android_app-1  | 01-29 15:02:16.557 I/NetConfig( 1274): subscribeDevice msgId = -1
cryze_api-1          | info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
cryze_android_app-1  | 01-29 15:02:16.558 E/RestreamingVideoPlayer::CameraInfo(cameraId=GW_GC1_***, streamName=back_yard_cam)( 127PreSubscribe: onError: -1, IOTC_ER_SERVER_NOT_RESPONSE, subscribe device is error:-1
cryze_api-1          |       Executed endpoint 'Cryze.API.Controllers.CameraController.GetCameraToken (cryze_api)'
cryze_android_app-1  | 01-29 15:02:16.558 I/RestreamingVideoPlayer::CameraInfo(cameraId=GW_GC1_***, streamName=back_yard_cam)( 1274Failed to subscribe device, retrying
cryze_api-1          | info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
cryze_api-1          |       Request finished HTTP/1.1 GET http://cryze_api:8080/Camera/CameraToken?deviceId=GW_GC1_*** - 200 - application/json;+charset=utf-8 2473.3843ms

What seems to be the error message is

cryze_android_app-1  | 01-29 15:02:16.558 E/RestreamingVideoPlayer::CameraInfo(cameraId=GW_GC1_***, streamName=back_yard_cam)( 127PreSubscribe: onError: -1, IOTC_ER_SERVER_NOT_RESPONSE, subscribe device is error:-1
cryze_api-1          |       Executed endpoint 'Cryze.API.Controllers.CameraController.GetCameraToken (cryze_api)'
cryze_android_app-1  | 01-29 15:02:16.558 I/RestreamingVideoPlayer::CameraInfo(cameraId=GW_GC1_***, streamName=back_yard_cam)( 1274Failed to subscribe device, retrying

In wyze app camera is still working. FW is 1.0.84. I tried 'Restart camera' and 'Reset Services' in the app. I got a new api token from wyze but doesn't help.
Anything else I could do for troubleshooting?

[xerootg]

1.0.84 was pushed out in october it looks like, did you just get that update? I suppose it's possible that the libraries from wyze for gwell cams in this project needs to be updated to a newer version. Note: I do not have any gwell cameras in use anymore due to just the sheer frustration of trying to keep this whole stack stable, but I can definitely try to update the solution for ya. Also, I'm assuming you restarted the whole stack as part of your troubleshooting?

[eckerse]

Thanks for looking.

I had not updated the camera for the last 2 years, hoping that there would be a community firmware for easy local access at some point. When I got your project working, I was surprised how stable it was and after 2-3 days I updated the camera. Cryze was still working at this point and continued to work after FW update. The last thing I did a day before it ceased working was to disable the notifications in the native wyze app (but I don't think this is relevant).

I'm using proxmox and have a debian VM with docker for cryze. I several times restarted the docker host as well as proxmox itself and even did a kernel update.

Anyway it would be great if you could update the libs.

[xerootg]

hmm, so that leaves only two options - the backend changed (at wyze) and the old library doesn't work anymore OR the token getter for gwell cameras changed. IOTC_ER_SERVER_NOT_RESPONSE is concerning. I don't think it's the firmware version. Updating the native library is sadly not plug and play, they changed the JNI interfaces in the newer SDK. Don't expect a quick turnaround, I have to essentially RE the new android app and move it all back into my kotlin interfaces, as this is based on a version of the wyze app from about two years ago.

[eckerse]

I just saw that in dmesg of docker host are a lot of messages. Never paid attention, only checked logs of docker compose. I assume those are normal. If you think they are relavant, I could send you the full output during container start. Here is the beginning:

[606245.201503] launcher: mkdir("/dev/pts", 0755) failed File exists
[606245.203645] launcher: mount("sysfs", "/sys", "sysfs", 0, NULL) failed Device or resource busy
[606245.208313] launcher: mount("selinuxfs", "/sys/fs/selinux", "selinuxfs", 0, NULL) failed No such file or directory
[606245.210672] launcher: mknod("/dev/kmsg", S_IFCHR | 0600, makedev(1, 11)) failed File exists
[606245.212521] launcher: mknod("/dev/random", S_IFCHR | 0666, makedev(1, 8)) failed File exists
[606245.226229] launcher: mknod("/dev/urandom", S_IFCHR | 0666, makedev(1, 9)) failed File exists
[606245.229228] launcher: mknod("/dev/ptmx", S_IFCHR | 0666, makedev(5, 2)) failed File exists
[606245.232510] launcher: mknod("/dev/null", S_IFCHR | 0666, makedev(1, 3)) failed File exists
[606245.237014] launcher: init first stage started!
[606245.237233] launcher: Unable to open /lib/modules, skipping module loading.
[606245.237404] launcher: [libfs_mgr]ReadFstabFromDt(): failed to read fstab from dt
[606245.237545] launcher: [libfs_mgr]ReadDefaultFstab(): failed to find device default fstab
[606245.237555] launcher: Failed to fstab for first stage mount
[606245.237636] launcher: Using Android DT directory /proc/device-tree/firmware/android/
[606245.237931] launcher: [libfs_mgr]ReadDefaultFstab(): failed to find device default fstab
[606245.237975] launcher: First stage mount skipped (missing/incompatible/empty fstab in device tree)
[606245.238016] launcher: Skipped setting INIT_AVB_VERSION (not in recovery mode)
[606245.274397] init: init second stage started!
[606245.297832] init: Using Android DT directory /proc/device-tree/firmware/android/
[606245.305023] init: Overriding previous 'ro.' property 'ro.dalvik.vm.native.bridge':'0' with new value 'libnb.so'
[606245.305478] init: Couldn't load property file '/factory/factory.prop': open() failed: No such file or directory: No such file or directory
[606245.305986] init: Could not set 'ro.hardware' to 'redroid' while loading .prop filesRead-only property was already set
[606245.308813] init: Setting product property ro.product.brand to 'redroid' (from ro.product.product.brand)
[606245.308840] init: Setting product property ro.product.device to 'redroid_x86_64' (from ro.product.product.device)
[606245.308863] init: Setting product property ro.product.manufacturer to 'redroid' (from ro.product.product.manufacturer)
[606245.308884] init: Setting product property ro.product.model to 'redroid11_x86_64' (from ro.product.product.model)
[606245.308905] init: Setting product property ro.product.name to 'redroid_x86_64' (from ro.product.product.name)
[606245.308937] init: Setting property 'ro.build.fingerprint' to 'redroid/redroid_x86_64/redroid_x86_64:11/RD2A.211001.002/eng.frank.20240527.144006:userdebug/test-keys'
[606245.309136] init: Running restorecon...
[606245.309355] init: Created socket '/dev/socket/property_service', mode 666, user 0, group 0
[606245.315768] init: SetupMountNamespaces done
[606245.316991] init: Forked subcontext for 'u:r:vendor_init:s0' with pid 8
[606245.317131] init: Parsing file /system/etc/init/hw/init.rc...
[606245.317596] init: Added '/init.environ.rc' to import list
[606245.317619] init: Added '/system/etc/init/hw/init.usb.rc' to import list
[606245.317635] init: Added '/init.redroid.rc' to import list
[606245.317657] init: Added '/vendor/etc/init/hw/init.redroid.rc' to import list
[606245.317668] init: Added '/system/etc/init/hw/init.usb.configfs.rc' to import list
[606245.317690] init: Added '/system/etc/init/hw/init.zygote64_32.rc' to import list
[606245.319173] init: Parsing file /init.environ.rc...
[606245.319407] init: Parsing file /system/etc/init/hw/init.usb.rc...
[606245.319790] init: Parsing file /init.redroid.rc...
[606245.319812] init: Unable to read config file '/init.redroid.rc': open() failed: No such file or directory
[606245.319851] init: Parsing file /vendor/etc/init/hw/init.redroid.rc...
[606245.319871] init: Unable to read config file '/vendor/etc/init/hw/init.redroid.rc': open() failed: No such file or directory
[606245.319897] init: Parsing file /system/etc/init/hw/init.usb.configfs.rc...
[606245.325000] init: Parsing file /system/etc/init/hw/init.zygote64_32.rc...
[606245.325387] init: Parsing directory /system/etc/init...
[606245.328862] init: Parsing file /system/etc/init/[email protected]...
[606245.329230] init: Parsing file /system/etc/init/[email protected]...
[606245.329432] init: last valid run-time capability is larger than CAP_LAST_CAP