Port to Android 14 #564

grote · 2023-09-25T09:13:04Z

Build Android and Java libraries from AOSP source code that include the hidden APIs required for Seedvault and add them to the (gradle) buildsystem
Identify all upgraded libraries and find matching versions for the non-AOSP buildsystem, bumping their versions as needed.
Upgrade other libraries and include jar binaries as needed.
Go through all backup related AOSP commits since last Android release and learn about changes, adapting Seedvault code as needed, especially for breaking changes preventing compilation.
Include all work in a new pull request creating a new android14 branch of Seedvault.

Related to #433

chirayudesai · 2023-10-06T15:11:28Z

targetSdk 34 - default with Android.bp

chirayudesai · 2023-10-06T15:11:42Z

13 SeedVault builds in 14, launches, but crashes on "Backup Now" with

                         E  FATAL EXCEPTION: main
                         E  Process: com.stevesoltys.seedvault, PID: 17278
                         E  java.lang.RuntimeException: Unable to start service com.stevesoltys.seedvault.storage.StorageBackupService@a68e85d with Intent
                            { cmp=com.stevesoltys.seedvault/.storage.StorageBackupService }: java.lang.SecurityException: Starting FGS with type dataSync c
                            allerApp=ProcessRecord{2d21933 17278:com.stevesoltys.seedvault/u0a121} targetSDK=34 requires permissions: all of the permission
                            s allOf=true [android.permission.FOREGROUND_SERVICE_DATA_SYNC]
                         E      at android.app.ActivityThread.handleServiceArgs(ActivityThread.java:4839)
                         E      at android.app.ActivityThread.-$$Nest$mhandleServiceArgs(Unknown Source:0)
                         E      at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2289)
                         E      at android.os.Handler.dispatchMessage(Handler.java:106)
                         E      at android.os.Looper.loopOnce(Looper.java:205)
                         E      at android.os.Looper.loop(Looper.java:294)
                         E      at android.app.ActivityThread.main(ActivityThread.java:8177)
                         E      at java.lang.reflect.Method.invoke(Native Method)
                         E      at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:552)
                         E      at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:971)
                         E  Caused by: java.lang.SecurityException: Starting FGS with type dataSync callerApp=ProcessRecord{2d21933 17278:com.stevesoltys.s
                            eedvault/u0a121} targetSDK=34 requires permissions: all of the permissions allOf=true [android.permission.FOREGROUND_SERVICE_DA
                            TA_SYNC]
                         E      at android.os.Parcel.createExceptionOrNull(Parcel.java:3057)
                         E      at android.os.Parcel.createException(Parcel.java:3041)
                         E      at android.os.Parcel.readException(Parcel.java:3024)
                         E      at android.os.Parcel.readException(Parcel.java:2966)
                         E      at android.app.IActivityManager$Stub$Proxy.setServiceForeground(IActivityManager.java:6761)
                         E      at android.app.Service.startForeground(Service.java:775)
                         E      at org.calyxos.backup.storage.backup.BackupService.onStartCommand(BackupService.kt:31)
                         E      at android.app.ActivityThread.handleServiceArgs(ActivityThread.java:4821)
                         E      ... 9 more
                         E  Caused by: android.os.RemoteException: Remote stack trace:
                         E      at com.android.server.am.ActiveServices.validateForegroundServiceType(ActiveServices.java:2611)
                         E      at com.android.server.am.ActiveServices.setServiceForegroundInnerLocked(ActiveServices.java:2322)
                         E      at com.android.server.am.ActiveServices.setServiceForegroundLocked(ActiveServices.java:1679)
                         E      at com.android.server.am.ActivityManagerService.setServiceForeground(ActivityManagerService.java:13265)
                         E      at android.app.IActivityManager$Stub.onTransact(IActivityManager.java:3385)

chirayudesai · 2023-10-06T23:38:09Z

Identify all upgraded libraries and find matching versions for the non-AOSP buildsystem, bumping their versions as needed.

Barely any changes. It'll actually be a good thing, to support Android 13 at the same time. Long term, I'd like to use AOSP libs as little as possible anyways, because going an year without an update is bad.

chirayudesai · 2023-10-07T19:09:16Z

Doing the port in two steps

Android.bp - it mostly seems to work, may just need minor changes. Easy to do as we work on it. Android 14 - Basic updates, Android.bp #574
gradle - looks like this will need a bit more work and testing. WiP Gradle updates to match AOSP 14 #576

grote · 2023-10-09T12:32:01Z

Long term, I'd like to use AOSP libs as little as possible anyways, because going an year without an update is bad.

How would we do this? AFAIK we need to build inside AOSP, right? So bundle all dependencies?

chirayudesai · 2023-10-09T13:04:14Z

How would we do this? AFAIK we need to build inside AOSP, right? So bundle all dependencies?

Yes. Using https://github.com/lineage-next/gradle-generatebp

We don't have to do them all right now, but we can start with anything that's really old / needs upgrading due to some reason or the other.

grote · 2023-10-09T16:05:07Z

First findings from going through commit messages:

timeout reduced to 5min (from 10min) "our data shows this is a
reasonable value." We should check if this affects our hook-ins like long running APK backups.
https://gitlab.com/CalyxOS/platform_frameworks_base/-/commit/3f82689d2b59f1437479ccb3fb69e662a98ac2fb
BackupAndRestoreFeatureFlags for setting custom timeouts via server syncs.
Maybe they noticed that the 5min sync was sometimes too short?
adb shell device_config put backup_and_restore backup_transport_callback_timeout_millis 1
https://gitlab.com/CalyxOS/platform_frameworks_base/-/commit/a7de6bf8bf195d300871dcd45d503240de32ba35
backuplib related feature confirmed cancelled: https://gitlab.com/CalyxOS/platform_frameworks_base/-/commit/b6faae9f7d1a9af3a4c548c5e01f72a53d1404ea

chirayudesai · 2023-10-09T16:07:44Z

BackupAndRestoreFeatureFlags for setting custom timeouts via server syncs.
Maybe they noticed that the 5min sync was sometimes too short?
adb shell device_config put backup_and_restore backup_transport_callback_timeout_millis 1

We can control this using https://github.com/CalyxOS/vendor_calyx/blob/android13/overlay/common/packages/apps/SimpleDeviceConfig/res/values/config.xml which uses on

grote · 2023-10-10T19:29:26Z

More findings from going through commit messages:

We seem to be able to get logs now from BackupAgents via the BackupMonitor
https://gitlab.com/CalyxOS/platform_frameworks_base/-/commit/104604674f01be4cf4b9d35d38ed14bd1046e987
OperationType (used for d2d) is now BackupDestination
https://gitlab.com/CalyxOS/platform_frameworks_base/-/commit/0828847a7e690520c9e02675e1e6f2cd5bf92e09
Depending on FeatureFlagUtils.SETTINGS_USE_NEW_BACKUP_ELIGIBILITY_RULES the system may ask the backup transport for which BackupDestination type it uses. This may make OS patches superfluous.
There is CompatChanges.isChangeEnabled(IGNORE_ALLOW_BACKUP_IN_D2D) which is probably enabled by default now
We can now disable framework scheduling of backups 🎉
IBackupManager#setFrameworkSchedulingEnabledForUser()
We have currently have these use-cases lined up for this:
- not trying to back-up when location is on flash drive
- not trying to back-up when location is on local server while in different network
- doing our own scheduling, so we can control BackupDestination without OS patches
- doing our own scheduling, so users can get more control about when backups are done
https://gitlab.com/CalyxOS/platform_frameworks_base/-/commit/748c64d9dd728cdbaf8004c8be1b4f372c4cd624
Support wallpaper backup for non-system users
https://gitlab.com/CalyxOS/platform_frameworks_base/-/commit/a48c085e564d380693180c36db1f99f1ab935dfa
Buffer size of full backups can be increased via DeviceConfig for better performance:
https://gitlab.com/CalyxOS/platform_frameworks_base/-/commit/f859b257dc35342a95dea4d15d9ab2c7c887e188
flag to allow the restore pipeline to continue, when a transport level error happens in K/V restore, defaults to true
https://gitlab.com/CalyxOS/platform_frameworks_base/-/commit/cedd0bec7dceaf038d26fff91065f6b827dff11a
we can provide a BackupManagerMonitor to the system for logging/debugging purposes
https://gitlab.com/CalyxOS/platform_frameworks_base/-/commit/c6215c0cf4bff479ca8d60aee8eca2c9df57f65b
https://gitlab.com/CalyxOS/platform_frameworks_base/-/commit/8c4d80519ba89297350d1a8498f023e7aa65c7ef

tl;dr nothing we need urgently react to before making a 14 release. The only thing we might want to check is the shorter timeouts, i.e. by doing a slow APK backup taking more than 5min.

grote · 2023-10-11T18:12:15Z

The bad news is that we are affected by the timeouts. Our backup transport service gets destroyed and the backups fails.

Error during PM metadata backup
    com.android.server.backup.keyvalue.TaskException
        at com.android.server.backup.keyvalue.KeyValueBackupTask.handleTransportStatus(KeyValueBackupTask.java:1138)
        at com.android.server.backup.keyvalue.KeyValueBackupTask.sendDataToTransport(KeyValueBackupTask.java:1072)
        at com.android.server.backup.keyvalue.KeyValueBackupTask.sendDataToTransport(KeyValueBackupTask.java:577)
        at com.android.server.backup.keyvalue.KeyValueBackupTask.backupPm(KeyValueBackupTask.java:682)
        at com.android.server.backup.keyvalue.KeyValueBackupTask.run(KeyValueBackupTask.java:368)

However, our APK backup code (which I used for testing) keeps running and updates the notification which is bad.

There's two places we hook in to back up APKs:

When finishBackup() gets called for @pm@ backup to back up APKs of apps that don't allow backup
When getBackupQuota() gets called for apps that do allow backup

The second one is less likely to run into timeouts, but the first one is uploading potentially many large APKs to a slow network storage which may take longer than the timeout we have. Making a @pm@ backup fail let's the system react harshly usually asking for a full re-init of the backup location. So that's also not good.

As a temporary measure, we could increase both DeviceConfig timeouts on a ROM level to a safer default:

device_config put backup_and_restore backup_transport_future_timeout_millis 900000
device_config put backup_and_restore backup_transport_callback_timeout_millis 900000

A long-term solution could be to move to our own backup scheduling and do APK backups as part of it, so we don't need to hack into places that are not mean to be hacked into. This would also help with the fact that our hacked in code in finishBackup() survives destruction of the backup transport service.

chirayudesai · 2023-10-11T22:51:06Z

As a temporary measure, we could increase both DeviceConfig timeouts on a ROM level to a safer default:

https://review.calyxos.org/c/CalyxOS/vendor_calyx/+/22132 overlay: SimpleDeviceConfig: Adjust values for SeedVault Backup usage

* https://android.googlesource.com/platform/frameworks/base/+/3f82689d2b59f1437479ccb3fb69e662a98ac2fb timeout reduced to 5min (from 10min) "our data shows this is a reasonable value." * However, this affects us, our backup transport service gets destroyed and the backups fails. * Android 13 had 600000, this is even a bit higher to make it less likely that users run into this problem. Ref: * seedvault-app/seedvault#564 (comment) * seedvault-app/seedvault#583 Test: adb shell device_config list backup_and_restore Change-Id: Iefe7d5f2ef53300e3368aa6e809b72dd0b0d1db5

* https://android.googlesource.com/platform/frameworks/base/+/3f82689d2b59f1437479ccb3fb69e662a98ac2fb timeout reduced to 5min (from 10min) "our data shows this is a reasonable value." * However, this affects us, our backup transport service gets destroyed and the backups fails. * Android 13 had 600000, this is even a bit higher to make it less likely that users run into this problem. Ref: * seedvault-app/seedvault#564 (comment) * seedvault-app/seedvault#583 Test: adb shell device_config list backup_and_restore Change-Id: Iefe7d5f2ef53300e3368aa6e809b72dd0b0d1db5 Signed-off-by: NRanjan-17 <[email protected]>

* https://android.googlesource.com/platform/frameworks/base/+/3f82689d2b59f1437479ccb3fb69e662a98ac2fb timeout reduced to 5min (from 10min) "our data shows this is a reasonable value." * However, this affects us, our backup transport service gets destroyed and the backups fails. * Android 13 had 600000, this is even a bit higher to make it less likely that users run into this problem. Ref: * seedvault-app/seedvault#564 (comment) * seedvault-app/seedvault#583 Test: adb shell device_config list backup_and_restore Change-Id: Iefe7d5f2ef53300e3368aa6e809b72dd0b0d1db5

grote self-assigned this Sep 25, 2023

chirayudesai mentioned this issue Oct 6, 2023

Android 14 - Basic updates, Android.bp #574

Merged

grote mentioned this issue Oct 10, 2023

Version upgrades of included libraries and test dependencies #581

Merged

This was linked to pull requests Oct 10, 2023

Version upgrades of included libraries and test dependencies #581

Merged

Get app working in Android 14, built via Android.bp #579

Merged

Gradle updates to match AOSP 14 #576

Merged

Android 14 - Basic updates, Android.bp #574

Merged

grote closed this as completed in #581 Oct 17, 2023

grote mentioned this issue Nov 7, 2023

Backup of many APKs time out and make backup fail #590

Closed

grote mentioned this issue Sep 5, 2024

Port to Android 15 #737

Closed

5 tasks

grote mentioned this issue Sep 24, 2024

List of apps that always show backup errors (use for inherently broken apps only) #147

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Port to Android 14 #564

Port to Android 14 #564

grote commented Sep 25, 2023 •

edited

Loading

chirayudesai commented Oct 6, 2023 •

edited by grote

Loading

chirayudesai commented Oct 6, 2023 •

edited

Loading

chirayudesai commented Oct 6, 2023

chirayudesai commented Oct 7, 2023 •

edited

Loading

grote commented Oct 9, 2023

chirayudesai commented Oct 9, 2023

grote commented Oct 9, 2023

chirayudesai commented Oct 9, 2023

grote commented Oct 10, 2023

grote commented Oct 11, 2023

chirayudesai commented Oct 11, 2023

Port to Android 14 #564

Port to Android 14 #564

Comments

grote commented Sep 25, 2023 • edited Loading

chirayudesai commented Oct 6, 2023 • edited by grote Loading

chirayudesai commented Oct 6, 2023 • edited Loading

chirayudesai commented Oct 6, 2023

chirayudesai commented Oct 7, 2023 • edited Loading

grote commented Oct 9, 2023

chirayudesai commented Oct 9, 2023

grote commented Oct 9, 2023

chirayudesai commented Oct 9, 2023

grote commented Oct 10, 2023

grote commented Oct 11, 2023

chirayudesai commented Oct 11, 2023

grote commented Sep 25, 2023 •

edited

Loading

chirayudesai commented Oct 6, 2023 •

edited by grote

Loading

chirayudesai commented Oct 6, 2023 •

edited

Loading

chirayudesai commented Oct 7, 2023 •

edited

Loading