Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assertions on HTML (webview) not working #2064

Open
1 task done
testcenter opened this issue Sep 23, 2024 · 0 comments
Open
1 task done

Assertions on HTML (webview) not working #2064

testcenter opened this issue Sep 23, 2024 · 0 comments

Comments

@testcenter
Copy link

Is there an existing issue for this?

  • I have searched the existing issues and didn't find mine.

Steps to reproduce

The problem is related to maestro tests on Android.
The Bitmovin player uses a Webview with HTML/Javascript to visualize the video player UI elements.

  • Clone the Bitmovin Samples Repo
  • Build and install the BasicPlaybackKotlin example app
    • You need to provide a key in the AndroidManifest which can be retrieved via creating a free account on the Bitmovin page
  • Execute following config and flow via maestro
  • the assertion - assertVisible: "en" fails
    • Asserting on a physical device works as expected
    • Asserting on an emulator with google_apis_playstore works as expected
    • Asserting on an emulator with google_apis does not work
    • Asserting on maestro cloud fails. I assume, maestro cloud uses maestro start-device to create a device. It uses google_apis, see DeviceConfig

config.yaml

flows:
  - "DisplaysSubtitles.yaml"

DisplaysSubtitles.yaml

appId: com.bitmovin.player.samples.playback.basic
name: "Playback: Displays subtitles"
---
- launchApp
# toggle settings
- tapOn:
    text: "settings"
    retryTapIfNoChange: false
- extendedWaitUntil:
    visible: "subtitles"
    timeout: 10000
# open subtitle menu
- tapOn: "Select subtitle"
# select subtitle "en"
- tapOn: "en"
# !!! following assertion fails !!!
- assertVisible: "en"
# close settings menu
- tapOn:
    text: "settings"
    retryTapIfNoChange: false
# start playing
- tapOn:
    text: "Play/Pause"
    retryTapIfNoChange: false
# subtitle must be visible
- assertVisible: "Welcome to bitmovin's bitdash!"
- stopApp

Actual results

The assertion - assertVisible: "en" fails although visible. (inside a WebView)
Using google_apis_playstore instead of google_apis would work

Expected results

Assertions behave the same on google_apis and google_apis_playstore (might be a general issue when parsing WebView hierarchy of google_apis vs google_apis_playstore)

About app

  • Android App
  • open source sample app (link provided in the steps to reproduce)
  • native app
  • native framework (android views, xml)
  • WebView (html + javascript) is used to visualize the player UI elements

About environment

java -version

openjdk version "17.0.12" 2024-07-16
OpenJDK Runtime Environment (build 17.0.12+7-Ubuntu-1ubuntu222.04)
OpenJDK 64-Bit Server VM (build 17.0.12+7-Ubuntu-1ubuntu222.04, mixed mode, sharing)

uname -a

Linux ubuntu 5.15.0-119-generic #129-Ubuntu SMP Fri Aug 2 19:25:20 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

please note, the problem also happens on maestro cloud!

Logs

Logs
Waiting for flows to complete...

[Failed] Playback: Displays subtitles (35s) (Assertion is false: "en" is visible)

1/1 Flow Failed

Maestro version

1.38.1

How did you install Maestro?

install script (https://get.maestro.mobile.dev)

Anything else?

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant