Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions .github/actions/setup-playwright/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: Setup Python + Playwright
description: Common setup for all Playwright-based smoke tests
inputs:
python-version:
description: Version of Python to use
required: false
default: "3.11"
exclude_inputs:
description: Comma-separated list of inputs to exclude from masking
required: false
default: target_url,course_id

runs:
using: composite
steps:
- name: Cache Python & Playwright dependencies
uses: actions/cache@v4
with:
path: |
~/.cache/pip
~/.cache/ms-playwright
key: >
${{ runner.os }}-py-${{ inputs.python-version }}-playwright-${{
hashFiles('actions-hub/.github/actions/setup-playwright/requirements.txt')
}}
restore-keys: |
${{ runner.os }}-py-${{ inputs.python-version }}-playwright

- name: Set up Python

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add here a cache step like this.
https://github.com/nelc/eox-nelp/blob/76fdc2245533faa2a5f6f34fdd5476854dc706f4/.github/workflows/tests.yml#L22-L32

Also we could add a integration requirements file to add eg pytest-playwright. And try to dont reconfigure python and the pip requirement for each test.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}

- name: Install dependencies
run: |
pip install -r ./actions-hub/.github/actions/setup-playwright/requirements.txt
playwright install
shell: bash

- name: Hide the inputs values to keep them private in the logs
uses: levibostian/action-hide-sensitive-inputs@v1
with:
exclude_inputs: ${{ inputs.exclude_inputs }}
1 change: 1 addition & 0 deletions .github/actions/setup-playwright/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
pytest-playwright

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please pin to a version to manage the hit or not of the cache. =)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This version cannot be pin for two reason

  1. python version is variable so we cannot ensure that the pinned version will be compatible with all the python versions
  2. Pinned versions are just useful when there are conflicts and pip should install the last available version compatible with the current version

The cache is handled by the python version if that changes the cache shouldn't return anything

171 changes: 171 additions & 0 deletions .github/workflows/smoke-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
name: Smoke Tests

on:
workflow_call:
inputs:
run_test:
description: Select desired test
required: true
type: string
target_url:
description: Base URL for the application
required: true
type: string
user_email:
description: Test user email
required: true
type: string
user_password:
description: Test user password
required: true
type: string
course_id:
description: Course ID to test viewing the course page
required: true
type: string
target_plugins:
description: Desired plugins to validate
required: true
type: string
unit_id:
description: Unit ID to test complete-multiplechoice-unit
required: true
type: string


jobs:
check-heartbeat:
if: ${{ inputs.run_test == 'All' || inputs.run_test == 'check-heartbeat'}}
name: Check Heartbeat
runs-on: ubuntu-latest

steps:
- name: Checkout actions-hub repo
uses: actions/checkout@v4
with:
repository: nelc/actions-hub
ref: main
path: actions-hub

- name: Setup Python and Playwright
uses: ./actions-hub/.github/actions/setup-playwright

- name: Run heartbeat test
run: |
python actions-hub/scripts/smoke-tests/test_heartbeat.py \
--base-url "${{ inputs.target_url }}"

validate-plugins:
if: ${{ inputs.run_test == 'All' || inputs.run_test == 'validate-plugins'}}
name: Validate eox plugins
runs-on: ubuntu-latest

steps:
- name: Checkout actions-hub repo
uses: actions/checkout@v4
with:
repository: nelc/actions-hub
ref: main
path: actions-hub

- name: Setup Python and Playwright
uses: ./actions-hub/.github/actions/setup-playwright

- name: Validate eox plugins
run: |
python actions-hub/scripts/smoke-tests/test_eox_plugins.py \
--base-url "${{ inputs.target_url }}" \
--plugins ${{ inputs.target_plugins }}

login-user:
if: ${{ inputs.run_test == 'All' || inputs.run_test == 'login-user'}}
name: Login User Test
runs-on: ubuntu-latest

steps:
- name: Checkout actions-hub repo
uses: actions/checkout@v4
with:
repository: nelc/actions-hub
ref: main
path: actions-hub

- name: Setup Python and Playwright
uses: ./actions-hub/.github/actions/setup-playwright

- name: Run login-user test
run: |
python actions-hub/scripts/smoke-tests/test_login_user.py \
--base-url "${{ inputs.target_url }}" \
--email "${{ inputs.user_email }}" \
--password "${{ inputs.user_password }}"

view-course:
if: ${{ inputs.run_test == 'All' || inputs.run_test == 'view-course'}}
name: View Course Page Test
runs-on: ubuntu-latest

steps:
- name: Checkout actions-hub repo
uses: actions/checkout@v4
with:
repository: nelc/actions-hub
ref: main
path: actions-hub

- name: Setup Python and Playwright
uses: ./actions-hub/.github/actions/setup-playwright

- name: Run view-course test
run: |
python actions-hub/scripts/smoke-tests/test_view_course.py \
--base-url "${{ inputs.target_url }}" \
--email "${{ inputs.user_email }}" \
--password "${{ inputs.user_password }}" \
--course-id "${{ inputs.course_id }}"

complete-multiplechoice-unit:
if: ${{ inputs.run_test == 'All' || inputs.run_test == 'complete-multiplechoice-unit'}}
name: Complete multiplechoice unit test
runs-on: ubuntu-latest

steps:
- name: Checkout actions-hub repo
uses: actions/checkout@v4
with:
repository: nelc/actions-hub
ref: main
path: actions-hub

- name: Setup Python and Playwright
uses: ./actions-hub/.github/actions/setup-playwright

- name: Run complete-multiplechoice-unit test
run: |
python actions-hub/scripts/smoke-tests/test_complete_multiplechoice_unit.py \
--base-url "${{ inputs.target_url }}" \
--email "${{ inputs.user_email }}" \
--password "${{ inputs.user_password }}" \
--course-id "${{ inputs.course_id }}" \
--unit-id "${{ inputs.unit_id }}"

register-user:
if: ${{ inputs.run_test == 'All' || inputs.run_test == 'register-user'}}
name: Register user test
runs-on: ubuntu-latest

steps:
- name: Checkout actions-hub repo
uses: actions/checkout@v4
with:
repository: nelc/actions-hub
ref: main
path: actions-hub

- name: Setup Python and Playwright
uses: ./actions-hub/.github/actions/setup-playwright

- name: Run register-user test
run: |
python actions-hub/scripts/smoke-tests/test_register_user.py \
--base-url "${{ inputs.target_url }}"
75 changes: 75 additions & 0 deletions scripts/smoke-tests/test_complete_multiplechoice_unit.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
import argparse

from playwright.sync_api import TimeoutError, sync_playwright
from utils import login_user


def test_complete_multiplechoice_unit(base_url: str, course_id: str, unit_id: str, email: str, password: str):
"""
Smoke test to verify that a multiple choice unit can be completed.

Args:
base_url (str): The base URL of the platform.
course_id (str): The course ID to navigate into.
unit_id (str): The specific unit ID (usage key).
email (str): Login email.
password (str): Login password.
"""
unit_url = f"{base_url}/courses/{course_id}/jump_to_id/{unit_id}"

with sync_playwright() as p:
browser = p.chromium.launch()
page = browser.new_page()
login_user(page, base_url, email, password)
page.goto(unit_url)
page.wait_for_load_state("networkidle")

# Try to find the iframe that contains the unit content
iframe = page.frame_locator("#unit-iframe")
assert iframe is not None, "No iframe found containing unit content"

# Get the first problem
problems = iframe.locator("div.problem")
assert problems.count() > 0, "No problems found"
problem = problems.first

# Get problem choices
radios = problem.locator("input[type='radio']")
assert radios.count() > 0, "No multiple choice options found"

# Get Submit button
submit_button = problem.locator("button.submit")
assert submit_button.is_visible(), "Submit button not found"

for radio in radios.all():
radio.click()
submit_button.click()
notification = problem.locator(".notification.success.notification-submit")

try:
notification.wait_for(state="visible", timeout=5000)
break
except TimeoutError:
continue

assert notification.is_visible(), "Success notification isn't visible"

browser.close()


if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Smoke test: Complete a multiple choice unit")
parser.add_argument("--base-url", required=True, help="Base URL of the LMS")
parser.add_argument("--course-id", required=True, help="Course ID")
parser.add_argument("--unit-id", required=True, help="Unit ID (usage key)")
parser.add_argument("--email", required=True, help="User email for login")
parser.add_argument("--password", required=True, help="User password for login")
args = parser.parse_args()

test_complete_multiplechoice_unit(
base_url=args.base_url,
course_id=args.course_id,
unit_id=args.unit_id,
email=args.email,
password=args.password,
)
44 changes: 44 additions & 0 deletions scripts/smoke-tests/test_eox_plugins.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
import argparse

from playwright.sync_api import sync_playwright


def test_plugin_versions(base_url: str, plugins: list[str]) -> None:
"""
Smoke test to verify that plugin info endpoints return a valid JSON with version info.

Args:
base_url (str): The base URL of the LMS.
plugins (list[str]): List of plugins to check.
"""
with sync_playwright() as p:
browser = p.chromium.launch()
page = browser.new_page()

for plugin in plugins:
full_url = f"{base_url.rstrip('/')}/{plugin}/eox-info"
print(f"→ Checking {full_url}...")

response = page.goto(full_url)
assert response is not None and response.ok, f"❌ Failed to load {full_url}"

try:
json_data = response.json()
except Exception as e:
raise AssertionError(f"❌ Invalid JSON response from {full_url}: {e}")

version = json_data.get("version")
assert version, f"❌ 'version' key not found in JSON from {full_url}"

print(f"✅ {plugin} version: {version}")

browser.close()


if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Smoke test: Validate eox plugins")
parser.add_argument("--base-url", required=True, help="Base LMS URL")
parser.add_argument("--plugins", nargs="+", required=True, help="List of plugin to check")
args = parser.parse_args()

test_plugin_versions(base_url=args.base_url, plugins=args.plugins)
34 changes: 34 additions & 0 deletions scripts/smoke-tests/test_heartbeat.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import argparse

from playwright.sync_api import sync_playwright


def test_heartbeat(base_url: str):
"""
Smoke test to verify that the /heartbeat endpoint is reachable and returns HTTP 200.

Args:
base_url (str): The base URL of the platform.
"""
with sync_playwright() as p:
browser = p.chromium.launch()
page = browser.new_page()

response = page.goto(f"{base_url.rstrip('/')}/heartbeat")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
response = page.goto(f"{base_url.rstrip('/')}/heartbeat")
response = page.goto(f"{base_url.rstrip('/')}/heartbeat?extended=true")

you could do this. And also the ingration test could check by different key of dict, other services like celery and forum. So we could have the info if those services are working as expected.

image

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the test wrong? the test just checks a 200 code status it doesn't matter the response itself so if the service is down the status code would be 200 no matter what but the status key would be false ?


if not response or response.status != 200:
raise AssertionError(
f"❌ Heartbeat check failed. Status: {response.status if response else 'No response'}"
)

print(f"✅ Heartbeat responded with HTTP {response.status}")

browser.close()


if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Smoke test: Heartbeat Endpoint")
parser.add_argument("--base-url", required=True, help="Base URL of the platform")
args = parser.parse_args()

test_heartbeat(base_url=args.base_url)
Loading
Loading