cmi5 Conformance Testing Procedures Draft

cmi5 Conformance Testing Procedures Draft

Draft as of Dec 15, 2017

---

Disclaimers/Limits

Because compliance cannot be verified – it does not mean that an LMS (or AU) is not cmi5 conformant.

Failure to import course structures (with over 1000 AU's) due to size/capacity of the system does not indicate failure of compliance with cmi5.

Test suite AU failure to execute due to software/environment conflicts…

---

LMS

• (TestSuite) Create Test Course Structure(s)
  • Multiple course structures
    • ZIP Package vs XML only
    • Mix of Relative and Fully Qualified URLS
    • One for size test 1000 + AU's
    • Smaller one for Completion (Satisfied Test)
      • Large enough to test all MoveOn Criteria
        • Blocks, AU's
• (Human) Import that structure(s) into the LMS (admin user)
• (Human) Register the learner (admin User)
• (Human) Access the course as a learner
  • Selected AU's (not all)
• (Machine) AU's (Testsuite) inspect LMS/LRS data
• (Human) Procedure for learner to follow thru the course(s).
  • Initial entry
  • Re-entry of AU's
• (Human) Modify course structure (admin user) and verify effect as a learner
  • Eg, MasteryScore, MoveOn, etc

AU (Content) - Individually

• (Human) add a reference (URL) to AU to inspect
• (Human) Launch the AU and Interact with AU
• (Human) Re-entry to AU's
  • Launch in different modes
    • Review, Normal, Browse
• (Machine) Testsuite will inspect AU xapi data

AU (Content) – Course Structure (on or more)

• (Machine) Testsuite (LMS) – import/validate a course structure
• (Human) Launch each AU in the course and Interact with AU
• (Human) Re-entry to AU's
  • Launch in different modes
    • Review, Normal, Browse
• (Machine) Testsuite will inspect AU xapi data

Course Structure/Package

• (Machine) Inspect XML
  • XSD validation
  • Relative pathing & Fully Qualified URL
• (Machine) Inspect Package
  • Inspect ZIP
  • File existence validation (ZIP)
    • Content
    • XML file
  • ZIP format (32/64)

---

Small Course Structure test (LMS)

• Zip File with content
• Mix of relative and fully qualified URL's
• Nested Blocks
• Mixed Blocks - Blocks with AU's and Block Children
• Blocks with Block children only

Block1

AU0

• launchMethod="OwnWindow"
• moveOn=" Completed "
• <Fully Qualified url>
• <launchParameters>
• <entitlementKey>

Block1.1

AU1

• launchMethod="OwnWindow"
• masteryScore="0.8501"
• moveOn="Passed"
• <Relative url>
• <launchParameters>
• <entitlementKey>

AU2

• launchMethod="OwnWindow"
• moveOn=" Completed "
• <Relative url>
• <launchParameters>
• <entitlementKey>

AU3

• launchMethod="OwnWindow"
• masteryScore="0.8501"
• moveOn="CompletedAndPassed "
• <Relative url>
• <launchParameters>
• <entitlementKey>

AU4

• launchMethod="OwnWindow"
• masteryScore="0.8501"
• moveOn="CompletedOrPassed "
• <Relative url>
• <launchParameters>
• <entitlementKey>

AU5

• launchMethod="OwnWindow"
• moveOn="NotApplicable"
• <Relative url>
• <launchParameters>
• <entitlementKey>

Block1.1.1

Block1.1.1.1

AU6

• launchMethod="OwnWindow"
• moveOn=" Completed"
• <Fully Qualified url>
• <launchParameters>
• <entitlementKey>

AUx

• launchMethod="OwnWindow"
• moveOn="NotApplicable"
• <Relative url>
• <launchParameters>
• <entitlementKey>

---

Large Course Structure test (LMS) Test

• Has 1001 AU's
• Fully Qualified URL
• Zipped and XML only

• Block1
  • AU1
    • launchMethod="OwnWindow"
    • moveOn=" Completed "
    • <url>
    • <launchParameters>
    • <entitlementKey>
  • …
  • AU1 1001
    • launchMethod="OwnWindow"
    • moveOn=" Completed "
    • <url>
    • <launchParameters>
    • <entitlementKey>

---

Testing Procedures

Small course testing

• (Human) Import that structure(s) into the LMS (admin user)
• (Human) Register multiple learners (admin User) – User 1, 2, 3….
• (Human) Access the course as a learner 1
  • Launch AU's in order
    • Do Not Launch (AU5 – Not applicable)
  • Complete all AU's with minimum MoveOn Criterial
  • Single Entry
  • Final Check AU
    • Run AU x after course complete (check for all satisfied)
    • A wait period should be established
    • AU x is Not Applicable
    • Alternate Credentials to Access to LRS for LMS issued statements.
• (Human) Access the course as a learner 2
  • Launch AU's in order
  • Complete all AU's with MoveOn Criterial
  • Multiple Entry
  • Final Check AU
    • Run AU x after course complete (check for all satisfied)
    • A wait period should be established
    • AU x is Not Applicable
    • Alternate Credentials to Access to LRS for LMS issued statements.
• (Human) Access the course as a learner 3
  • Launch AU's in any order
  • Multiple Entry (Re-launching - Exiting and Re-entering AU)
  • Complete all AU's except AU6
    • Partially Complete AU6 and re-enter
    • Complete AU6
  • Final Check AU
    • Run AU x after course complete (check for all satisfied)
    • A wait period should be established
    • AU x is Not Applicable
    • Alternate Credentials to Access to LRS for LMS issued statements.

Large course testing

• (Machine) AU's (Testsuite) inspect LMS/LRS data
• (Human) Procedure for learner to follow thru the course(s).
  • Initial entry
  • Re-entry of AU's
• (Human) Modify course structure (admin user) and verify effect as a learner
  • Eg, MasteryScore, MoveOn, etc