Testing and QA (OLD)
This part of the survey describes the purpose of the Softserve Academy application, a platform designed to create online learning paths, where users can access centralized resources in an organized way. The test asks participants to perform several steps within the app, such as creating an account, setting up their preferences, and completing a learning path with Python as the target skill.
The purpose of the test is to evaluate the usability, focusing on how intuitive the interface is, its visual aesthetics, and the functionality of the platform. This will help gather feedback on how users perceive the overall experience and whether it meets the expectations of facilitating the learning process in a fast and personalized manner.
This part of the survey is designed for users to share their opinion on how easy they found the application to use. They are asked about their experience with things like signing up, using the control panel, customizing their preferences, and navigating through different sections. In addition to a rating from 1 to 5, they are optionally asked to explain their answers, which helps to better understand if there was anything they didn’t like or if they encountered any problems.
This part of the survey focuses on the creativity and innovation of the application. Users are asked whether they consider the app to be novel or disruptive in the market. The idea is for users to give a rating from 1 to 5, depending on how much they feel the app offers something different or innovative. Then, they are optionally asked to justify their answer by explaining what aspects of the app seem original or different to them, or, if they think it doesn't bring anything new, why they believe that.
This part of the survey focuses on evaluating the viability of the application, specifically whether it effectively fulfills its intended purpose. In this case, the question seems to ask whether the application successfully creates and displays routes as it was designed to. Users rate how well they believe the app performs this specific function, and then, optionally, they can justify their rating by explaining whether they were able to complete these actions smoothly or if they encountered any difficulties.
This part of the survey focuses on understanding how visually appealing the application is to users. They are asked to evaluate the design and general appearance of several important sections of the app. The goal is to determine whether users find the application visually pleasant and whether its design contributes to a positive user experience. They also have the opportunity to justify their ratings, which provides valuable insight into which areas of the visual design may need improvement.
EXCEL WITH CALCULATED AVERAGES
Design: The average score is 3.7. It's not bad, but there's room for improvement. Users think the design could be a bit easier to use, mostly in the parts of the profile since they didn't know how to fill some information.
Creativity: It scored 4.2, which is pretty good. Users see the app as innovative, but there’s always room to make it feel more unique or different.
Viability: The average here is 4, meaning the app serves its purpose, but there may be small issues that didn’t work perfectly for some users. Overall, it's on the right track.
Aesthetics: With a 3.96, most people think the app looks good, but there are a few details that could be improved to make it more visually appealing like in the design the profile was really simple in comparison to the other views.
To test our application we focus on doing E2E testing simulating all the user stories, like how the user enters the application, creates an account, logs ing, updates their profile, creates a learning route and enters a content. We made tests for the admin too, how he adds a new content to the page and how he creates an skill.
Some parts o the test where made to be like random with the user imputing a random information in the page, like selecting between, video, text or audio for their preferences, some others where made to test the complete path of a given user.
We use Playwright as our testing tool at the end for this sprint, playwright allowed us to create the user tests with ease.
While the test where being made we manage to find many bugs of how the application behaves, but most of the come to the conclusion that we are missing information in the database, but the app worked in the parts where the data base has information.
We didn't plan any unit testing since like we saw in the class it didn't look feasible, for example testing the database alone didn't give us the whole perspective of how the user interacted with it, we may do unit testing in the future to test something like performance but we see that it's best to keep doing and improving E2E testing since most of the problems can be seen through it.
Title: Error when trying to create content with video preference selected
Description: When the user selects video as the preferred content type, the application fails to create the content. This prevents further interaction with the application and blocks access to the learning route.
Steps to Reproduce:
1. Log in
2. Updating the profile selecting Video as content
3. Selecting a technology
4. And creating a learning path
Expected Result: The application should allow the user to create content with video preference and provide access to the learning route.
Actual Result: The user is unable to create content, and access to the learning route is blocked.
Severity: High
Environment:
Operating System: Windows 10
Node.js version: 21
Playwright version: Latest
Python version: 3.12
Django version: 5.1
Title: Error when trying to create content with audio preference selected
Description: The application fails to create content when audio is selected as the preferred content type. This blocks access to the learning route and prevents the user from using the application.
Steps to Reproduce:
1. Log in
2. Updating the profile selecting Video as content
3. Selecting a technology
4. Creating a learning path
Expected Result: The application should allow content creation when audio is selected and give access to the learning route.
Actual Result: The user is unable to create content, and the learning route is inaccessible.
Severity: High
Environment:
Operating System: Windows 10
Node.js version: 21
Playwright version: Latest
Python version: 3.12
Django version: 5.1
Title: Ability to log in when already logged in
Description: The system allows a user to log in again even when they are already logged into their account. The previous session's data is not retained, and no security vulnerabilities were identified, but it disrupts access to the previous session's data.
Steps to Reproduce:
1. Log in
2. Go home
3. Go to the log in view through the searching bar
4. Log in again
Expected Result: The system should prevent a user from logging in again while they are already logged in.
Actual Result: A user can initiate a second login, which results in the loss of access to the previous session’s data.
Severity: Low
Environment:
Operating System: Windows 10
Node.js version: 21
Playwright version: Latest
Python version: 3.12
Django version: 5.1
Title: Negative time values allowed when setting preferences
Description: The application allows users to enter negative time values when configuring preferences, causing the system to malfunction.
Steps to Reproduce:
1. Log in
2. Go in to preferences
3. Fill time with negative values
4. Create a learning route
Expected Result: The system should validate inputs and prevent negative time values from being entered.
Actual Result: Users are able to input negative time values, causing the application to crash.
Severity: Medium
Environment:
Operating System: Windows 10
Node.js version: 21
Playwright version: Latest
Python version: 3.12
Django version: 5.1
Title: Unrealistically large time values allowed in preferences
Description: The system allows users to input excessively large time values when configuring preferences, which causes the application to behave unrealistically.
Steps to Reproduce:
1. Log in
2. Go in to preferences
3. Fill time with excessively large values
4. Create a learning route
Expected Result: The system should set reasonable limits for the time values allowed in preferences.
Actual Result: Users can input extremely large time values, although the application continues to function.
Severity: Low
Environment:
Operating System: Windows 10
Node.js version: 21
Playwright version: Latest
Python version: 3.12
Django version: 5.1
We tried to use Pylint for static code analysis to ensure that our Django project adheres to best practices in terms of code quality. We used a custom configuration file, .pylintrc, has been set up to enforce our code style, ensuring consistency across the project. This file includes specific rules and thresholds to manage warnings and errors, optimizing both development workflow and maintainability. But it didn´t work well
Early detection of syntax errors, unused variables, and potential bugs. Ensures adherence to naming conventions and code standards. Improves the readability and quality of the codebase.
Our project follows a strict naming convention to maintain clarity and consistency:
- PascalCase is used for naming Django models in alignment with Django's default style. This ensures that model class names are distinct and easily recognizable.
- snake_case is adopted for naming functions and variables. This is the standard in Python and provides consistency across the codebase.
- UPPER_CASE is reserved for constants, making them easily identifiable and ensuring that they are not unintentionally modified.
- Enhances readability and reduces ambiguity in large codebases.
- Promotes uniformity, making the code more maintainable and easier for teams to collaborate on.
| Functionality | Type of Test | Justification |
|---|---|---|
| User Registration | End-to-End | Ensures that new users can successfully create accounts and start using the platform without issues. |
| Profile Setup | End-to-End | Verifies that users can customize their profiles and preferences, a critical feature for personalized learning paths. |
| Path Creation | End-to-End | Validates that users can create new learning paths, ensuring the core functionality of the platform works as expected. |
| Resource Upload | End-to-End | Ensures that users can upload educational resources to their learning paths, maintaining the integrity of content management. |
| Content Update | End-to-End | Confirms that users can update and manage existing content, ensuring flexibility and dynamic content in the platform. |
We manage to implement a Github Actions that tracks when a user does a push to the main branch, runs the apps and its tests to make sure that the bare minimum of the app is working and saves the result.
When we tried to use Pylint, we couldn’t manage to make it work. The reason for the problem was that when we ran the analysis, the program also analyzed the libraries we used. If the libraries didn’t follow our conventions, the program failed and gave us the following console prompt.
PS C:\Users\USuario\Desktop\Code\py\SoftServe-Academy> pylint .\home\
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File "C:\Users\USuario\AppData\Local\Programs\Python\Python312\Scripts\pylint.exe\_main_.py", line 7, in <module>
File "C:\Users\USuario\AppData\Local\Programs\Python\Python312\Lib\site-packages\pylint\_init_.py", line 34, in run_pylint
PylintRun(argv or sys.argv[1:])
File "C:\Users\USuario\AppData\Local\Programs\Python\Python312\Lib\site-packages\pylint\lint\run.py", line 162, in _init_
args = _config_initialization(
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\USuario\AppData\Local\Programs\Python\Python312\Lib\site-packages\pylint\config\config_initialization.py", line 45, in _config_initialization
config_data, config_args = config_file_parser.parse_config_file(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\USuario\AppData\Local\Programs\Python\Python312\Lib\site-packages\pylint\config\config_file_parser.py", line 126, in parse_config_file
return _RawConfParser.parse_config_file(file_path, self.verbose_mode)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\USuario\AppData\Local\Programs\Python\Python312\Lib\site-packages\pylint\config\config_file_parser.py", line 113, in parse_config_file
return _RawConfParser.parse_ini_file(file_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\USuario\AppData\Local\Programs\Python\Python312\Lib\site-packages\pylint\config\config_file_parser.py", line 40, in parse_ini_file
parser.read_file(fp)
File "C:\Users\USuario\AppData\Local\Programs\Python\Python312\Lib\configparser.py", line 705, in read_file
self._read(f, source)
File "C:\Users\USuario\AppData\Local\Programs\Python\Python312\Lib\configparser.py", line 999, in _read
for lineno, line in enumerate(fp, start=1):
^^^^^^^^^^^^^^^^^^^^^^
File "<frozen codecs>", line 322, in decode
File "C:\Users\USuario\AppData\Local\Programs\Python\Python312\Lib\encodings\utf_8_sig.py", line 69, in _buffer_decode
return codecs.utf_8_decode(input, errors, final)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byteSoftServeAcademy
To evaluate the usability of the web platform to ensure an effective, efficient, and satisfying experience in creating and tracking personalized learning paths in the tech sector.
The interaction of users with key system functionalities, such as creating learning paths, tracking progress, and customizing content, will be evaluated through a series of predefined tasks that will measure user efficiency and satisfaction in performing these activities.
- University peers
- Family members of the group
- Personal devices, as the test is already deployed.
Once the evaluation team is assembled, the following activities will be carried out:
- Preparing the equipment for testing.
- Providing context to the test user.
- Conducting the session (performing tasks and answering questions).
- Thanking the user for their participation in the research.
Hypothesis: The registration process is clear and easy to complete.
Task: Complete the registration process.
Question: Did you encounter any obstacles or confusion during the registration process?
Hypothesis: Users can log in without issues.
Task: Log into the platform.
Question: Were you able to log in without error? If not, what problem did you encounter?
Hypothesis: The profile setup is intuitive and easy to use.
Task: Set up the user profile.
Question: Was there any part of the profile setup that you found confusing or complicated?
Hypothesis: The learning path creation process is straightforward and does not present difficulties.
Task: Create a new learning path.
Question: Were you able to create a path without issues? What would you change about this process?
Hypothesis: Users can access courses without issues, and the course information is clear.
Task: Enter a course from the learning path.
Question: Were you able to access the course without issues? Did you find the course information clear?
