This Python program is designed to conduct performance testing on a serverless application named e-commerce-Amplify. The testing involves simulating user interactions with the application to evaluate its performance under various loads of a specific scenario.
In this testing, users will perform a series of specific actions:
- Login: Call to the Cognito API to allow the user to continue browsing as an authenticated user.
- Retrieve list of products for sale: GET request to the products REST API, obtaining a list containing details of products currently for sale.
- Product selection: Random selection of a product from the list obtained in the previous step.
- Add to cart: The chosen product will be subsequently added to the cart via a PUT request to the cart REST API.
- Order creation: POST request to the orders REST API. This call, after invoking the Lambda function of products to obtain further information (especially the number of available products and the price), invoking the Lambda function of the cart to remove the corresponding item from this order present in the cart, and making a call to the Cognito API to decrease the buyer's balance, will lead to the creation of an order with "drafted" status.
- Logout: Call to the Cognito API to log out.
The first and last actions are repeated only once per user, while the rest of the actions represent the workload and are repeated for the duration of the test.
To make the proposed case study realistic, a waiting period (≈ 48 seconds) will be introduced between the moment the list of products for sale is obtained and the subsequent random selection of a product. This is aimed at emulating the browsing time within the application. It is not guaranteed that a user necessarily makes a purchase during their browsing, so each action will be weighted by a purchase probability (2.7%).
The program generates 5 threads (which are using 5 different user accounts) and initiates the simulation. Once completed, the threads record the results, including the average time required for each call and the number of successful and failed calls, in a shared memory area. These data will be subsequently used by the program to generate histograms related to the average time required for each call and the number of calls for each API. he documentation also includes how to interact with the AWS Amplify wizard (through CLI) to generate the services used in the application.
Some information has been defined as global variables, such as the number of user accounts used, the average session duration, the conversion rate (purchase probability), etc. While some of these variables need to be modified directly in the code (for example, the number of users accounts used and the variables associated with the Gaussian distribution of the waiting period), the program allows manipulation of the average session duration and the conversion rate through program arguments. The argument "[number of users to simulate per account]" allows dividing the average session duration to simulate a greater number of users per account. Similarly, the argument "[conversion rate multiplier]" allows multiplying the conversion rate, thus influencing the number of orders created.
Follow the documentation of the serverless application e-commerce-Amplify to built it.
- Access the AWS Cognito console and locate the User Pool generated by AWS Amplify.
- Navigate to the "App integration" section and scroll down to the bottom of the page, where you'll find the section named "App client settings and analytics".
- Select the Web client generated by AWS Amplify by clicking on the app client's name.
- Once inside the dedicated app client page, locate the Client ID, which will be required as an environment variable for test execution. Click the "Edit" button in the first section of the page titled "App client details".
- Add the "ALLOW_USER_PASSWORD_AUTH" option among the Authentication flows to allow authentication via username and password. Additionally, you can adjust the access token expiration, ensuring that it is at least equal to the expected duration for test execution to prevent token expiration during the test itself.
- Access the AWS IAM (Identity and Access Management) console and locate the user used for the architecture setup via AWS Amplify. It's important to use the same user as APIs can only be directly invoked through this IAM user.
- In the permissions Policy section, select "Add permissions" from the "Add permissions" dropdown menu.
- Choose the option "Attach directly policies" and proceed to add the policy named "AmazonAPIGatewayInvokeFullAccess."
To prepare the necessary data inside the application for testing, perform the following operations:
- Create a seller user within the application and add at least one product. For the conducted tests, 5 products have been inserted.
- Create 5 buyer users within the application. These users must have identical usernames and passwords. The values to use as usernames and passwords are: "FaxyBuyer0", "FaxyBuyer1", "FaxyBuyer2", "FaxyBuyer3", and "FaxyBuyer4". Additionally, ensure that each of these users has sufficient balance for the experiment. Different usernames and passwords can be used, but in that case, the implemented code will need to be updated. The use of different email addresses for each account is not required.
To avoid inserting sensitive data directly into the code, a .env file has been used. Before executing the test, it is necessary to complete this file, which contains environment variables accessible in the program through their names and the Python "dotenv" module.
It is recommended to conduct the test within an environment managed by AWS Cloud9. To create the environment, simply access the AWS Cloud9 console, create an environment with default settings corresponding to the minimum requirements for the EC2 instance that will be used for Cloud9 execution. Subsequently, open the integrated development environment (IDE) where you'll run the test.
If you wish to retain the generated graphs during different executions, it's essential to have a directory named "plots" present.
The program needs the following command-line arguments:
- Test length (with a minimum value of 600 seconds)
- Number of users to simulate per account (with a minimum value of 1)
- Conversion rate multiplier (with a minimum value of 0)
The syntax to execute the program via the command line is as follows:
python test.py [test length] [number of users to simulate per account] [conversion rate multiplier]
The command:
python test.py 3600 20 10
Will produce in output the following histograms:
Distributed under the MIT License. See LICENSE.txt for more information.