Merge pull request #498 from atlassian/release/4.0.0

Release 4.0.0

.dockerignore

@@ -0,0 +1,3 @@
+*
+!requirements.txt
+!Dockerfile

.gitignore

@@ -8,4 +8,5 @@ outputs/
 *.iml
 *.log
 *.pyc
-local_*.yml
+local_*.yml
+**jmeter.properties

Dockerfile

@@ -16,10 +16,10 @@ RUN apt-get -y update \
   && apt-get clean
 
 COPY requirements.txt /tmp/requirements.txt
-RUN pip install -r /tmp/requirements.txt
+RUN pip install --no-cache-dir -r /tmp/requirements.txt
 
 RUN rm -rf /root/.bzt/jmeter-taurus/
 
 WORKDIR /dc-app-performance-toolkit/app
 
-ENTRYPOINT ["bzt", "-o", "modules.console.disable=true"]
+ENTRYPOINT ["bzt", "-o", "modules.console.disable=true"]

README.md

@@ -1,28 +1,30 @@
 # Data Center App Performance Toolkit 
 The Data Center App Performance Toolkit extends [Taurus](https://gettaurus.org/) which is an open source performance framework that executes JMeter and Selenium.
 
-This repository contains Taurus scripts for performance testing of Atlassian Data Center products: Jira, Confluence, and Bitbucket.
+This repository contains Taurus scripts for performance testing of Atlassian Data Center products: Jira, Jira Service Management, Confluence, and Bitbucket.
 
 ## Supported versions
 * Supported Jira versions: 
-    * Jira [Long Term Support release](https://confluence.atlassian.com/enterprise/atlassian-enterprise-releases-948227420.html): 7.13.15 and 8.5.8
-    * Jira Platform release: 8.0.3
+    * Jira [Long Term Support release](https://confluence.atlassian.com/enterprise/atlassian-enterprise-releases-948227420.html): `8.13.3`, `8.5.11`
+
+* Supported Jira Service Management versions: 
+    * Jira Service Management [Long Term Support release](https://confluence.atlassian.com/enterprise/atlassian-enterprise-releases-948227420.html): `4.13.2`, `4.5.10`
 
 * Supported Confluence versions:
-    * Confluence [Long Term Support release](https://confluence.atlassian.com/enterprise/atlassian-enterprise-releases-948227420.html): 6.13.13 and 7.4.4
-    * Confluence Platform release: 7.0.5
+    * Confluence [Long Term Support release](https://confluence.atlassian.com/enterprise/atlassian-enterprise-releases-948227420.html): `7.4.6`  
+    * Confluence Platform release: `7.0.5`
 
 * Supported Bitbucket Server versions:
-    * Bitbucket Server [Long Term Support release](https://confluence.atlassian.com/enterprise/atlassian-enterprise-releases-948227420.html): 6.10.5
-    * Bitbucket Server Platform release: 7.0.5
+    * Bitbucket Server [Long Term Support release](https://confluence.atlassian.com/enterprise/atlassian-enterprise-releases-948227420.html): `7.6.2`, `6.10.7`  
+    * Bitbucket Server Platform release: `7.0.5`
 
 ## Support
 In case of technical questions, issues or problems with DC Apps Performance Toolkit, contact us for support in the [community Slack](http://bit.ly/dcapt_slack) **#data-center-app-performance-toolkit** channel.
 
 ## Installation and set up
 
 #### Dependencies
-* Python 3.6+ and pip
+* Python 3.6-3.8 and pip
 * JDK 8
 * Google Chrome web browser
 * Git client (only for Bitbucket Server)
@@ -32,7 +34,7 @@ Please make sure you have a version of Chrome browser that is compatible with [C
 If a first part of ChromeDriver version does not match with a first part of your Chrome browser version, update Chrome browser or set compatible [ChromeDriver](http://chromedriver.chromium.org/downloads) version in .yml file.
 
 ### macOS/Linux
-Make sure that you have [Python](https://www.python.org/downloads/) 3.6+, pip, and [JDK 8](https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html) installed:
+Make sure that you have [Python](https://www.python.org/downloads/) (see [dependencies](#dependencies) section for supported versions), pip, and [JDK 8](https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html) installed:
 ```
 python3 --version
 pip --version
@@ -64,12 +66,13 @@ pip install -r requirements.txt
 
 ### Windows
 #### Installing Taurus manually
-Make sure you have [Python](https://www.python.org/downloads/) 3.6+, pip, and [JDK 8](https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html) installed:
+Make sure you have [Python](https://www.python.org/downloads/) (see [dependencies](#dependencies) section for supported versions), pip, and [JDK 8](https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html) installed:
 ```
 python --version or python3 --version
 pip --version
 java -version
 Microsoft Visual C++ 14
+Windows 10 SDK
 ```
 For Bitbucket Server check that [Git](https://git-scm.com/downloads) is installed:
 ```
@@ -84,8 +87,9 @@ Otherwise, download it from [Microsoft Visual C++ Build Tools:](https://visualst
 4. Select the **MSVC v142 - VS 2019 C++ x64/x86 build tools (v14.22)** check box (clear all the others).
 5. Click **Install**.
 
-We recommend using virtualenv for Taurus.
+Setup [Windows 10 SDK](https://developer.microsoft.com/en-us/windows/downloads/windows-10-sdk/)
 
+We recommend using virtualenv for Taurus.
 1. Install virtualenv with pip:
 ```
 pip install virtualenv
@@ -134,4 +138,4 @@ The request include the following data, and will in no way contain PII (Personal
 To help us continue improving the Toolkit, we’d love you to keep these analytics enabled in testing, staging, and production. If you don’t want to send us analytics, you can turn off the `allow_analytics` toggle in `$product.yml` file.
 
 ## Running Taurus
-Navigate to [docs](docs) folder and follow instructions.
+Navigate to [docs](docs) folder and follow instructions.

app/bitbucket.yml

@@ -85,7 +85,7 @@ modules:
       httpsampler.ignore_failed_embedded_resources: "true"
   selenium:
     chromedriver:
-      version: "86.0.4240.22" # Supports Chrome version 86. You can refer to http://chromedriver.chromium.org/downloads
+      version: "88.0.4324.27" # Supports Chrome version 88. You can refer to http://chromedriver.chromium.org/downloads
 reporting:
 - data-source: sample-labels
   module: junit-xml

app/confluence.yml

@@ -109,7 +109,7 @@ modules:
       httpsampler.ignore_failed_embedded_resources: "true"
   selenium:
     chromedriver:
-      version: "86.0.4240.22" # Supports Chrome version 86. You can refer to http://chromedriver.chromium.org/downloads
+      version: "88.0.4324.27" # Supports Chrome version 88. You can refer to http://chromedriver.chromium.org/downloads
 reporting:
 - data-source: sample-labels
   module: junit-xml

app/datasets/jsm/examples/agents.csv

@@ -0,0 +1 @@
+test_username,password

app/datasets/jsm/examples/customers.csv

@@ -0,0 +1 @@
+testusername, password

app/datasets/jsm/examples/requests.csv

@@ -0,0 +1 @@
+1,PRJ-1,1,10000,PRJ

app/datasets/jsm/examples/service_desks_large.csv

@@ -0,0 +1,3 @@
+# Projects with > 100k requests.
+service_desk_id,project_id,project_key,total_requests_count,all_open_queue_id,created_vs_resolved_id_report_id,time_to_resolution_id_report_id
+1,1,TESTPRJ,350000,10,11,12

app/datasets/jsm/examples/service_desks_medium.csv

@@ -0,0 +1,3 @@
+# Projects with > 10k and less than 100k requests.
+service_desk_id,project_id,project_key,total_requests_count,all_open_queue_id,created_vs_resolved_id_report_id,time_to_resolution_id_report_id
+1,1,TESTPRJ,350000,10,11,12

app/datasets/jsm/examples/service_desks_small.csv

@@ -0,0 +1,3 @@
+# Projects with < 10k requests.
+service_desk_id,project_id,project_key,total_requests_count,all_open_queue_id,created_vs_resolved_id_report_id,time_to_resolution_id_report_id
+1,1,TESTPRJ,350000,10,11,12

app/extension/bitbucket/extension_ui.py

@@ -1,15 +1,18 @@
+import random
+
 from selenium.webdriver.common.by import By
-from selenium_ui.conftest import print_timing
-from util.conf import BITBUCKET_SETTINGS
 
 from selenium_ui.base_page import BasePage
+from selenium_ui.conftest import print_timing
+from util.conf import BITBUCKET_SETTINGS
 
 
 def app_specific_action(webdriver, datasets):
     page = BasePage(webdriver)
-    repo = datasets['repos']
-    repo_slug = repo[0]
-    project_key = repo[1]
+    rnd_repo = random.choice(datasets["repos"])
+
+    project_key = rnd_repo[1]
+    repo_slug = rnd_repo[0]
 
     @print_timing("selenium_app_custom_action")
     def measure():

app/extension/confluence/extension_locust.py

@@ -4,7 +4,7 @@
 logger = init_logger(app_type='confluence')
 
 
-@confluence_measure
+@confluence_measure("locust_app_specific_action")
 def app_specific_action(locust):
     r = locust.get('/app/get_endpoint', catch_response=True)  # call app-specific GET endpoint
     content = r.content.decode('utf-8')   # decode response content
@@ -14,7 +14,7 @@ def app_specific_action(locust):
     token = re.findall(token_pattern_example, content)  # get TOKEN from response using regexp
     id = re.findall(id_pattern_example, content)    # get ID from response using regexp
 
-    logger.locust_info(f'token: {token}, id: {id}')  # log information for debug when verbose is true in jira.yml file
+    logger.locust_info(f'token: {token}, id: {id}')  # log info for debug when verbose is true in confluence.yml file
     if 'assertion string' not in content:
         logger.error(f"'assertion string' was not found in {content}")
     assert 'assertion string' in content  # assert specific string in response content

app/extension/confluence/extension_ui.py

@@ -1,15 +1,16 @@
+import random
+
 from selenium.webdriver.common.by import By
-from selenium_ui.conftest import print_timing
-from util.conf import CONFLUENCE_SETTINGS
 
 from selenium_ui.base_page import BasePage
+from selenium_ui.conftest import print_timing
+from util.conf import CONFLUENCE_SETTINGS
 
 
 def app_specific_action(webdriver, datasets):
     page = BasePage(webdriver)
     if datasets['custom_pages']:
-        app_specific_page = datasets['custom_pages']
-        app_specific_page_id = app_specific_page[0]
+        app_specific_page_id = datasets['custom_page_id']
 
     @print_timing("selenium_app_custom_action")
     def measure():

app/extension/jira/extension_locust.py

@@ -4,7 +4,7 @@
 logger = init_logger(app_type='jira')
 
 
-@jira_measure
+@jira_measure("locust_app_specific_action")
 def app_specific_action(locust):
     r = locust.get('/app/get_endpoint', catch_response=True)  # call app-specific GET endpoint
     content = r.content.decode('utf-8')   # decode response content
@@ -14,7 +14,7 @@ def app_specific_action(locust):
     token = re.findall(token_pattern_example, content)  # get TOKEN from response using regexp
     id = re.findall(id_pattern_example, content)    # get ID from response using regexp
 
-    logger.locust_info(f'token: {token}, id: {id}')  # log information for debug when verbose is true in confluence.yml file
+    logger.locust_info(f'token: {token}, id: {id}')  # log info for debug when verbose is true in jira.yml file
     if 'assertion string' not in content:
         logger.error(f"'assertion string' was not found in {content}")
     assert 'assertion string' in content  # assert specific string in response content

app/extension/jira/extension_ui.py

@@ -10,8 +10,7 @@
 def app_specific_action(webdriver, datasets):
     page = BasePage(webdriver)
     if datasets['custom_issues']:
-        app_specific_issue = random.choice(datasets['custom_issues'])
-        issue_key = app_specific_issue[0]
+        issue_key = datasets['custom_issue_key']
 
     @print_timing("selenium_app_custom_action")
     def measure():

app/extension/jsm/extension_locust_agents.py

@@ -0,0 +1,28 @@
+import re
+from locustio.common_utils import init_logger, jsm_agent_measure
+
+logger = init_logger(app_type='jsm')
+
+
+@jsm_agent_measure('locust_agent_app_specific_action')
+def app_specific_action(locust):
+    r = locust.get('/app/get_endpoint', catch_response=True)  # call app-specific GET endpoint
+    content = r.content.decode('utf-8')   # decode response content
+
+    token_pattern_example = '"token":"(.+?)"'
+    id_pattern_example = '"id":"(.+?)"'
+    token = re.findall(token_pattern_example, content)  # get TOKEN from response using regexp
+    id = re.findall(id_pattern_example, content)    # get ID from response using regexp
+
+    logger.locust_info(f'token: {token}, id: {id}')  # log info for debug when verbose is true in jsm.yml file
+    if 'assertion string' not in content:
+        logger.error(f"'assertion string' was not found in {content}")
+    assert 'assertion string' in content  # assert specific string in response content
+
+    body = {"id": id, "token": token}  # include parsed variables to POST request body
+    headers = {'content-type': 'application/json'}
+    r = locust.post('/app/post_endpoint', body, headers, catch_response=True)  # call app-specific POST endpoint
+    content = r.content.decode('utf-8')
+    if 'assertion string after successful POST request' not in content:
+        logger.error(f"'assertion string after successful POST request' was not found in {content}")
+    assert 'assertion string after successful POST request' in content  # assertion after POST request

app/extension/jsm/extension_locust_customers.py

@@ -0,0 +1,28 @@
+import re
+from locustio.common_utils import init_logger, jsm_customer_measure
+
+logger = init_logger(app_type='jsm')
+
+
+@jsm_customer_measure('locust_customer_app_specific_action')
+def app_specific_action(locust):
+    r = locust.get('/app/get_endpoint', catch_response=True)  # call app-specific GET endpoint
+    content = r.content.decode('utf-8')   # decode response content
+
+    token_pattern_example = '"token":"(.+?)"'
+    id_pattern_example = '"id":"(.+?)"'
+    token = re.findall(token_pattern_example, content)  # get TOKEN from response using regexp
+    id = re.findall(id_pattern_example, content)    # get ID from response using regexp
+
+    logger.locust_info(f'token: {token}, id: {id}')  # log info for debug when verbose is true in jsm.yml file
+    if 'assertion string' not in content:
+        logger.error(f"'assertion string' was not found in {content}")
+    assert 'assertion string' in content  # assert specific string in response content
+
+    body = {"id": id, "token": token}  # include parsed variables to POST request body
+    headers = {'content-type': 'application/json'}
+    r = locust.post('/app/post_endpoint', body, headers, catch_response=True)  # call app-specific POST endpoint
+    content = r.content.decode('utf-8')
+    if 'assertion string after successful POST request' not in content:
+        logger.error(f"'assertion string after successful POST request' was not found in {content}")
+    assert 'assertion string after successful POST request' in content  # assertion after POST request

app/extension/jsm/extension_ui_agents.py

@@ -0,0 +1,24 @@
+from selenium.webdriver.common.by import By
+
+from selenium_ui.base_page import BasePage
+from selenium_ui.conftest import print_timing
+from util.conf import JSM_SETTINGS
+
+
+def app_specific_action(webdriver, datasets):
+    page = BasePage(webdriver)
+    if datasets['custom_issues']:
+        issue_key = datasets['custom_issue_key']
+
+    @print_timing("selenium_agent_app_custom_action")
+    def measure():
+
+        @print_timing("selenium_agent_app_custom_action:view_request")
+        def sub_measure():
+            page.go_to_url(f"{JSM_SETTINGS.server_url}/browse/{issue_key}")
+            # Wait for summary field visible
+            page.wait_until_visible((By.ID, "summary-val"))
+            # Wait for you app-specific UI element by ID selector
+            page.wait_until_visible((By.ID, "ID_OF_YOUR_APP_SPECIFIC_UI_ELEMENT"))
+        sub_measure()
+    measure()

app/extension/jsm/extension_ui_customers.py

@@ -0,0 +1,26 @@
+from selenium.webdriver.common.by import By
+
+from selenium_ui.base_page import BasePage
+from selenium_ui.conftest import print_timing
+from util.conf import JSM_SETTINGS
+
+
+def app_specific_action(webdriver, datasets):
+    page = BasePage(webdriver)
+    if datasets['custom_issues']:
+        custom_request_key = datasets['custom_issue_key']
+        custom_service_desk_id = datasets['custom_service_desk_id']
+
+    @print_timing("selenium_customer_app_custom_action")
+    def measure():
+
+        @print_timing("selenium_customer_app_custom_action:view_request")
+        def sub_measure():
+            page.go_to_url(f"{JSM_SETTINGS.server_url}/servicedesk/customer/portal/"
+                           f"{custom_service_desk_id}/{custom_request_key}")
+            # Wait for options element visible
+            page.wait_until_visible((By.CLASS_NAME, 'cv-request-options'))
+            # Wait for you app-specific UI element by ID selector
+            page.wait_until_visible((By.ID, "ID_OF_YOUR_APP_SPECIFIC_UI_ELEMENT"))
+        sub_measure()
+    measure()

app/jira.yml

@@ -113,7 +113,7 @@ modules:
       httpsampler.ignore_failed_embedded_resources: "true"
   selenium:
     chromedriver:
-      version: "86.0.4240.22" # Supports Chrome version 86. You can refer to http://chromedriver.chromium.org/downloads
+      version: "88.0.4324.27" # Supports Chrome version 88. You can refer to http://chromedriver.chromium.org/downloads
 reporting:
 - data-source: sample-labels
   module: junit-xml