Merge pull request #53 from atlassian/release-1.1.0-beta

Release 1.1.0 beta

.gitignore

@@ -1,10 +1,11 @@
 .idea/
 outputs/
 **/results/
-**/datasets/*.csv
+**/datasets/*/*.csv
 *venv*
 .DS_Store
 *.swp
 *.iml
 *.log
 *.pyc
+local_*.yml

README.md

@@ -5,18 +5,24 @@ This repository contains Taurus scripts for performance testing of Atlassian Dat
 
 At the moment, Jira DC support is in beta. Confluence DC and Bitbucket DC support is coming soon.
 
-## Known issues/limitations
+## Supported versions
 * Supported Jira versions: 
-    * Latest platform release: 8.0.3
-    * Latest enterprise release: 7.13.6
-* The SQL import is flaky. In case of a failure, run it again.
+    * The latest Platform Release: 8.0.3
+    * The following Jira [Enterprise Releases](https://confluence.atlassian.com/enterprise/atlassian-enterprise-releases-948227420.html): 7.13.6 and 8.5.0
+
+* Supported Confluence versions:
+    * The latest Confluence [Enterprise Release](https://confluence.atlassian.com/enterprise/atlassian-enterprise-releases-948227420.html): 6.13.8
 
 ## Installation and set up
 
 #### Dependencies
 * Python 3.6+ and pip
 * JDK 8
-* Google Chrome web browser, version 76
+* Google Chrome web browser. 
+
+Please make sure you have a version of Chrome browser that is compatible with [ChromeDriver](http://chromedriver.chromium.org/downloads) version set in app/%product%.yml file (modules->selenium->chromedriver->version).
+
+If a first part of ChromeDriver version does not match with a first part of your Chrome browser version, update Chrome browser or set compatible [ChromeDriver](http://chromedriver.chromium.org/downloads) version in .yml file.
 
 ### macOS/Linux
 Make sure that you have [Python](https://www.python.org/downloads/) 3.6+, pip, and [JDK 8](https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html) installed:
@@ -25,7 +31,6 @@ python3 --version
 pip --version
 java -version
 ```
-Check that Chrome browser version is 76.
 
 We recommend using virtualenv for Taurus.
 
@@ -41,22 +46,12 @@ virtualenv venv -p python3
 ```
 source venv/bin/activate
 ```
-4. Install bzt:
-```
-pip install bzt==1.13.8
+4. Install dependencies:
 ```
-5. Install dependencies:
-```
-cd jira
 pip install -r requirements.txt
 ```
 
-
 ### Windows
-There are two ways of installing Taurus on Windows.
-One way is to use the prebuilt installer that will install latest Taurus on your PC including local Python 3.6 and all its dependencies.
-However, if you already have Python installed, you can install Taurus manually with pip, Python package manager.
-
 #### Installing Taurus manually
 Make sure you have [Python](https://www.python.org/downloads/) 3.6+, pip, and [JDK 8](https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html) installed:
 ```
@@ -65,7 +60,6 @@ pip --version
 java -version
 Microsoft Visual C++ 14
 ```
-Check that Chrome browser version is 76.
 
 Make sure you have Visual Studio build tool v14.22 installed. 
 Otherwise, download it from [Microsoft Visual C++ Build Tools:](https://visualstudio.microsoft.com/downloads) and do the following:
@@ -75,7 +69,6 @@ Otherwise, download it from [Microsoft Visual C++ Build Tools:](https://visualst
 4. Select the **MSVC v142 - VS 2019 C++ x64/x86 build tools (v14.22)** check box (clear all the others).
 5. Click **Install**.
 
-
 We recommend using virtualenv for Taurus.
 
 1. Install virtualenv with pip:
@@ -90,31 +83,20 @@ virtualenv venv -p python
 ```
 venv\Scripts\activate
 ```
-4. Install bzt:
-```
-pip install bzt==1.13.8
-```
-5. Install dependencies:
+4. Install dependencies:
 ```
-cd jira
 pip install -r requirements.txt
 ```
 
-#### Installing Taurus with prebuilt installer
-Download an [installer](https://gettaurus.org/builds/TaurusInstaller_1.13.8_x64.exe) and run it.
-It will install local Python 3.6 and Taurus with all its dependencies.
-
-`bzt-pip` is a wrapper for pip that can be used to install packages.
-
-Install dependencies:
+## Upgrading
+If you see "There is newer version of Taurus available, consider upgrading." warning message during your 
+test execution, please use the next command to apply the upgrade in your virtual environment.
 ```
-cd jira
-bzt-pip install setuptools wheel
-bzt-pip install -r requirements.txt
+pip install -r requirements.txt
 ```
 
 ## Additional info
 Official Taurus installation instructions are located [here](https://gettaurus.org/docs/Installation/).
 
 ## Running Taurus
-Navigate to product folder and follow README.md instructions.
+Navigate to [docs](docs) folder and follow instructions.

app/confluence.yml

@@ -0,0 +1,85 @@
+---
+settings:
+  artifacts-dir: results/confluence/%Y-%m-%d_%H-%M-%S
+  aggregator: consolidator
+  verbose: false
+  env:
+    application_hostname: test_confluence_instance.atlassian.com   # Confluence DC hostname without protocol and port e.g. test-jira.atlassian.com or localhost
+    application_protocol: http      # http or https
+    application_port: 80            # 80, 443, 8080, 2990, etc
+    application_postfix:            # e.g. /jira in case of url like http://localhost:2990/jira
+    admin_login: admin
+    admin_password: admin
+    concurrency: 200
+    test_duration: 45m
+    WEBDRIVER_VISIBLE: False
+    JMETER_VERSION: 5.1
+services:
+  - module: shellexec
+    prepare:
+      - python util/data_preparation/confluence/prepare-data.py
+    shutdown:
+      - python util/jtl_convertor/jtls-to-csv.py kpi.jtl selenium.jtl
+execution:
+  - scenario: jmeter
+    concurrency: ${concurrency}
+    hold-for: ${test_duration}
+    ramp-up: 3m
+  - scenario: selenium
+    executor: selenium
+    runner: pytest
+    hold-for: ${test_duration}
+scenarios:
+  selenium:
+    script: selenium_ui/confluence-ui.py
+  jmeter:
+    script: jmeter/confluence.jmx
+    properties:
+      application_hostname: ${application_hostname}
+      application_protocol: ${application_protocol}
+      application_port: ${application_port}
+      application_postfix: ${application_postfix}
+      # Workload model
+      total_actions_per_hr: 20000
+      perc_view_page: 54
+      perc_view_dashboard: 6
+      perc_view_blog: 8
+      perc_search_cql: 7
+      perc_create_blog: 3
+      perc_create_and_edit_page: 6
+      perc_comment_page: 5
+      perc_view_attachment: 3
+      perc_upload_attachment: 5
+      perc_like_page: 3
+      perc_standalone_extension: 0 # By default disabled
+modules:
+  consolidator:
+    rtimes-len: 0 # CONFSRVDEV-7631 reduce sampling
+    percentiles: [] # CONFSRVDEV-7631 disable all percentiles due to Taurus's excessive memory usage
+  jmeter:
+    version: ${JMETER_VERSION}
+    detect-plugins: true
+    memory-xmx: 8G  # allow JMeter to use up to 8G of memory
+    plugins:
+      - bzm-parallel=0.4
+      - bzm-random-csv=0.6
+      - jpgc-casutg=2.5
+      - jpgc-dummy=0.2
+      - jpgc-ffw=2.0
+      - jpgc-fifo=0.2
+      - jpgc-functions=2.1
+      - jpgc-json=2.6
+      - jpgc-perfmon=2.1
+      - jpgc-prmctl=0.4
+      - jpgc-tst=2.4
+      - jpgc-wsc=0.3
+      - tilln-sshmon=1.0
+      - jpgc-cmd=2.2
+      - jpgc-synthesis=2.2
+    system-properties:
+      server.rmi.ssl.disable: true
+      java.rmi.server.hostname: localhost
+      httpsampler.ignore_failed_embedded_resources: "true"
+  selenium:
+    chromedriver:
+      version: "78.0.3904.70" # Supports Chrome version 78. You can refer to http://chromedriver.chromium.org/downloads

app/datasets/confluence/static-content/files_upload.csv

@@ -0,0 +1,6 @@
+datasets/confluence/static-content/upload/test1.png,image/png,test1.png
+datasets/confluence/static-content/upload/test2.jpg,image/jpeg,test2.jpg
+datasets/confluence/static-content/upload/test3.png,image/png,test3.png
+datasets/confluence/static-content/upload/test4.png,image/png,test4.png
+datasets/confluence/static-content/upload/test5.jpg,image/jpeg,test5.jpg
+datasets/confluence/static-content/upload/test.pdf,application/pdf,test.pdf

app/extension/confluence/extension.jmx

@@ -0,0 +1,90 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.1 r1853635">
+  <hashTree>
+    <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Test Plan" enabled="true">
+      <stringProp name="TestPlan.comments"></stringProp>
+      <boolProp name="TestPlan.functional_mode">false</boolProp>
+      <boolProp name="TestPlan.tearDown_on_shutdown">true</boolProp>
+      <boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
+      <elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
+        <collectionProp name="Arguments.arguments"/>
+      </elementProp>
+      <stringProp name="TestPlan.user_define_classpath"></stringProp>
+    </TestPlan>
+    <hashTree>
+      <TestFragmentController guiclass="TestFragmentControllerGui" testclass="TestFragmentController" testname="extension" enabled="false"/>
+      <hashTree>
+        <IfController guiclass="IfControllerPanel" testclass="IfController" testname="extend_view_page" enabled="true">
+          <stringProp name="IfController.condition">${__groovy(&quot;${extend_action}&quot; == &quot;view_page&quot;)}</stringProp>
+          <boolProp name="IfController.evaluateAll">false</boolProp>
+          <boolProp name="IfController.useExpression">true</boolProp>
+        </IfController>
+        <hashTree/>
+        <IfController guiclass="IfControllerPanel" testclass="IfController" testname="extend_view_dashboard" enabled="true">
+          <stringProp name="IfController.condition">${__groovy(&quot;${extend_action}&quot; == &quot;view_dashboard&quot;)}</stringProp>
+          <boolProp name="IfController.evaluateAll">false</boolProp>
+          <boolProp name="IfController.useExpression">true</boolProp>
+        </IfController>
+        <hashTree/>
+        <IfController guiclass="IfControllerPanel" testclass="IfController" testname="extend_view_blog" enabled="true">
+          <stringProp name="IfController.condition">${__groovy(&quot;${extend_action}&quot; == &quot;view_blog&quot;)}</stringProp>
+          <boolProp name="IfController.evaluateAll">false</boolProp>
+          <boolProp name="IfController.useExpression">true</boolProp>
+        </IfController>
+        <hashTree/>
+        <IfController guiclass="IfControllerPanel" testclass="IfController" testname="extend_search_cql" enabled="true">
+          <stringProp name="IfController.condition">${__groovy(&quot;${extend_action}&quot; == &quot;search_cql&quot;)}</stringProp>
+          <boolProp name="IfController.evaluateAll">false</boolProp>
+          <boolProp name="IfController.useExpression">true</boolProp>
+        </IfController>
+        <hashTree/>
+        <IfController guiclass="IfControllerPanel" testclass="IfController" testname="extend_create_blog" enabled="true">
+          <stringProp name="IfController.condition">${__groovy(&quot;${extend_action}&quot; == &quot;create_blog&quot;)}</stringProp>
+          <boolProp name="IfController.evaluateAll">false</boolProp>
+          <boolProp name="IfController.useExpression">true</boolProp>
+        </IfController>
+        <hashTree/>
+        <IfController guiclass="IfControllerPanel" testclass="IfController" testname="extend_create_and_edit_page" enabled="true">
+          <stringProp name="IfController.condition">${__groovy(&quot;${extend_action}&quot; == &quot;create_and_edit_page&quot;)}</stringProp>
+          <boolProp name="IfController.evaluateAll">false</boolProp>
+          <boolProp name="IfController.useExpression">true</boolProp>
+        </IfController>
+        <hashTree/>
+        <IfController guiclass="IfControllerPanel" testclass="IfController" testname="extend_edit_page" enabled="true">
+          <stringProp name="IfController.condition">${__groovy(&quot;${extend_action}&quot; == &quot;edit_page&quot;)}</stringProp>
+          <boolProp name="IfController.evaluateAll">false</boolProp>
+          <boolProp name="IfController.useExpression">true</boolProp>
+        </IfController>
+        <hashTree/>
+        <IfController guiclass="IfControllerPanel" testclass="IfController" testname="extend_comment_page" enabled="true">
+          <stringProp name="IfController.condition">${__groovy(&quot;${extend_action}&quot; == &quot;comment_page&quot;)}</stringProp>
+          <boolProp name="IfController.evaluateAll">false</boolProp>
+          <boolProp name="IfController.useExpression">true</boolProp>
+        </IfController>
+        <hashTree/>
+        <IfController guiclass="IfControllerPanel" testclass="IfController" testname="extend_standalone_extension" enabled="true">
+          <stringProp name="IfController.condition">${__groovy(&quot;${extend_action}&quot; == &quot;standalone_extension&quot;)}</stringProp>
+          <boolProp name="IfController.evaluateAll">false</boolProp>
+          <boolProp name="IfController.useExpression">true</boolProp>
+        </IfController>
+        <hashTree/>
+        <kg.apc.jmeter.samplers.DummySampler guiclass="kg.apc.jmeter.samplers.DummySamplerGui" testclass="kg.apc.jmeter.samplers.DummySampler" testname="/example/request/extension" enabled="false">
+          <boolProp name="WAITING">true</boolProp>
+          <boolProp name="SUCCESFULL">true</boolProp>
+          <stringProp name="RESPONSE_CODE">200</stringProp>
+          <stringProp name="RESPONSE_MESSAGE">OK</stringProp>
+          <stringProp name="REQUEST_DATA">Dummy Sampler used to simulate requests and responses
+without actual network activity. This helps debugging tests.</stringProp>
+          <stringProp name="RESPONSE_DATA">Dummy Sampler used to simulate requests and responses
+without actual network activity. This helps debugging tests.</stringProp>
+          <stringProp name="RESPONSE_TIME">${__Random(50,500)}</stringProp>
+          <stringProp name="LATENCY">${__Random(1,50)}</stringProp>
+          <stringProp name="CONNECT">${__Random(1,5)}</stringProp>
+          <stringProp name="URL"></stringProp>
+          <stringProp name="RESULT_CLASS">org.apache.jmeter.samplers.SampleResult</stringProp>
+        </kg.apc.jmeter.samplers.DummySampler>
+        <hashTree/>
+      </hashTree>
+    </hashTree>
+  </hashTree>
+</jmeterTestPlan>

jira/extension/extension_ui.py → app/extension/confluence/extension_ui.py

@@ -1,13 +1,11 @@
-import random
-import time
-import urllib.parse
 from selenium.webdriver.common.by import By
-from selenium.webdriver.common.keys import Keys
-from selenium.webdriver.support.wait import WebDriverWait
 from selenium.webdriver.support import expected_conditions as EC
-from jira.selenium_ui.conftest import print_timing, AnyEc, application_url, generate_random_string
+from selenium.webdriver.support.wait import WebDriverWait
+
+from selenium_ui.conftest import print_timing
+from util.conf import CONFLUENCE_SETTINGS
 
-APPLICATION_URL = application_url()
+APPLICATION_URL = CONFLUENCE_SETTINGS.server_url
 timeout = 20
 
 
@@ -18,11 +16,14 @@ def measure(webdriver, interaction):
         def measure(webdriver, interaction):
             webdriver.get(f'{APPLICATION_URL}/plugins/servlet/some-app/reporter')
             WebDriverWait(webdriver, timeout).until(EC.visibility_of_element_located((By.ID, 'plugin-element')))
+
         measure(webdriver, 'selenium_app_custom_action:view_report')
 
         @print_timing
         def measure(webdriver, interaction):
             webdriver.get(f'{APPLICATION_URL}/plugins/servlet/some-app/administration')
             WebDriverWait(webdriver, timeout).until(EC.visibility_of_element_located((By.ID, 'plugin-dashboard')))
+
         measure(webdriver, 'selenium_app_custom_action:view_dashboard')
+
     measure(webdriver, 'selenium_app_custom_action')

app/extension/jira/extension_ui.py

@@ -0,0 +1,26 @@
+from selenium.webdriver.common.by import By
+from selenium.webdriver.support import expected_conditions as EC
+from selenium.webdriver.support.wait import WebDriverWait
+
+from selenium_ui.conftest import print_timing
+from util.conf import JIRA_SETTINGS
+
+APPLICATION_URL = JIRA_SETTINGS.server_url
+timeout = 20
+
+
+def custom_action(webdriver, datasets):
+    @print_timing
+    def measure(webdriver, interaction):
+        @print_timing
+        def measure(webdriver, interaction):
+            webdriver.get(f'{APPLICATION_URL}/plugins/servlet/some-app/reporter')
+            WebDriverWait(webdriver, timeout).until(EC.visibility_of_element_located((By.ID, 'plugin-element')))
+        measure(webdriver, 'selenium_app_custom_action:view_report')
+
+        @print_timing
+        def measure(webdriver, interaction):
+            webdriver.get(f'{APPLICATION_URL}/plugins/servlet/some-app/administration')
+            WebDriverWait(webdriver, timeout).until(EC.visibility_of_element_located((By.ID, 'plugin-dashboard')))
+        measure(webdriver, 'selenium_app_custom_action:view_dashboard')
+    measure(webdriver, 'selenium_app_custom_action')