added config and documentation

Issue 183

imixs-archive-exporter/README.md

@@ -2,49 +2,70 @@
 
 The _Imixs-Archive-Exporter Service_ is a microservice to export documents form a Imixs-Workflow data into an external storage like a FTP server or a local filesystem. This export service is decoupled from the a Imixs-Workflow Instance and can be run in a independent server environment. You can run this microservice on different hardware or in a external cluster.
 
-<img src="https://github.com/imixs/imixs-archive/raw/master/docs/imixs-backup-screen.png"/>
+<img src="https://github.com/imixs/imixs-archive/raw/master/docs/imixs-exporter-screen.png"/>
 
-# Configuration
+The Exporter Service can be run as an independent Microservice connecting to an Imixs-Workflow Instance. The exporter processes EventLog entries generated by the Workflow Instance to store file attachments into a separate data source like a local filesystem or a FTP server.
 
-The Exporter Service can be run in a container environment. To connect the exporter service with your workflow instance the following environment parameters are mandatory:
+# Export EventLog Entries
 
-      WORKFLOW_SERVICE_ENDPOINT: [REST-API-ENDPOINT-OF-WORKFLOW-INSTANCE]
-      WORKFLOW_SERVICE_USER: [BACKUP-USER]
-      WORKFLOW_SERVICE_PASSWORD: [PASSWORD]
-      WORKFLOW_SERVICE_AUTHMETHOD: [AUTHMETHOD]
+The Imixs-Archive-Exporter is controlled by corresponding [EventLog Entries](https://www.imixs.org/doc/engine/eventlogservice.html). An EventLog entity describes a unique event created during the processing life-cycle of a workitem or the update life-cycle of a Document. An eventLog entry for the Imixs-Archive-Exporter can be created by a BPMN Event result:
 
-      EXPORT_PATH: [DIRECTORY]
-      EXPORT_FTP_HOST: [FTP-SERVER-ADDRESS]
-      EXPORT_FTP_PORT: "21"
-      EXPORT_FTP_USER: [USER]
-      EXPORT_FTP_PASSWORD: [PASSWORD]
+```xml
+<eventlog name="file.export">
+	<ref><itemvalue>$uniqueid</itemvalue></ref>
+	<timeout>60000</timeout>
+        <document>
+             <path>my-documents/</path>
+        </document>
+ </eventlog>
+```
 
-      EVENTLOG_TOPIC : "file.export" (default topic)
-      EVENTLOG_DEADLOCK: 10000 (default timeout)
+On each corresponding worklfow event a new event log entry `file.export` will be created. The Imixs-Archive-Exporter periodically check this event log entries and exports the corresponding files into a configured data space.
 
-**Note:** The EXPORT-USER must have manager access.
+# Configuration
+
+The Exporter Service can be run in a container environment. To connect the exporter service can be configured via environment variables:
+
+| Environment Param           | Description                                                                          | Example                 |
+| --------------------------- | ------------------------------------------------------------------------------------ | ----------------------- |
+| WORKFLOW_SERVICE_ENDPOINT   | The Rest-API endpoint of a Imixs Workflow Instance providing export eventLog entries | http://my-host:8080/api |
+| WORKFLOW_SERVICE_USER       | Workflow user to access the Workflow Instance. This user must have manager access.   |                         |
+| WORKFLOW_SERVICE_PASSWORD   | Workflow user password                                                               |                         |
+| WORKFLOW_SERVICE_AUTHMETHOD | Authentication method (form/basic)                                                   | form                    |
+| EVENTLOG_TOPIC              | The EventLog topic (default = file.export), can be overwritten                       | file.export             |
+| EVENTLOG_DEADLOCK           | EventLog timeout in ms (default = 60000)                                             |                         |
+| EXPORT_PATH                 | Target directory to write files                                                      |                         |
 
 The following environment variables are optional and depend on the Jakarta EE Runtime:
 
-- METRICS_ENDPOINT = endpoint for metrics API (default = http://localhost:9990/metrics)
-- HEALTH_ENDPOINT = endpoint for health API (default = http://localhost:9990/health)
+| Environment Param   | Description                 | Example                       |
+| ------------------- | --------------------------- | ----------------------------- |
+| METRICS_ENDPOINT    | endpoint for metrics API    | http://localhost:9990/metrics |
+| HEALTH_ENDPOINT     | endpoint for health API     | http://localhost:9990/health  |
+| EXPORT_FTP_HOST     | Optional: FTP Server host   | my-ftp-server                 |
+| EXPORT_FTP_PORT     | Optional: FTP Server port   | 21                            |
+| EXPORT_FTP_USER     | Optional: FTP user          |                               |
+| EXPORT_FTP_PASSWORD | Optional: FTP user password |                               |
 
-To connect a Imixs-Workflow instance with the Exporter Service you need to add the Imixs-Archive API as a dependency.
+The Imixs-Workflow instance is not needed to be configured. Only a service user account need to be provided.
 
-```xml
-  <dependency>
-   <groupId>org.imixs.workflow</groupId>
-   <artifactId>imixs-archive-api</artifactId>
-   <version>${org.imixs.archive.version}</version>
-   <scope>provided</scope>
-  </dependency>
-```
+# Monitoring
+
+The Imixs-Archive-Exporter provides a Health and Metrics Endpoint that can be used to monitor the status of the service.
+
+The Health-Check endpoint 'http://localhost:9991/health' indicates if the server and exporter service are running.
+
+The Metric endpoint `http://localhost:9991/metric` provides detailed information about the processing status. The metrics are provided in a prometeus format and can be analyzed with corresponding Tools like Grafana.
+
+To run an example with A Prometeus and Grafana Service see the Docker-Compose Example `docker-compose-monitor.yml`. You can start the example with
+
+      $ docker-compose -f docker/docker-compose-monitor.yml
 
-Next set the environment parameter `backup.service.endpoint` to activate the backup.
+The Prometeus Dashboard can be loaded via : http://localhost:9090
 
-      EXPORT_SERVICE_ENDPOINT: [REST-API-ENDPOINT-OF-EXPORTSERVICE]
+<img src="https://github.com/imixs/imixs-archive/raw/master/docs/imixs-exporter-prometheus.png"/>
 
-On each save event a new event log entry `snapshot.export` will be created. The backup Service periodically check this event log entries and stores the corresponding snapshot into the backup space.
+The Grafana Dashboard can be loaded via : http://localhost:3000/login
 
 # Development
 

docker-compose-test.yml → imixs-archive-exporter/docker-compose-dev.yml

@@ -59,37 +59,37 @@ services:
       - "8788:8787"
       - "9991:9990"
     volumes:
-      - ./imixs-archive-exporter/docker/deployments:/opt/jboss/wildfly/standalone/deployments/
+      - ./docker/deployments:/opt/jboss/wildfly/standalone/deployments/
 
   ##################
   # Backup Service
   ##################
-  imixsbackup-app:
-    image: imixs/imixs-archive-backup:latest
-    environment:
-      WILDFLY_PASS: adminadmin
-      DEBUG: "true"
-
-      # EXAMPLE SETUP
-      WORKFLOW_SERVICE_ENDPOINT: "http://imixs-office:8080/api"
-      WORKFLOW_SERVICE_USER: "dataservice"
-      WORKFLOW_SERVICE_PASSWORD: "adminadmin"
-      WORKFLOW_SERVICE_AUTHMETHOD: "form"
-
-      EXPORT_FTP_HOST: "ftp.storage.foo"
-      EXPORT_FTP_PATH: "/test/backup-service"
-      EXPORT_FTP_PORT: "21"
-      EXPORT_FTP_USER: ""
-      EXPORT_FTP_PASSWORD: ""
-
-      METRICS_ENDPOINT: "http://localhost:9992/metrics"
-      HEALTH_ENDPOINT: "http://localhost:9992/health"
-    ports:
-      - "8082:8080"
-      - "8789:8787"
-      - "9992:9990"
-    volumes:
-      - ./imixs-archive-backup/docker/deployments:/opt/jboss/wildfly/standalone/deployments/
+  #imixsbackup-app:
+  #  image: imixs/imixs-archive-backup:latest
+  #  environment:
+  #    WILDFLY_PASS: adminadmin
+  #    DEBUG: "true"
+  #
+  #    # EXAMPLE SETUP
+  #    WORKFLOW_SERVICE_ENDPOINT: "http://imixs-office:8080/api"
+  #    WORKFLOW_SERVICE_USER: "dataservice"
+  #    WORKFLOW_SERVICE_PASSWORD: "adminadmin"
+  #    WORKFLOW_SERVICE_AUTHMETHOD: "form"
+  #
+  #    EXPORT_FTP_HOST: "ftp.storage.foo"
+  #    EXPORT_FTP_PATH: "/test/backup-service"
+  #    EXPORT_FTP_PORT: "21"
+  #    EXPORT_FTP_USER: ""
+  #    EXPORT_FTP_PASSWORD: ""
+  #
+  #    METRICS_ENDPOINT: "http://localhost:9992/metrics"
+  #    HEALTH_ENDPOINT: "http://localhost:9992/health"
+  #  ports:
+  #    - "8082:8080"
+  #    - "8789:8787"
+  #    - "9992:9990"
+  #  volumes:
+  #    - ./imixs-archive-backup/docker/deployments:/opt/jboss/wildfly/standalone/deployments/
 
   ##################
   # Admin Client

imixs-archive-exporter/docker-compose-monitor.yml

@@ -0,0 +1,94 @@
+version: "3"
+services:
+  ##################
+  # Database
+  ##################
+  db:
+    image: postgres:9.6.1
+    environment:
+      POSTGRES_PASSWORD: adminadmin
+      POSTGRES_DB: office
+    volumes:
+      - dbdata:/var/lib/postgresql/data
+
+  ##################
+  # Imixs-Office-Worklfow
+  ##################
+  imixs-office:
+    image: imixs/imixs-office-workflow:latest
+    depends_on:
+      - db
+    environment:
+      JAVA_OPTS: "-Dnashorn.args=--no-deprecation-warning"
+      POSTGRES_USER: "postgres"
+      POSTGRES_PASSWORD: "adminadmin"
+      POSTGRES_CONNECTION: "jdbc:postgresql://db/office"
+      TZ: "Europe/Berlin"
+      MAILGATEWAY: "localhost"
+    ports:
+      - "8080:8080"
+      - "8787:8787"
+      - "9990:9990"
+
+  ##################
+  # Exporter Service
+  ##################
+  imixsexporter-app:
+    image: imixs/imixs-archive-exporter:latest
+    environment:
+      WILDFLY_PASS: adminadmin
+      DEBUG: "true"
+
+      # EXAMPLE SETUP
+      WORKFLOW_SERVICE_ENDPOINT: "http://imixs-office:8080/api"
+      WORKFLOW_SERVICE_USER: "dataservice"
+      WORKFLOW_SERVICE_PASSWORD: "adminadmin"
+      WORKFLOW_SERVICE_AUTHMETHOD: "form"
+
+      METRICS_ENDPOINT: "http://localhost:9991/metrics"
+      HEALTH_ENDPOINT: "http://localhost:9991/health"
+      # Optional FTP Export
+      #EXPORT_FTP_HOST: "ftp.storage.foo"
+      #EXPORT_FTP_PATH: "/test/export-service/"
+      #EXPORT_FTP_PORT: "21"
+      #EXPORT_FTP_USER: ""
+      #EXPORT_FTP_PASSWORD: ""
+      EXPORT_PATH: "/opt/jboss/wildfly/test-export"
+    ports:
+      - "8081:8080"
+      - "8788:8787"
+      - "9991:9990"
+
+
+  ##################
+  # Prometheus
+  ##################
+  prometheus:
+    image: prom/prometheus:latest
+    ports:
+      - "9090:9090"
+    volumes:
+      - ./docker/configuration/prometheus.yml:/etc/prometheus/prometheus.yml 
+      - prometheusdata:/prometheus/
+
+  ##################
+  # Grafana
+  ##################
+  grafana:
+    image: grafana/grafana:latest
+    ports: 
+      - "3000:3000" 
+
+
+
+  ##################
+  # Admin Client
+  ##################
+  #imixsadmin:
+  #  image: imixs/imixs-admin
+  #  ports:
+  #    - "8888:8080"
+
+volumes:
+  dbdata:
+  prometheusdata:

imixs-archive-exporter/docker-compose.yml

@@ -1,25 +1,101 @@
 version: "3"
 services:
+  ##################
+  # Database
+  ##################
+  db:
+    image: postgres:9.6.1
+    environment:
+      POSTGRES_PASSWORD: adminadmin
+      POSTGRES_DB: office
+    volumes:
+      - dbdata:/var/lib/postgresql/data
+
+  ##################
+  # Imixs-Office-Worklfow
+  ##################
+  imixs-office:
+    image: imixs/imixs-office-workflow:latest
+    depends_on:
+      - db
+    environment:
+      JAVA_OPTS: "-Dnashorn.args=--no-deprecation-warning"
+      POSTGRES_USER: "postgres"
+      POSTGRES_PASSWORD: "adminadmin"
+      POSTGRES_CONNECTION: "jdbc:postgresql://db/office"
+      TZ: "Europe/Berlin"
+      MAILGATEWAY: "localhost"
+    ports:
+      - "8080:8080"
+      - "8787:8787"
+      - "9990:9990"
+
+  ##################
+  # Exporter Service
+  ##################
   imixsexporter-app:
     image: imixs/imixs-archive-exporter:latest
     environment:
       WILDFLY_PASS: adminadmin
       DEBUG: "true"
 
       # EXAMPLE SETUP
-      WORKFLOW_SERVICE_ENDPOINT: "http://app:8080/api"
-      WORKFLOW_SERVICE_USER: test
-      WORKFLOW_SERVICE_PASSWORD: ""
+      WORKFLOW_SERVICE_ENDPOINT: "http://imixs-office:8080/api"
+      WORKFLOW_SERVICE_USER: "dataservice"
+      WORKFLOW_SERVICE_PASSWORD: "adminadmin"
       WORKFLOW_SERVICE_AUTHMETHOD: "form"
 
-      EXPORT_FTP_HOST: "ftp.storage.foo"
-      EXPORT_FTP_PATH: "/test/"
-      EXPORT_FTP_PORT: "21"
-      EXPORT_FTP_USER: ""
-      EXPORT_FTP_PASSWORD: ""
+      METRICS_ENDPOINT: "http://localhost:9991/metrics"
+      HEALTH_ENDPOINT: "http://localhost:9991/health"
+      # Optional FTP Export
+      #EXPORT_FTP_HOST: "ftp.storage.foo"
+      #EXPORT_FTP_PATH: "/test/export-service/"
+      #EXPORT_FTP_PORT: "21"
+      #EXPORT_FTP_USER: ""
+      #EXPORT_FTP_PASSWORD: ""
+      EXPORT_PATH: "/opt/jboss/wildfly/test-export"
     ports:
-      - "8080:8080"
-      - "8787:8787"
-      - "9990:9990"
-    volumes:
-      - ./docker/deployments:/opt/jboss/wildfly/standalone/deployments/
+      - "8081:8080"
+      - "8788:8787"
+      - "9991:9990"
+
+  ##################
+  # Backup Service
+  ##################
+  #imixsbackup-app:
+  #  image: imixs/imixs-archive-backup:latest
+  #  environment:
+  #    WILDFLY_PASS: adminadmin
+  #    DEBUG: "true"
+  #
+  #    # EXAMPLE SETUP
+  #    WORKFLOW_SERVICE_ENDPOINT: "http://imixs-office:8080/api"
+  #    WORKFLOW_SERVICE_USER: "dataservice"
+  #    WORKFLOW_SERVICE_PASSWORD: "adminadmin"
+  #    WORKFLOW_SERVICE_AUTHMETHOD: "form"
+  #
+  #    EXPORT_FTP_HOST: "ftp.storage.foo"
+  #    EXPORT_FTP_PATH: "/test/backup-service"
+  #    EXPORT_FTP_PORT: "21"
+  #    EXPORT_FTP_USER: ""
+  #    EXPORT_FTP_PASSWORD: ""
+  #
+  #    METRICS_ENDPOINT: "http://localhost:9992/metrics"
+  #    HEALTH_ENDPOINT: "http://localhost:9992/health"
+  #  ports:
+  #    - "8082:8080"
+  #    - "8789:8787"
+  #    - "9992:9990"
+  #  volumes:
+  #    - ./imixs-archive-backup/docker/deployments:/opt/jboss/wildfly/standalone/deployments/
+
+  ##################
+  # Admin Client
+  ##################
+  imixsadmin:
+    image: imixs/imixs-admin
+    ports:
+      - "8888:8080"
+
+volumes:
+  dbdata: