RDoc-2882 ETL to Azure Queue Storage

Documentation/4.1/Raven.Documentation.Pages/server/ongoing-tasks/.docs.json

@@ -1,19 +1,25 @@
 [
-  {
-    "Path": "general-info.markdown",
-    "Name": "General Info",
-    "DiscussionId": "d24b385e-5eae-4171-b35d-277157b8e933",
-    "Mappings": []
-  },
-  {
-    "Path": "/etl",
-    "Name": "ETL",
-    "Mappings": []
-  },
-  {
-    "Path": "backup-overview.markdown",
-    "Name": "Backup Overview",
-    "DiscussionId": "2b3b34b3-04f0-4c1a-bfcf-40a9ed314f8a",
-    "Mappings": []
-  }
-]
+    {
+        "Path": "general-info.markdown",
+        "Name": "General Info",
+        "LastSupportedVersion": "4.1",
+        "DiscussionId": "d24b385e-5eae-4171-b35d-277157b8e933",
+        "Mappings": [
+            {
+                "Version": 4.2,
+                "Key": "studio/database/tasks/ongoing-tasks/general-info"
+            }
+        ]
+    },
+    {
+        "Path": "/etl",
+        "Name": "ETL",
+        "Mappings": []
+    },
+    {
+        "Path": "backup-overview.markdown",
+        "Name": "Backup Overview",
+        "DiscussionId": "2b3b34b3-04f0-4c1a-bfcf-40a9ed314f8a",
+        "Mappings": []
+    }
+]

Documentation/5.2/Raven.Documentation.Pages/client-api/operations/what-are-operations.dotnet.markdown

@@ -9,8 +9,8 @@
   and the **[Session](../../client-api/session/what-is-a-session-and-how-does-it-work)**.  
   They in turn are built on top of the lower-level __Operations__ and __RavenCommands__ API.
 
-* __RavenDB provides access to this lower-level API__, so that instead of using the higher session API,  
-  you can generate requests directly to the server by executing operations on the DocumentStore.
+* **RavenDB provides direct access to this lower-level API**, allowing you to send requests  
+  directly to the server via DocumentStore Operations instead of using the higher-level Session API.
 
 * In this page:  
   * [Why use operations](../../client-api/operations/what-are-operations#why-use-operations)  

Documentation/5.2/Raven.Documentation.Pages/client-api/operations/what-are-operations.js.markdown

@@ -9,8 +9,8 @@
   and the **[Session](../../client-api/session/what-is-a-session-and-how-does-it-work)**.  
   They in turn are built on top of the lower-level __Operations__ and __RavenCommands__ API.
 
-* __RavenDB provides access to this lower-level API__, so that instead of using the higher session API,  
-  you can generate requests directly to the server by executing operations on the DocumentStore.
+* **RavenDB provides direct access to this lower-level API**, allowing you to send requests  
+  directly to the server via DocumentStore Operations instead of using the higher-level Session API.
 
 * In this page:  
   * [Why use operations](../../client-api/operations/what-are-operations#why-use-operations)  

Documentation/5.2/Raven.Documentation.Pages/server/ongoing-tasks/etl/olap.markdown

@@ -59,8 +59,10 @@ The OLAP connection string contains the configurations for each destination of t
 | `GoogleCloudSettings` | Settings for Google Cloud Platform. |
 | `FTPSettings` | Settings for File Transfer Protocol. |
 
-{NOTE: ETL Destination Settings}
-<br/>
+{NOTE: }
+
+#### ETL destination settings
+
 This is the list of different settings objects that the `OlapConnectionString` object can contain.  
 
 #### `LocalSettings`

Documentation/5.2/Raven.Documentation.Pages/server/security/authorization/security-clearance-and-permissions.markdown

@@ -125,7 +125,7 @@ The following operations are **forbidden**:
 
 - Creating documents or modifying existing documents  
 - Changing any configurations or settings  
-- Creating or modifying [ongoing tasks](../../../server/ongoing-tasks/general-info)  
+- Creating or modifying [ongoing tasks](../../../studio/database/tasks/ongoing-tasks/general-info)  
 - Defining [static indexes](../../../indexes/creating-and-deploying#static-indexes) (the database will create 
 [auto-indexes](../../../indexes/creating-and-deploying#auto-indexes) if there is no existing index that satisfies a query.)
 

Documentation/5.3/Raven.Documentation.Pages/client-api/operations/maintenance/etl/add-etl.java.markdown

@@ -0,0 +1,52 @@
+# Operations: How to Add ETL
+
+You can add ETL task by using **AddEtlOperation**.
+
+## Syntax
+
+{CODE:java add_etl_operation@ClientApi\Operations\AddEtl.java /}
+
+| Parameters | | |
+| ------------- | ----- | ---- |
+| **configuration** | `EtlConfiguration<T>` | ETL configuration where `T` is connection string type |
+
+## Example - Add Raven ETL
+
+{NOTE: Secure servers}
+ To [connect secure RavenDB servers](../../../../server/security/authentication/certificate-management#enabling-communication-between-servers:-importing-and-exporting-certificates) 
+ you need to 
+
+  1. Export the server certificate from the source server. 
+  2. Install it as a client certificate on the destination server.  
+
+ This can be done in the RavenDB Studio -> Server Management -> [Certificates view](../../../../server/security/authentication/certificate-management#studio-certificates-management-view).
+{NOTE/}
+
+
+{CODE:java add_raven_etl@ClientApi\Operations\AddEtl.java /}
+
+## Example - Add Sql ETL
+
+{CODE:java add_sql_etl@ClientApi\Operations\AddEtl.java /}
+
+## Example - Add OLAP ETL
+
+{CODE:java add_olap_etl@ClientApi\Operations\AddEtl.java /}
+
+## Related Articles
+
+### ETL
+
+- [Basics](../../../../server/ongoing-tasks/etl/basics)
+- [RavenDB ETL](../../../../server/ongoing-tasks/etl/raven)
+- [SQL ETL](../../../../server/ongoing-tasks/etl/sql)
+
+### Studio
+
+- [RavenDB ETL Task](../../../../studio/database/tasks/ongoing-tasks/ravendb-etl-task)
+
+### Connection Strings
+
+- [Add](../../../../client-api/operations/maintenance/connection-strings/add-connection-string)
+- [Get](../../../../client-api/operations/maintenance/connection-strings/get-connection-string)
+- [Remove](../../../../client-api/operations/maintenance/connection-strings/remove-connection-string)

Documentation/5.3/Raven.Documentation.Pages/client-api/operations/maintenance/etl/add-etl.js.markdown

@@ -0,0 +1,41 @@
+# Operations: How to Add ETL
+
+You can add ETL task by using **AddEtlOperation**.
+
+## Usage
+
+{CODE:nodejs add_etl_operation@client-api\Operations\addEtl.js  /}
+
+| Parameters | | |
+| ------------- | ----- | ---- |
+| **configuration** | `EtlConfiguration<T>` | ETL configuration where `T` is connection string type |
+
+## Example - Add Raven ETL
+
+{CODE:nodejs add_raven_etl@client-api\Operations\addEtl.js  /}
+
+## Example - Add Sql ETL
+
+{CODE:nodejs add_sql_etl@client-api\Operations\addEtl.js  /}
+
+## Example - Add OLAP ETL
+
+{CODE:nodejs add_olap_etl@client-api\Operations\addEtl.js  /}
+
+## Related Articles
+
+### ETL
+
+- [Basics](../../../../server/ongoing-tasks/etl/basics)
+- [RavenDB ETL](../../../../server/ongoing-tasks/etl/raven)
+- [SQL ETL](../../../../server/ongoing-tasks/etl/sql)
+
+### Studio
+
+- [RavenDB ETL Task](../../../../studio/database/tasks/ongoing-tasks/ravendb-etl-task)
+
+### Connection Strings
+
+- [Add](../../../../client-api/operations/maintenance/connection-strings/add-connection-string)
+- [Get](../../../../client-api/operations/maintenance/connection-strings/get-connection-string)
+- [Remove](../../../../client-api/operations/maintenance/connection-strings/remove-connection-string)

Documentation/5.3/Raven.Documentation.Pages/client-api/operations/what-are-operations.dotnet.markdown

@@ -9,8 +9,8 @@
   and the **[Session](../../client-api/session/what-is-a-session-and-how-does-it-work)**.  
   They in turn are built on top of the lower-level __Operations__ and __RavenCommands__ API.
 
-* __RavenDB provides access to this lower-level API__, so that instead of using the higher session API,  
-  you can generate requests directly to the server by executing operations on the DocumentStore.
+* **RavenDB provides direct access to this lower-level API**, allowing you to send requests  
+  directly to the server via DocumentStore Operations instead of using the higher-level Session API.
 
 * In this page:  
   * [Why use operations](../../client-api/operations/what-are-operations#why-use-operations)  

Documentation/5.3/Raven.Documentation.Pages/client-api/operations/what-are-operations.js.markdown

@@ -9,8 +9,8 @@
   and the **[Session](../../client-api/session/what-is-a-session-and-how-does-it-work)**.  
   They in turn are built on top of the lower-level __Operations__ and __RavenCommands__ API.
 
-* __RavenDB provides access to this lower-level API__, so that instead of using the higher session API,  
-  you can generate requests directly to the server by executing operations on the DocumentStore.
+* **RavenDB provides direct access to this lower-level API**, allowing you to send requests  
+  directly to the server via DocumentStore Operations instead of using the higher-level Session API.
 
 * In this page:  
   * [Why use operations](../../client-api/operations/what-are-operations#why-use-operations)  

Documentation/5.3/Raven.Documentation.Pages/studio/database/tasks/ongoing-tasks/olap-etl-task.markdown

@@ -1,6 +1,5 @@
 # OLAP ETL Task
-
---
+---
 
 {NOTE: }
 

Documentation/5.3/Samples/java/src/test/java/net/ravendb/ClientApi/Operations/AddEtl.java

@@ -0,0 +1,122 @@
+package net.ravendb.ClientApi.Operations;
+
+import net.ravendb.client.documents.DocumentStore;
+import net.ravendb.client.documents.IDocumentStore;
+import net.ravendb.client.documents.operations.etl.*;
+import net.ravendb.client.documents.operations.etl.olap.OlapConnectionString;
+import net.ravendb.client.documents.operations.etl.olap.OlapEtlConfiguration;
+import net.ravendb.client.documents.operations.etl.sql.SqlConnectionString;
+import net.ravendb.client.documents.operations.etl.sql.SqlEtlConfiguration;
+import net.ravendb.client.documents.operations.etl.sql.SqlEtlTable;
+
+import java.util.Arrays;
+
+public class AddEtl {
+
+    private interface IFoo<T> {
+        /*
+        //region add_etl_operation
+        public AddEtlOperation(EtlConfiguration<T> configuration);
+        //endregion
+        */
+    }
+
+    public AddEtl() {
+        try (IDocumentStore store = new DocumentStore()) {
+            //region add_raven_etl
+            RavenEtlConfiguration configuration = new RavenEtlConfiguration();
+            configuration.setName("Employees ETL");
+            Transformation transformation = new Transformation();
+            transformation.setName("Script #1");
+            transformation.setScript("loadToEmployees ({\n" +
+                "  Name: this.FirstName + ' ' + this.LastName,\n" +
+                "  Title: this.Title\n" +
+                "});");
+
+            configuration.setTransforms(Arrays.asList(transformation));
+            AddEtlOperation<RavenConnectionString> operation = new AddEtlOperation<>(configuration);
+            AddEtlOperationResult result = store.maintenance().send(operation);
+            //endregion
+        }
+
+        try (IDocumentStore store = new DocumentStore()) {
+            //region add_sql_etl
+            SqlEtlConfiguration configuration = new SqlEtlConfiguration();
+            SqlEtlTable table1 = new SqlEtlTable();
+            table1.setTableName("Orders");
+            table1.setDocumentIdColumn("Id");
+            table1.setInsertOnlyMode(false);
+
+            SqlEtlTable table2 = new SqlEtlTable();
+            table2.setTableName("OrderLines");
+            table2.setDocumentIdColumn("OrderId");
+            table2.setInsertOnlyMode(false);
+
+            configuration.setSqlTables(Arrays.asList(table1, table2));
+            configuration.setName("Order to SQL");
+            configuration.setConnectionStringName("sql-connection-string-name");
+
+            Transformation transformation = new Transformation();
+            transformation.setName("Script #1");
+            transformation.setCollections(Arrays.asList("Orders"));
+            transformation.setScript("var orderData = {\n" +
+                "    Id: id(this),\n" +
+                "    OrderLinesCount: this.Lines.length,\n" +
+                "    TotalCost: 0\n" +
+                "};\n" +
+                "\n" +
+                "    for (var i = 0; i < this.Lines.length; i++) {\n" +
+                "        var line = this.Lines[i];\n" +
+                "        orderData.TotalCost += line.PricePerUnit;\n" +
+                "\n" +
+                "        // Load to SQL table 'OrderLines'\n" +
+                "        loadToOrderLines({\n" +
+                "            OrderId: id(this),\n" +
+                "            Qty: line.Quantity,\n" +
+                "            Product: line.Product,\n" +
+                "            Cost: line.PricePerUnit\n" +
+                "        });\n" +
+                "    }\n" +
+                "    orderData.TotalCost = Math.round(orderData.TotalCost  * 100) / 100;\n" +
+                "\n" +
+                "    // Load to SQL table 'Orders'\n" +
+                "    loadToOrders(orderData)");
+
+            configuration.setTransforms(Arrays.asList(transformation));
+
+            AddEtlOperation<SqlConnectionString> operation = new AddEtlOperation<>(configuration);
+
+            AddEtlOperationResult result = store.maintenance().send(operation);
+            //endregion
+        }
+
+        try (IDocumentStore store = new DocumentStore()) {
+            //region add_olap_etl
+            OlapEtlConfiguration configuration = new OlapEtlConfiguration();
+
+            configuration.setName("Orders ETL");
+            configuration.setConnectionStringName("olap-connection-string-name");
+
+            Transformation transformation = new Transformation();
+            transformation.setName("Script #1");
+            transformation.setCollections(Arrays.asList("Orders"));
+            transformation.setScript("var orderDate = new Date(this.OrderedAt);\n"+
+                "var year = orderDate.getFullYear();\n"+
+                "var month = orderDate.getMonth();\n"+
+                "var key = new Date(year, month);\n"+
+                "loadToOrders(key, {\n"+
+                "    Company : this.Company,\n"+
+                "    ShipVia : this.ShipVia\n"+
+                "})"
+            );
+
+            configuration.setTransforms(Arrays.asList(transformation));
+
+            AddEtlOperation<OlapConnectionString> operation = new AddEtlOperation<OlapConnectionString>(configuration);
+
+            AddEtlOperationResult result = store.maintenance().send(operation);
+            //endregion
+        }
+
+    }
+}