-
Notifications
You must be signed in to change notification settings - Fork 53
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
360 changed files
with
8,482 additions
and
1,773 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -14,3 +14,5 @@ bolts: | |
| parallelism: 1 | ||
| - id: "TICK_BOLT" | ||
| parallelism: 1 | ||
| - id: "FLOW_STATE_CACHE_BOLT" | ||
| parallelism: 1 | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,6 +10,9 @@ kilda_orientdb_user: "kilda" | |
| kilda_orientdb_password: "kilda" | ||
| kilda_orientdb_database: "kilda" | ||
|
|
||
| kilda_persistence_implementation: "orientdb" | ||
| #kilda_persistence_implementation: "orientdb-sql-mixture1" | ||
|
|
||
| kilda_kafka_hosts: "kafka.pendev:9092" | ||
| kilda_zookeeper_hosts: "zookeeper.pendev" | ||
| kilda_zookeeper_state_root: "kilda" | ||
|
|
@@ -20,7 +23,17 @@ kilda_opentsdb_hosts: "opentsdb.pendev" | |
| kilda_opentsdb_port: "4242" | ||
| kilda_storm_numbus_hosts: "nimbus.pendev" | ||
|
|
||
| kilda_hibernate_user: kilda | ||
| kilda_hibernate_password: kilda | ||
| kilda_hibernate_url: jdbc:mysql://mysql.pendev:3306/kilda | ||
|
|
||
| kilda_gui_port: 1010 | ||
| kilda_gui_db_dialect: org.hibernate.dialect.DerbyTenSevenDialect | ||
| kilda_gui_db_url: jdbc:derby:/app/data/database;create=true | ||
| kilda_gui_db_username: '' | ||
| kilda_gui_db_password: '' | ||
| kilda_gui_application_name_prefix: "Dev" | ||
|
|
||
| kilda_smtp_host: "127.0.0.1" | ||
| kilda_smtp_port: "25" | ||
| kilda_mail_from: "[email protected]" | ||
|
|
@@ -141,7 +154,6 @@ kilda_logging_port_server42_control_server_stub: 5008 | |
| kilda_logging_port_server42_control_storm_stub: 5009 | ||
| kilda_logging_port_tests: 5006 | ||
|
|
||
| kilda_gui_application_name_prefix: "Dev" | ||
|
|
||
| kilda_grpc_endpoint: "http://grpc-speaker.pendev" | ||
| kilda_grpc_rest_port: "8091" | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
48 changes: 48 additions & 0 deletions
48
docker/db-migration/migrations/010-enforce-definite-types-on-history-objects.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| --- | ||
| databaseChangeLog: | ||
| - changeSet: | ||
| id: tag | ||
| author: surabujin | ||
| changes: | ||
| - tagDatabase: | ||
| tag: 010-enforce-definite-types-on-history-objects | ||
|
|
||
| - changeSet: | ||
| id: add_unique_index_for_port_history | ||
| author: surabujin | ||
| changes: | ||
| - sql: "CREATE PROPERTY port_history.id IF NOT EXISTS STRING" | ||
| - sql: "CREATE INDEX port_history.id UNIQUE_HASH_INDEX" | ||
| rollback: | ||
| - sql: "DROP INDEX port_history.id" | ||
| - sql: "DROP PROPERTY port_history.id" | ||
|
|
||
| - changeSet: | ||
| id: enforce_definite_types_for_flow_dump_objects | ||
| author: surabujin | ||
| changes: | ||
| - sql: "CREATE PROPERTY flow_dump.bandwidth IF NOT EXISTS LONG" | ||
| - sql: "CREATE PROPERTY flow_dump.forward_cookie IF NOT EXISTS LONG" | ||
| - sql: "CREATE PROPERTY flow_dump.reverse_cookie IF NOT EXISTS LONG" | ||
| - sql: "CREATE PROPERTY flow_dump.src_port IF NOT EXISTS INTEGER" | ||
| - sql: "CREATE PROPERTY flow_dump.dst_port IF NOT EXISTS INTEGER" | ||
| - sql: "CREATE PROPERTY flow_dump.src_vlan IF NOT EXISTS INTEGER" | ||
| - sql: "CREATE PROPERTY flow_dump.dst_vlan IF NOT EXISTS INTEGER" | ||
| - sql: "CREATE PROPERTY flow_dump.src_inner_vlan IF NOT EXISTS INTEGER" | ||
| - sql: "CREATE PROPERTY flow_dump.dst_inner_vlan IF NOT EXISTS INTEGER" | ||
| - sql: "CREATE PROPERTY flow_dump.forward_meter_id IF NOT EXISTS LONG" | ||
| - sql: "CREATE PROPERTY flow_dump.reverse_meter_id IF NOT EXISTS LONG" | ||
| - sql: "CREATE PROPERTY flow_dump.max_latency IF NOT EXISTS LONG" | ||
| rollback: | ||
| - sql: "DROP PROPERTY flow_dump.max_latency" | ||
| - sql: "DROP PROPERTY flow_dump.reverse_meter_id" | ||
| - sql: "DROP PROPERTY flow_dump.forward_meter_id" | ||
| - sql: "DROP PROPERTY flow_dump.dst_inner_vlan" | ||
| - sql: "DROP PROPERTY flow_dump.src_inner_vlan" | ||
| - sql: "DROP PROPERTY flow_dump.dst_vlan" | ||
| - sql: "DROP PROPERTY flow_dump.src_vlan" | ||
| - sql: "DROP PROPERTY flow_dump.dst_port" | ||
| - sql: "DROP PROPERTY flow_dump.src_port" | ||
| - sql: "DROP PROPERTY flow_dump.reverse_cookie" | ||
| - sql: "DROP PROPERTY flow_dump.forward_cookie" | ||
| - sql: "DROP PROPERTY flow_dump.bandwidth" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| -- initial mysql schema | ||
|
|
||
| CREATE TABLE flow_event | ||
| ( | ||
| id integer primary key auto_increment, | ||
| flow_id text not null, | ||
| task_id text not null, | ||
| task_id_unique_key char(100) not null, | ||
| action text not null, | ||
| unstructured text not null, | ||
| time_create datetime(3) not null default now(3), | ||
| time_modify datetime(3), | ||
| event_time datetime(3), | ||
| index `ix_event_time` (event_time), | ||
| index `ix_flow_id` (flow_id(64)), | ||
| unique `ix_task_id` (task_id_unique_key(100)) | ||
| ); | ||
|
|
||
| CREATE TABLE flow_event_action | ||
| ( | ||
| id integer primary key auto_increment, | ||
| flow_event_id integer not null, | ||
| action text not null, | ||
| details text, | ||
| time_create datetime(3) not null default now(3), | ||
| time_modify datetime(3), | ||
| event_time datetime(3), | ||
| index `ix_flow_event_id` (flow_event_id), | ||
| foreign key `fk_flow_event_action_2_flow_event` (flow_event_id) | ||
| REFERENCES flow_event(id) | ||
| ON UPDATE CASCADE ON DELETE RESTRICT | ||
| ); | ||
|
|
||
| CREATE TABLE flow_event_dump | ||
| ( | ||
| id integer primary key auto_increment, | ||
| flow_event_id integer not null, | ||
| kind varchar(64) not null, | ||
| unstructured text not null, | ||
| time_create datetime(3) not null default now(3), | ||
| time_modify datetime(3), | ||
| index `ix_flow_event_id` (flow_event_id), | ||
| foreign key `fk_flow_event_dump_2_flow_event` (flow_event_id) | ||
| REFERENCES flow_event (id) | ||
| ON UPDATE CASCADE ON DELETE RESTRICT | ||
| ); | ||
|
|
||
| CREATE TABLE port_event | ||
| ( | ||
| id char(36) primary key, -- uuid | ||
| switch_id char(23) not null, | ||
| port_number integer not null, | ||
| event varchar(64) not null, | ||
| unstructured text not null, | ||
| time_create datetime(3) not null default now(3), | ||
| time_modify datetime(3), | ||
| event_time datetime(3), | ||
| index `ix_event_time` (event_time) | ||
| ); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| # Apache Derby To MySQL Migration | ||
| ## _Objective :_ | ||
|
|
||
| Goal of this document is to provide a stepwise guide to migrate your application from using Apache Derby to MySQL. We need to ensure that exported data structure is compatible with MySQL so that data can be imported to MySQL without any issue. | ||
|
|
||
| ## Pre-requisite: | ||
|
|
||
| - **Required JAVA:** To check if JAVA_HOME is setup run below command | ||
| - ```echo $JAVA_HOME``` | ||
| - **Required MySQL:** Click [here](https://dev.mysql.com/doc/mysql-installation-excerpt/5.7/en/) to view Mysql Installation guide | ||
| - **Script Files** : Below mentioned script files that are needed for complete migration and are available [here](/tools/derby-to-mysql-migration/script-files). | ||
| **a. derbymysqlmigration.sh:** bash file that contains script to setup and install derby tools (dblook and ij). script to connect to ij and export the derby data to CSV files. These CSV files will later be used to import the data in MYSQL. | ||
| **b. derby-mysql-import.sh:** bash file that contains the script which will lookup the derby db metadata in form of SQLs and then will update those SQLs to make them compatible with MYSQL and will create a MySQL compatible .sql metadata file. Then will create the database, along with tables and import all the table records. | ||
| **c. derby-export.sql:** sql file that contains scripts to export the data from each Derby db table into CSV files. | ||
| **d. derby-import.sql:** sql file that contains commands to import the data in MYSQL database tables. | ||
| **e. derby.properties:** properties file to configure MySQL Database | ||
| **f. input.txt:** input file for IJ configuration | ||
|
|
||
| ## Migration | ||
|
|
||
| User will have to navigate to below mentioned location and execute following commands: | ||
|
|
||
| - ```cd /tools/derby-to-mysql-migration``` | ||
| - ```make pre-req "olddb=<oldDBName>" "newdb=<newDBName>" "derbydb=<derbyDBPath>"``` | ||
|
|
||
| **a. oldDBName:** Derby database name, which is SA by default | ||
| **b. newDBName:** Name of the database you want to create in MySQL | ||
| **c. derbyDBPath:** Path of your existing Derby database | ||
| - Stop the application that is running with Derby database from which DB migration has to be done, in order to avoid any data loss. Also, this is mandatory for the successful execution of bash scripts. | ||
|
|
||
| ## Export & Import metadata and data from existing Derby to MySQL DB: | ||
| - ```make export``` | ||
| - ```make import``` | ||
|
|
||
| This will lookup the metadata in form of SQLs and then will update those SQLs to make them compatible to MySQL. On execution, It will create the database in MySQL, along with all the tables and will also import all the records in db tables. **It will promt you for MySQL password twice**. | ||
|
|
||
| ## Verification | ||
| Follow below mentioned steps to Verify if scripts execution is successful and database is created successfully along with each table and its records. | ||
| - ```mysql -u root -p ``` | ||
| Enter Password: | ||
| - ```mysql> use {dbname};``` | ||
| - ```mysql> show tables;``` | ||
| - ```mysql> select * from {tablename};``` | ||
| ## Cleanup | ||
| In case there's a failure in migration at any step, first navigate to /opt directory and execute below command to cleanup the process and then start again from scratch i.e by navigating to /tools/derby-to-mysql-migration directory and then execute commands. | ||
|
|
||
| - ```cd /opt``` | ||
| - ```rm -rf /opt/derby``` |
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.