updates for version 3.2.7

Author: WillTheFarmer

.github/CHANGELOG.md

@@ -13,6 +13,7 @@
 - version 3.2.0 - 02/01/2025 - MariaDB compatible and Log Rotation
 - version 3.2.5 - 02/06/2025 - Log Generator Stress Test Improvements
 - version 3.2.6 - 02/14/2025 - consolidation of 25 source code scripts refined for MySQL & MariaDB
+- version 3.2.7 - 02/28/2025 - added error message Exception details, resolved backslash & forward slash issues, schema script improvements - see changelog
 - [1.0.1] apache_logs.error_systemCodeID corrected line - INTO logsystemCode to INTO logsystemCodeID
 - [1.0.1] remove debugging - SELECT statement from apache_logs.process_access_import, process_error_import & normalize_useragent.
 - [1.0.1] remove whitespace and commented out old code on all stored programs
@@ -121,3 +122,7 @@
 - [3.2.6] repository had MySQL Workbench generated script that added CHARSET and COLLAT. The raw source code scripts did not specify CHARSET or COLLATE. Raw source code scripts worked fine on MySQL and MariaDB testing.
 - [3.2.6] MySQL 9.1 default is CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci. MariaDB 11.6 default is CHARSET=utf8mb4 COLLATE=utf8mb4_uca1400_ai_ci. COLLATE=utf8mb4_0900_ai_ci does not exist in MariaDB causing script errors.
 - [3.2.6] The solution that works for both MySQL and MariaDB is only specify CHARSET=utf8mb4. If CHARACTER SET charset_name is specified without COLLATE, character set charset_name and its default collation are used.
+- [3.2.7] add except Exception as e: to all previous except: statements. e is printed with error message to console and logged to `import_error` TABLE.
+- [3.2.7] fixed MacOS and Linux platforms issue with double seperators in paths stored in `import_file` TABLE. This was a result of fixing the double separator on Windows platform. Issue now fixed on all 3 platforms.
+- [3.2.7] modify `apache_logs_schema.sql` generation script to comment out DROP statements and add comment to start of each merged file. 
+- [3.2.7] add two indexes for companion Web Interface - mysql-to-apache-echarts which is due to be released mid-March.

.github/README.md

@@ -221,7 +221,7 @@ Only MySQL server must be configured in `my.ini`, `mysqld.cnf` or `my.cnf` depen
 local-infile=1
 ```
 ### 3. Create Database USER & GRANTS
-To minimize data exposure and breach risks create a Database USER for Python module with GRANTS to only schema objects and privileges required to execute import processes. Replace hostname from `%` to hostname of database such as `localhost` to only allow USER access from single location. (`mysql_user_and_grants.sql` in repository)
+To minimize data exposure and breach risks create a Database USER for Python module with GRANTS to only schema objects and privileges required to execute import processes. Replace hostname from `localhost` to hostname of installed database if different. (`mysql_user_and_grants.sql` in repository)
 ![mysql_user_and_grants.sql in repository](./assets/mysql_user_and_grants.png)
 ### 4. Settings.env Variables
 settings.env with default settings for Windows. Use backslash `\` on Windows platform. Subfolder searches return backslashes in path results. Backslash `\` for Windows and forward slash `/` for Linux and MacOS. 

.github/assets/mysql_user_and_grants.png

@@ -1,27 +1,27 @@
 -- Python MySQL USER for all client module connections
-CREATE USER `apache_upload`@`%` IDENTIFIED BY 'password';
--- Python module CALLS 6 Stored Procedures for log processing
-GRANT EXECUTE ON PROCEDURE apache_logs.process_access_parse TO `apache_upload`@`%`;
-GRANT EXECUTE ON PROCEDURE apache_logs.process_access_import TO `apache_upload`@`%`;
-GRANT EXECUTE ON PROCEDURE apache_logs.process_error_parse TO `apache_upload`@`%`;
-GRANT EXECUTE ON PROCEDURE apache_logs.process_error_import TO `apache_upload`@`%`;
-GRANT EXECUTE ON PROCEDURE apache_logs.normalize_useragent TO `apache_upload`@`%`;
-GRANT EXECUTE ON PROCEDURE apache_logs.normalize_client TO `apache_upload`@`%`;
+CREATE USER 'apache_upload'@'localhost' IDENTIFIED BY 'password';
+-- Python CALLS 5 Stored Procedures for log processing & 1 Stored Procedure for error logging
+GRANT EXECUTE ON PROCEDURE apache_logs.process_access_parse TO `apache_upload`@`localhost`;
+GRANT EXECUTE ON PROCEDURE apache_logs.process_access_import TO `apache_upload`@`localhost`;
+GRANT EXECUTE ON PROCEDURE apache_logs.process_error_parse TO `apache_upload`@`localhost`;
+GRANT EXECUTE ON PROCEDURE apache_logs.process_error_import TO `apache_upload`@`localhost`;
+GRANT EXECUTE ON PROCEDURE apache_logs.normalize_useragent TO `apache_upload`@`localhost`;
+GRANT EXECUTE ON PROCEDURE apache_logs.normalize_client TO `apache_upload`@`localhost`;
 -- Python module CALLS 1 Stored Procedures for error logging
-GRANT EXECUTE ON PROCEDURE apache_logs.errorLoad TO `apache_upload`@`%`;
+GRANT EXECUTE ON PROCEDURE apache_logs.errorLoad TO `apache_upload`@`localhost`;
 -- Python module SELECTS 5 Stored Functions for log processing
-GRANT EXECUTE ON FUNCTION apache_logs.importDeviceID TO `apache_upload`@`%`;
-GRANT EXECUTE ON FUNCTION apache_logs.importClientID TO `apache_upload`@`%`;
-GRANT EXECUTE ON FUNCTION apache_logs.importLoadID TO `apache_upload`@`%`;
-GRANT EXECUTE ON FUNCTION apache_logs.importFileExists TO `apache_upload`@`%`;
-GRANT EXECUTE ON FUNCTION apache_logs.importFileID TO `apache_upload`@`%`;
+GRANT EXECUTE ON FUNCTION apache_logs.importDeviceID TO `apache_upload`@`localhost`;
+GRANT EXECUTE ON FUNCTION apache_logs.importClientID TO `apache_upload`@`localhost`;
+GRANT EXECUTE ON FUNCTION apache_logs.importLoadID TO `apache_upload`@`localhost`;
+GRANT EXECUTE ON FUNCTION apache_logs.importFileExists TO `apache_upload`@`localhost`;
+GRANT EXECUTE ON FUNCTION apache_logs.importFileID TO `apache_upload`@`localhost`;
 -- Python module INSERTS into 4 TABLES executing LOAD DATA LOCAL INFILE for log processing
-GRANT INSERT ON apache_logs.load_access_combined TO `apache_upload`@`%`;
-GRANT INSERT ON apache_logs.load_access_csv2mysql TO `apache_upload`@`%`;
-GRANT INSERT ON apache_logs.load_access_vhost TO `apache_upload`@`%`;
-GRANT INSERT ON apache_logs.load_error_default TO `apache_upload`@`%`;
+GRANT INSERT ON apache_logs.load_access_combined TO `apache_upload`@`localhost`;
+GRANT INSERT ON apache_logs.load_access_csv2mysql TO `apache_upload`@`localhost`;
+GRANT INSERT ON apache_logs.load_access_vhost TO `apache_upload`@`localhost`;
+GRANT INSERT ON apache_logs.load_error_default TO `apache_upload`@`localhost`;
 -- Python module issues SELECT and UPDATE statements on 3 TABLES due to converting parameters.
 -- Only reason TABLE direct access is number of parameters required for Stored Procedure.
-GRANT SELECT,UPDATE ON apache_logs.access_log_useragent TO `apache_upload`@`%`;
-GRANT SELECT,UPDATE ON apache_logs.import_load TO `apache_upload`@`%`;
-GRANT SELECT,UPDATE ON apache_logs.log_client TO `apache_upload`@`%`;
+GRANT SELECT,UPDATE ON apache_logs.access_log_useragent TO `apache_upload`@`localhost`;
+GRANT SELECT,UPDATE ON apache_logs.import_load TO `apache_upload`@`localhost`;
+GRANT SELECT,UPDATE ON apache_logs.log_client TO `apache_upload`@`localhost`;