Skip to content

Enhancement - SinergiaDA - October2/November/December #478

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Jan 22, 2025

Conversation

juanSTIC
Copy link
Collaborator

@juanSTIC juanSTIC commented Nov 13, 2024

1. Añadir ColourPicker y collection como tipos de campo excluidos del procesamiento.

Como validarlo

2. Mejoras en la gestión de tabla sda_def_permissions

Se ha visto que en escenarios con un alto número de permisos (usuarios*grupos) el procesamiento es excesivamente lento si sda_def_permissions es una vista y no una tabla, por lo que se revierte este cambio hecho en la anterior versión. Paralelamente se mejora el proceso de guardado de los registros de esta tabla, que hasta el momento se hacía por cada registro, y ahora se procesan en bloque, haciendo
Se ha establecido una limitación práctica de 100 usuarios no administradores + todos los usuarios administradores de la instancia (los usuarios administradores no añadirán muchos registros ya que no tienen limitaciones de acceso a módulos). Este límite puede ser aumentado o disminuido usando la variable $sugar_config['stic_sinergiada']['max_users_processed']=<num> en config_override.php.

Cómo probarlo

  1. Ejecutar rebuild sin modificar la variable `$sugar_config['stic_sinergiada']['max_users_processed'] y verificar que se han procesado 100 usuarios no administradores + los usuarios no administradores
  2. Verificar que todos las tablas sda_def_* han sido pobladas correctamente, especialmente la tabla sda_def_permissions, que debe incluir los permisos de los usuarios procesados.
  3. Analizar los tiempos de respuesta y verificar que está dentro de unos límites aceptables. (< 1 minuto)
  4. Hacer alguna prueba estableciendo valores diferentes en `$sugar_config['stic_sinergiada']['max_users_processed']

3. Incluir solo a usuarios activos en la tabla sda_def_users

Ahora solo se incluyen en la tabla sda_def_users a los usuarios activos con SinergiaDA activado, puesto que esta tabla ya solamente se usa para el proceso de login de los usuarios y no es necesario que estén en la misma, como se creía hasta ahora, los usuario inactivos o con el acceso a SDA desactivado.
Cómo probarlo
Verificar que en tras el rebuild, la tabla sda_def_users y sda_def_users_groups unicamente incluyen a los usuarios activo y con SDA habilitado.

4. Eliminar operaciones DROP redundantes

Se han eliminado las operaciones DROP table redundantes existentes, sobre tablas que generaban errores del tipo [FATAL] en los logs. Aunque estos errores eran inocuos en cuanto a sus consecuencias, se ha corregido esta situación anómala.

Cómo probarlo
Verificar que tras reconstruir, no aparecen en el log entradas como las siguientes:

2024-12-06 02:02:03 [1944709][41b8185c-3817-3a9a-697e-551937375073][FATAL] Mysqli_query failed.
2024-12-06 02:02:03 [1944709][41b8185c-3817-3a9a-697e-551937375073][FATAL]  Query Failed: DROP TABLE sda_def_config: MySQL error 1051: Unknown table 'myaladina.sda_def_config'
2024-12-06 02:02:03 [1944709][41b8185c-3817-3a9a-697e-551937375073][FATAL] Mysqli_query failed.
2024-12-06 02:02:03 [1944709][41b8185c-3817-3a9a-697e-551937375073][FATAL]  Query Failed: DROP TABLE sda_def_tables: MySQL error 1051: Unknown table 'myaladina.sda_def_tables'
2024-12-06 02:02:03 [1944709][41b8185c-3817-3a9a-697e-551937375073][FATAL] Mysqli_query failed.
2024-12-06 02:02:03 [1944709][41b8185c-3817-3a9a-697e-551937375073][FATAL]  Query Failed: DROP TABLE sda_def_enumerations: MySQL error 1051: Unknown table 'myaladina.sda_def_enumerations'
2024-12-06 02:02:03 [1944709][41b8185c-3817-3a9a-697e-551937375073][FATAL] Mysqli_query failed.
2024-12-06 02:02:03 [1944709][41b8185c-3817-3a9a-697e-551937375073][FATAL]  Query Failed: DROP TABLE sda_stic_payments: MySQL error 1051: Unknown table 'myaladina.sda_stic_payments'
2024-12-06 02:02:03 [1944709][41b8185c-3817-3a9a-697e-551937375073][FATAL] Mysqli_query failed.
2024-12-06 02:02:03 [1944709][41b8185c-3817-3a9a-697e-551937375073][FATAL]  Query Failed: DROP TABLE sda_def_permissions_actions: MySQL error 1051: Unknown table 'myaladina.sda_def_permissions_actions'
2024-12-06 02:02:03 [1944709][41b8185c-3817-3a9a-697e-551937375073][FATAL] Mysqli_query failed.
2024-12-06 02:02:03 [1944709][41b8185c-3817-3a9a-697e-551937375073][FATAL]  Query Failed: DROP TABLE sda_def_relationships: MySQL error 1051: Unknown table 'myaladina.sda_def_relationships'
2024-12-06 02:02:03 [1944709][41b8185c-3817-3a9a-697e-551937375073][FATAL] Mysqli_query failed.
2024-12-06 02:02:03 [1944709][41b8185c-3817-3a9a-697e-551937375073][FATAL]  Query Failed: DROP TABLE sda_def_columns: MySQL error 1051: Unknown table 'myaladina.sda_def_columns'

5. Cambio de motor de tablas de SDA a InnoDB

Ahora las tablas sda_* se crean siempre como InnoDB, en consonancia con el cambio realizado en SinergiaCRM
Cómo probarlo
Verificar que tras reconstruir todas las tablas con prefijo sda_ tienen el motor InnoDB

6. Se corrige la causa de algunas entradas de log indebidas.

@juanSTIC juanSTIC marked this pull request as draft November 13, 2024 15:49
@juanSTIC juanSTIC self-assigned this Nov 13, 2024
@juanSTIC juanSTIC added SinergiaDA Improvement ... that do not affect user functionality labels Nov 26, 2024
@juanSTIC juanSTIC marked this pull request as ready for review November 26, 2024 15:28
@juanSTIC juanSTIC added the documentation Improvements or additions to documentation label Nov 26, 2024
@juanSTIC
Copy link
Collaborator Author

@BelenFFernandez te he citado en este PR porque aunque no es de SinergiaDA directamente establece límites en el uso de SinergiaDA y necesita ser documentado. Lo comentamos.

@AlbertoSTIC AlbertoSTIC changed the title Enhancement - SinergiaDA -October2/November Enhancement - SinergiaDA - October2/November Nov 28, 2024
PaulaaSTIC
PaulaaSTIC previously approved these changes Nov 28, 2024
Copy link
Collaborator

@PaulaaSTIC PaulaaSTIC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(A)probado

@juanSTIC juanSTIC changed the title Enhancement - SinergiaDA - October2/November Enhancement - SinergiaDA - October2/November/December Dec 10, 2024
Copy link
Collaborator

@AlbertoSTIC AlbertoSTIC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lo he probado en alberto.sinergiacrm.org y, aunque parece que las vistas se han creado, aparecen multiples mensajes de error de este tipo:

2024-12-16 11:36:31 [2585984][2][ERROR] Line 1917: ExternalReporting::batchInsertPermissions: Invalid record structure: Array
(
    [user_name] => pastic
    [table] => sda_stic_work_calendar
    [column] => users_id
    [stic_permission_source] => ACL_ALLOW_ALL
    [global] => 1
)

y

2024-12-16 11:36:32 [2585984][2][FATAL] Mysqli_query failed.
2024-12-16 11:36:32 [2585984][2][FATAL]  Query Failed: CREATE OR REPLACE VIEW sda_l_campaigns_notification_prospect_list_ids_dynamic_prospect_ AS : MySQL error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 1
2024-12-16 11:36:32 [2585984][2][ERROR] Line 1610: ExternalReporting::createEnumView: Error has occurred: [ Query Failed: CREATE OR REPLACE VIEW sda_l_campaigns_notification_prospect_list_ids_dynamic_prospect_ AS : MySQL error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 1] running Query: [CREATE OR REPLACE VIEW sda_l_campaigns_notification_prospect_list_ids_dynamic_prospect_ AS ]
2024-12-16 11:36:32 [2585984][2][FATAL] Mysqli_query failed.
2024-12-16 11:36:32 [2585984][2][FATAL]  Query Failed: CREATE OR REPLACE VIEW sda_l_campaigns_notification_template_id_dynamic_email_template_ AS : MySQL error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 1
2024-12-16 11:36:32 [2585984][2][ERROR] Line 1610: ExternalReporting::createEnumView: Error has occurred: [ Query Failed: CREATE OR REPLACE VIEW sda_l_campaigns_notification_template_id_dynamic_email_template_ AS : MySQL error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 1] running Query: [CREATE OR REPLACE VIEW sda_l_campaigns_notification_template_id_dynamic_email_template_ AS ]
2024-12-16 11:36:32 [2585984][2][FATAL] Mysqli_query failed.
2024-12-16 11:36:32 [2585984][2][FATAL]  Query Failed: CREATE OR REPLACE VIEW sda_l_campaigns_notification_outbound_email_id_dynamic_outbound_ AS : MySQL error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 1
2024-12-16 11:36:32 [2585984][2][ERROR] Line 1610: ExternalReporting::createEnumView: Error has occurred: [ Query Failed: CREATE OR REPLACE VIEW sda_l_campaigns_notification_outbound_email_id_dynamic_outbound_ AS : MySQL error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 1] running Query: [CREATE OR REPLACE VIEW sda_l_campaigns_notification_outbound_email_id_dynamic_outbound_ AS ]
2024-12-16 11:36:32 [2585984][2][FATAL] Mysqli_query failed.
2024-12-16 11:36:32 [2585984][2][FATAL]  Query Failed: CREATE OR REPLACE VIEW sda_l_campaigns_notification_inbound_email_id_dynamic_inbound_em AS : MySQL error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 1
2024-12-16 11:36:32 [2585984][2][ERROR] Line 1610: ExternalReporting::createEnumView: Error has occurred: [ Query Failed: CREATE OR REPLACE VIEW sda_l_campaigns_notification_inbound_email_id_dynamic_inbound_em AS : MySQL error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 1] running Query: [CREATE OR REPLACE VIEW sda_l_campaigns_notification_inbound_email_id_dynamic_inbound_em AS ]

@juanSTIC
Copy link
Collaborator Author

2024-12-16 11:36:31 [2585984][2][ERROR] Line 1917: ExternalReporting::batchInsertPermissions: Invalid record structure: Array
(
[user_name] => pastic
[table] => sda_stic_work_calendar
[column] => users_id
[stic_permission_source] => ACL_ALLOW_ALL
[global] => 1
)

Para este caso se ha añadido el valor para el campo group (null) en los dos sitios dónde no se estaba haciendo

@juanSTIC
Copy link
Collaborator Author

2024-12-16 11:36:32 [2585984][2][FATAL] Query Failed: CREATE OR REPLACE VIEW sda_l_campaigns_notification_prospect_list_ids_dynamic_prospect_ AS : MySQL error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' at line 1

Los logs de este tipo se producen porque algún campo referencia a listas desplegables que no existen, lo que no es habitual en producción, pero más común en entornos de desarrollo.

Se corrije haciendo un comprobación específica de si existe o no la lista desplegable, en caso de que no exista se muestra un log con más información y se sale de la función sin intentar hacer la query.

@juanSTIC juanSTIC requested a review from AlbertoSTIC December 16, 2024 15:30
Copy link
Collaborator

@AlbertoSTIC AlbertoSTIC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revisados los puntos anteriores y son correctos.

En mi instancia se ve este mensaje en errors.log:

[Tue Dec 17 11:06:24.601405 2024] [proxy_fcgi:error] [pid 2155803:tid 2155822] [remote 79.116.233.62:33474] AH01071: Got error 'PHP message: PHP Warning:  unlink(sdaRebuildError.txt): No such file or directory in /usr/home/alberto.sinergiacrm.org/web/SticInclude/SinergiaDARebuild.php on line 66', referer: https://alberto.sinergiacrm.org/index.php?module=Administration&action=sticmanagesdaintegration

@juanSTIC juanSTIC requested a review from AlbertoSTIC December 17, 2024 14:34
@juanSTIC
Copy link
Collaborator Author

juanSTIC commented Dec 17, 2024

En mi instancia se ve este mensaje en errors.log:
Aunque no es un código de este PR se ha corregido, evitando el mensaje de warning

Copy link
Collaborator

@AlbertoSTIC AlbertoSTIC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aprobado

Copy link
Collaborator

@PaulaaSTIC PaulaaSTIC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(A)probado

@enricsinergia enricsinergia merged commit 3392902 into develop Jan 22, 2025
1 check failed
@enricsinergia enricsinergia deleted the enhancement/SDAOctober2 branch January 22, 2025 05:19
Copy link

Actions executed at: 2025-01-22 05:19:14.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation Improvement ... that do not affect user functionality SinergiaDA
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incidencia - SinergiaDA - Errores de reconstrucción SDA en suitecrm.log
4 participants