Refactor to prepare for visits (#44)

* Fix/db migrations (#31)

* feat(db): upgrade down_revision following rebase

Since rebase with develop: changed the down_revision number

* fix(db): fix bind params enabling downgrade

Beforehand the downgrade was not possible...

* refactor(db): removed cor_site_type_category

* refactor(db): changed category into type in cor

* refactor(db): create cor_type_site

* fix(db): renamed column

* refactor(api): update models to fit migrations

* fix(db):change bib_categorie_site to bib_type_site

Adding :
cor_site_module
cor_site_type
revision alembic to create function and trigger in order to add
bib_type_site but only with nomenclature 'TYPE_SITE'
upgrade and downgrade works

[Refs ticket]: #3
Reviewed-by: andriac

* fix(api): updated models from migrations

* fix(api): wip: fix admin following migrations

* fix(api): update routes and tests

To match migration changes

* feat: flask admin bib_type_site

Change bib_categories to bib_type_site into flask admin
Adding filtering in list label_fr of type_site to secure the unique
constraint

Reviewed-by: andriac
[Refs ticket]: #3

* fix(api): updated schema to match models

* fix(api): module edition

* style(api): uniformize type_site

* style(api): change relationship name for type_site

* feat(api): validator admin

* fix(api): make unique BibTypeSite in admin

* test(api): fix test when existing nomenclatures

In database

Co-authored-by: Andria Capai <[email protected]>

* refactor: object.service with observable obj

Change the objectType string to objectType obj
in order to subscribe to multilple properties link to this object

Reviewed-by: andriacap
[Refs_ticket]: #40

* feat: adapt service to get config json

Get fieldsName and fieldsLabel for display properties
(properties component)

Add logic to setItem in LocalStorage to keepLast Value of observable on reload page

TODO:
- [ ]  see if Localstorage is really necessary with the configService used inside api-geom.service )
- [ ] adapt monitoring-form-g.component with the new service

Reviewed-by: andriacap
[Refs ticket]: #4

* refactor(front): svc: better use of types

* feat(front): add Visits component and service

* feat(back): add get site by id route & test

* feat(api): add visits routes and schema

* fix(api): join modules to have the modulecode

For the frontend to be able to redirect to the correct route

* feat(api): add sites/id/module route

To retrieve all the modules compatibles with this site
This lead to add a relationship. Set to noload so that it is not
loaded by other not "raiseloaded" queries

* test(api): add test to test the /sites/id/module

Route.
Also changed some fixture to be able to write these tests

* fix(front): remove double def of IGeomService

* refactor(front): remove Resp interface

Since it is not usefull anymore

* fix(api): exclude sites relationship

* fix(config): change KeyValue

Since id changed

* feat(front): make datatable accepts other add btn

To be able to customize the "add object" button

* feat(front): add btn select protocole for visit

Add new component: select-btn
Add call to route to get modules from a base site id
Call the new component in the visit component
Add interfaces
Add type in config service

* feat(front): parameter for label and placeholder

For the btn component.
Style menu and form-field to make them larger

* refactor(front): api service with generic types

* fix(front): div removed following rebase

* refactor(front): add service as input for formComp

So that formComp is more type generic

* fix(config): remove "s" from sites_group

* refactor(front): put initConfig in ApiService

* fix(front): remove "s" from visit

* fix(front): put back condition on css class

To make ng-content conditionnal

* fix: following rebase

Fix imports, double declarations...

* fix(api): redirect to url

* refactor(front): rename select-btn to option-list

* fix: remove contrib

---------

Co-authored-by: Andria Capai <[email protected]>

backend/gn_module_monitoring/config/generic/module.json

@@ -126,7 +126,7 @@
       "type_widget": "datalist",
       "attribut_label": "Types de sites",
       "type_util": "types_site",
-      "keyValue": "id_nomenclature",
+      "keyValue": "id_nomenclature_type_site",
       "keyLabel": "label",
       "multiple": true,
       "api" : "__MONITORINGS_PATH/sites/types",

backend/gn_module_monitoring/monitoring/admin.py

@@ -73,3 +73,4 @@ def list_label_nomenclature_formatter(view, _context, model, _name):
 
     column_list = ("nomenclature", "config")
     column_formatters = dict(nomenclature=list_label_nomenclature_formatter)
+    form_excluded_columns = "sites"

backend/gn_module_monitoring/monitoring/models.py

@@ -104,6 +104,12 @@ class BibTypeSite(DB.Model, GenericModel):
         uselist=False,
         backref=DB.backref('bib_type_site', uselist=False)
     )
+
+    sites = DB.relationship(
+        "TMonitoringSites",
+        secondary=cor_type_site,
+        lazy="noload"
+    )
 
 @serializable
 class TMonitoringObservationDetails(DB.Model):
@@ -209,7 +215,6 @@ class TMonitoringVisits(TBaseVisits, GenericModel):
         )
     )
 
-
     module = DB.relationship(
         TModules,
         lazy="select",

backend/gn_module_monitoring/monitoring/schemas.py

@@ -9,7 +9,7 @@
     BibTypeSite,
     TMonitoringSites,
     TMonitoringSitesGroups,
-    TMonitoringVisits
+    TMonitoringVisits,
 )
 
 

backend/gn_module_monitoring/routes/site.py

@@ -1,10 +1,13 @@
 from flask import request
 from flask.json import jsonify
+from sqlalchemy.orm import Load, joinedload
 from werkzeug.datastructures import MultiDict
 
+from geonature.core.gn_commons.schemas import ModuleSchema
+
 from gn_module_monitoring.blueprint import blueprint
 from gn_module_monitoring.config.repositories import get_config
-from gn_module_monitoring.monitoring.models import BibTypeSite, TMonitoringSites, TNomenclatures
+from gn_module_monitoring.monitoring.models import BibTypeSite, TMonitoringSites, TNomenclatures, TMonitoringModules
 from gn_module_monitoring.monitoring.schemas import BibTypeSiteSchema, MonitoringSitesSchema
 from gn_module_monitoring.routes.sites_groups import create_or_update_object_api
 from gn_module_monitoring.utils.routes import (
@@ -111,6 +114,20 @@ def get_all_site_geometries():
     return jsonify(result)
 
 
+@blueprint.route("/sites/<int:id_base_site>/modules", methods=["GET"])
+def get_module_by_id_base_site(id_base_site: int):
+    query = TMonitoringModules.query.options(
+        Load(TMonitoringModules).raiseload("*"),
+        joinedload(TMonitoringModules.types_site).options(joinedload(BibTypeSite.sites)),
+    ).filter(TMonitoringModules.types_site.any(BibTypeSite.sites.any(id_base_site=id_base_site)))
+
+    result = query.all()
+    schema = ModuleSchema()
+    # TODO: Is it usefull to put a limit here? Will there be more than 200 modules?
+    # If limit here, implement paginated/infinite scroll on frontend side
+    return [schema.dump(res) for res in result]
+
+
 @blueprint.route("/sites/module/<string:module_code>", methods=["GET"])
 def get_module_sites(module_code: str):
     # TODO: load with site_categories.json API

backend/gn_module_monitoring/routes/visit.py

@@ -24,7 +24,7 @@ def get_visits():
         params=params, default_sort="id_base_visit", default_direction="desc"
     )
     query = TMonitoringVisits.query.options(joinedload(TMonitoringVisits.module))
-    query = filter_params(query=TMonitoringVisits.query, params=params)
+    query = filter_params(query=query, params=params)
     query = sort(query=query, sort=sort_label, sort_dir=sort_dir)
 
     return paginate(

backend/gn_module_monitoring/tests/fixtures/module.py

@@ -21,3 +21,19 @@ def monitoring_module(types_site):
         db.session.add(t_monitoring_module)
 
     return t_monitoring_module
+
+
+@pytest.fixture
+def monitoring_module_wo_types_site():
+    t_monitoring_module = TMonitoringModules(
+        module_code=uuid4(),
+        module_label="NoType",
+        active_frontend=True,
+        active_backend=False,
+        module_path="NoType",
+    )
+
+    with db.session.begin_nested():
+        db.session.add(t_monitoring_module)
+
+    return t_monitoring_module

backend/gn_module_monitoring/tests/fixtures/site.py

@@ -23,6 +23,22 @@ def sites(users, types_site, site_group_with_sites):
             types_site=[types_site[key]],
             id_sites_group=site_group_with_sites.id_sites_group,
         )
+
+    # Add a special site that has no type
+    sites["no-type"] = TMonitoringSites(
+        id_inventor=user.id_role,
+        id_digitiser=user.id_role,
+        base_site_name=f"no-type",
+        base_site_description=f"Description-no-type",
+        base_site_code=f"Code-no-type",
+        geom=geom_4326,
+        # Random id_nomenclature_type_site
+        # FIXME: when id_nomenclature_type_site disapears => remove this line
+        id_nomenclature_type_site=list(types_site.values())[0].id_nomenclature_type_site,
+        types_site=[],
+        id_sites_group=site_group_with_sites.id_sites_group,
+    )
+
     with db.session.begin_nested():
         db.session.add_all(sites.values())
     return sites

backend/gn_module_monitoring/tests/fixtures/visit.py

@@ -1,7 +1,6 @@
 import datetime
 
 import pytest
-from geonature.tests.fixtures import datasets
 from geonature.utils.env import db
 
 from gn_module_monitoring.monitoring.models import TMonitoringVisits

backend/gn_module_monitoring/tests/test_routes/test_site.py

@@ -57,6 +57,14 @@ def test_get_sites_id_base_site(self, sites):
         assert len(r.json["items"]) == 1
         assert r.json["items"][0]["id_base_site"] == id_base_site
 
+    def test_get_sites_by_id(self, sites):
+        site = list(sites.values())[0]
+        id_base_site = site.id_base_site
+
+        r = self.client.get(url_for("monitorings.get_site_by_id", id_base_site=id_base_site))
+
+        assert r.json["id_base_site"] == id_base_site
+
     def test_get_all_site_geometries(self, sites):
         r = self.client.get(url_for("monitorings.get_all_site_geometries"))
 
@@ -85,6 +93,43 @@ def test_get_all_site_geometries_filter_site_group(self, sites, site_group_witho
         features = json_resp.get("features")
         assert features is None
 
+    def test_get_module_by_id_base_site(self, sites, monitoring_module):
+        site = list(sites.values())[0]
+        id_base_site = site.id_base_site
+
+        r = self.client.get(
+            url_for("monitorings.get_module_by_id_base_site", id_base_site=id_base_site)
+        )
+
+        expected_modules = {monitoring_module.id_module}
+        current_modules = {module["id_module"] for module in r.json}
+        assert expected_modules.issubset(current_modules)
+
+    def test_get_module_by_id_base_site_no_type_module(
+        self, sites, monitoring_module_wo_types_site
+    ):
+        site = list(sites.values())[0]
+        id_base_site = site.id_base_site
+
+        r = self.client.get(
+            url_for("monitorings.get_module_by_id_base_site", id_base_site=id_base_site)
+        )
+
+        expected_absent_modules = {monitoring_module_wo_types_site.id_module}
+        current_modules = {module["id_module"] for module in r.json}
+        assert expected_absent_modules.isdisjoint(current_modules)
+
+    def test_get_module_by_id_base_site_no_type_site(self, sites, monitoring_module):
+        id_base_site = sites["no-type"].id_base_site
+
+        r = self.client.get(
+            url_for("monitorings.get_module_by_id_base_site", id_base_site=id_base_site)
+        )
+
+        expected_modules = {monitoring_module.id_module}
+        current_modules = {module["id_module"] for module in r.json}
+        assert expected_modules.isdisjoint(current_modules)
+
     def test_get_module_sites(self):
         module_code = "TEST"
         r = self.client.get(url_for("monitorings.get_module_sites", module_code=module_code))

frontend/app/components/monitoring-datatable-g/monitoring-datatable-g.component.css

@@ -45,3 +45,9 @@
 .custom-dt {
   box-shadow: none !important;
 }
+
+/* Work around to make ng-content conditional 
+(not supported, maybe in Angular16) */
+.wrapper-button:not(:empty) + .default-button {
+    display: none;
+}

frontend/app/components/monitoring-datatable-g/monitoring-datatable-g.component.html

@@ -7,19 +7,24 @@
     (keyup)="updateFilter($event)"
   /> -->
 
-  <div class="btn-float-right md-2">
+  <div class="md-2 d-flex justify-content-between">
     <button
       class="btn btn-info"
       (click)="displayFilter = !displayFilter"
       matTooltip="{{ displayFilter ? 'Cacher' : 'Afficher' }} les filtres"
     >
       <i class="fa fa-filter" aria-hidden="true"></i>
     </button>
-    <!--TODO  Revoir pour rendre générique le bouton "Ajouter" , avant : {{ (child0.template["label_art_undef_new"] || "") }}-->
-    <!-- TODO Revoir pour les droits pour pouvoir créer un groupe de site avec anciennement:  *ngIf=obj.moduleCode && (currentUser['cruved_object'][child0.objectType] || currentUser['cruved']).C >= child0.cruved('C')" -->
-    <button class="btn btn-success float-right" (click)="navigateToAddChildren(null, null)">
-      <i class="fa fa-plus" aria-hidden="true"></i> {{ objectType.addObjLabel }}
-    </button>
+    <div class="wrapper-button">
+      <ng-content select="[add-button]"></ng-content>
+    </div>
+    <div class="default-button">
+      <!--TODO  Revoir pour rendre générique le bouton "Ajouter" , avant : {{ (child0.template["label_art_undef_new"] || "") }}-->
+      <!-- TODO Revoir pour les droits pour pouvoir créer un groupe de site avec anciennement:  *ngIf=obj.moduleCode && (currentUser['cruved_object'][child0.objectType] || currentUser['cruved']).C >= child0.cruved('C')" -->
+      <button class="btn btn-success float-right" (click)="navigateToAddChildren(null, null)">
+        <i class="fa fa-plus" aria-hidden="true"></i> {{ objectType.addObjLabel }}
+      </button>
+    </div>
   </div>
   <ngx-datatable
     *ngIf="rows"

frontend/app/components/monitoring-form-g/monitoring-form.component-g.ts

@@ -33,6 +33,7 @@ export class MonitoringFormComponentG implements OnInit {
   @Output() bEditChange = new EventEmitter<boolean>();
 
   @Input() sites: {};
+  @Input() apiService: ApiGeomService;
   dataForm: IDataForm;
   searchSite = '';
 
@@ -58,7 +59,6 @@ export class MonitoringFormComponentG implements OnInit {
     private _commonService: CommonService,
     private _dynformService: DynamicFormService,
     private _formService: FormService,
-    private _apiGeomService: ApiGeomService,
     private _router: Router,
   ) {}
 
@@ -69,7 +69,7 @@ export class MonitoringFormComponentG implements OnInit {
         tap((data) => {
           this.obj = data;
           this.obj.bIsInitialized = true;
-          this._apiGeomService.init(this.obj.endPoint, this.obj.objSelected);
+          this.apiService.init(this.obj.endPoint, this.obj.objSelected);
         }),
         mergeMap((data: any) => this._configService.init(data.moduleCode))
       )
@@ -272,7 +272,7 @@ export class MonitoringFormComponentG implements OnInit {
     const urlPathDetail = [this.obj.urlRelative].concat(urlSegment).join('/');
     this.objChanged.emit(this.obj);
     this.bEditChange.emit(false);
-    this._router.navigateByUrl(urlPathDetail);
+    this.obj.urlRelative ? this._router.navigateByUrl(urlPathDetail): null;
   }
 
   /**
@@ -296,8 +296,8 @@ export class MonitoringFormComponentG implements OnInit {
     const objToUpdateOrCreate = this._formService.postData(sendValue, this.obj);
     console.log(objToUpdateOrCreate);
     const action = this.obj.id
-      ? this._apiGeomService.patch(this.obj.id, objToUpdateOrCreate)
-      : this._apiGeomService.create(objToUpdateOrCreate);
+      ? this.apiService.patch(this.obj.id, objToUpdateOrCreate)
+      : this.apiService.create(objToUpdateOrCreate);
     const actionLabel = this.obj.id ? 'Modification' : 'Création';
     action.subscribe((objData) => {
       this._commonService.regularToaster('success', this.msgToaster(actionLabel));
@@ -344,7 +344,7 @@ export class MonitoringFormComponentG implements OnInit {
     this.bDeleteSpinner = true;
     this._commonService.regularToaster('info', this.msgToaster('Suppression'));
     // : this.obj.post(this.objForm.value);
-    this._apiGeomService.delete(this.obj.id).subscribe((del) => {
+    this.apiService.delete(this.obj.id).subscribe((del) => {
       this.bDeleteSpinner = this.bDeleteModal = false;
       this.objChanged.emit(this.obj);
       setTimeout(() => {

frontend/app/components/monitoring-properties-g/monitoring-properties-g.component.ts

@@ -24,7 +24,7 @@ export class MonitoringPropertiesGComponent implements OnInit {
   color: string = 'white';
   dataDetails: ISitesGroup;
   fields: JsonData;
-  fieldDefinitions: JsonData;
+  fieldDefinitions: JsonData = {};
   fieldsNames: string[];
   endPoint: string;
   datasetForm = new FormControl();

frontend/app/components/monitoring-sites-create/monitoring-sites-create.component.html

@@ -7,6 +7,7 @@
 ></btn-select>
 <div>
   <pnx-monitoring-form-g
+    [apiService]="siteService"
     [objForm]="form"
     #subscritionObjConfig
   ></pnx-monitoring-form-g>

frontend/app/components/monitoring-sites-create/monitoring-sites-create.component.ts

@@ -34,7 +34,7 @@ export class MonitoringSitesCreateComponent implements OnInit {
   constructor(
     private _formService: FormService,
     private _formBuilder: FormBuilder,
-    private siteService: SitesService,
+    public siteService: SitesService,
     private route: ActivatedRoute,
     private _objService: ObjectService
   ) {}

frontend/app/components/monitoring-sites/monitoring-sites.component.html

@@ -6,6 +6,7 @@
 >
 </pnx-monitoring-properties-g>
 <pnx-monitoring-form-g
+  [apiService]="_sitesGroupService"
   [obj]="sitesGroup"
   (objChanged)="onObjChanged($event)"
   *ngIf="bEdit"

frontend/app/components/monitoring-sites/monitoring-sites.component.ts

@@ -37,7 +37,7 @@ export class MonitoringSitesComponent extends MonitoringGeomComponent implements
   objParent: any;
 
   constructor(
-    private _sitesGroupService: SitesGroupService,
+    public _sitesGroupService: SitesGroupService,
     private _siteService: SitesService,
     private _objService: ObjectService,
     private router: Router,

frontend/app/components/monitoring-sitesgroups-create/monitoring-sitesgroups-create.component.html

@@ -1,4 +1,4 @@
 <!-- IF prefered observable compare to ngOnChanges we can remove rows and colsname-->
 <div>
-  <pnx-monitoring-form-g [objForm]="form"></pnx-monitoring-form-g>
+  <pnx-monitoring-form-g [apiService]="sitesGroupService" [objForm]="form"></pnx-monitoring-form-g>
 </div>

frontend/app/components/monitoring-sitesgroups-create/monitoring-sitesgroups-create.component.ts

@@ -5,6 +5,7 @@ import { ActivatedRoute } from '@angular/router';
 import { endPoints } from '../../enum/endpoints';
 import { ISitesGroup } from '../../interfaces/geom';
 import { FormService } from '../../services/form.service';
+import { SitesGroupService } from '../../services/api-geom.service';
 
 @Component({
   selector: 'monitoring-sitesgroups-create',
@@ -18,7 +19,7 @@ export class MonitoringSitesGroupsCreateComponent implements OnInit {
   constructor(
     private _formService: FormService,
     private _formBuilder: FormBuilder,
-    private route: ActivatedRoute
+    public sitesGroupService: SitesGroupService
   ) {}
 
   ngOnInit() {

frontend/app/components/monitoring-visits/monitoring-visits.component.html

@@ -3,6 +3,8 @@
   <pnx-monitoring-properties-g [newParentType]="objParent" [objectType]="objectType" *ngIf="!bEdit" [(bEdit)]="bEdit" [selectedObj]="site">
   </pnx-monitoring-properties-g>
   <pnx-monitoring-datatable-g [rows]="visits" [colsname]="colsname" (onSort)="setSort($event)" [page]="page"
-    (onFilter)="setFilter($event)" (onSetPage)="setPage($event)" [obj]="visits"
-    (onDetailsRow)="seeDetails($event)"></pnx-monitoring-datatable-g>
+    (onFilter)="setFilter($event)" (onSetPage)="setPage($event)" [obj]="visits" (onDetailsRow)="seeDetails($event)">
+    <option-list-btn add-button placeholder="Sélectionner le protocole" label="Ajouter une visite" [optionList]="modules"
+      (onDeployed)="getModules()" (onSaved)="addNewVisit($event)"></option-list-btn>
+  </pnx-monitoring-datatable-g>
 </div>