From 243592b4576c1b6367ce62ac7dd14cfb7dbbb5b6 Mon Sep 17 00:00:00 2001 From: siddheshraze <81591724+siddheshraze@users.noreply.github.com> Date: Mon, 4 Nov 2024 07:37:44 -0500 Subject: [PATCH 1/9] Production Upgrade: Multiline Form Input, CSV Download, Post-Validation Statistics (#181) * receiving the following error: TypeError: Expected signal to be an instanceof AbortSignal. Attempting to debug this error * after research, it seems that a now-default setting to next's serverMinification experimental setting may be causing the error. manually setting this var to false to see if the error continues * missing column headers and data from viewuploadedfiles table * saving progress. email-based authentication is still incomplete * full-scale changes. Added the following changes: 1. global indicator file implemented 2. middleware file updated to handle system redirection and post-authentication flow 3. all contexts updated to correctly utilize the enhanced dispatch setup 4. hashing system implemented to reduce data load when first opening app. If data found in IDB matches database, database call will not occur. 5. within sidebar -- session resume dialog implemented. system will check IDB to see if previously selected plot and census exist. if yes, user will be prompted via dialog to resume session. contexts will be updated once session is resumed or user opts to reset session. 6. redefined system load functions' connection to react dispatches completed 7. after seeing error with usage of nextUrl.searchParams, all API routes reworked to move searchParams calls outside of try-catch. 8. crypto-js package added and implemented in hash generation in place of node:crypto, which doesn't seem to work quite right with NextJS v14 9. NextJS updated to v14.1.3 and system implemented updated to correctly work with new updates 10. deprecated coremeasurements page removed. 11. updateContextsFromIDB reworked to correctly work with 1) enhanced dispatches and 2) IDB storage. 12. upload system reworked to reduce number of user-required button presses. system will now trigger 5-second countdown timers before continuing to the next stage. 13. viewuploadedfiles --> table display reworked to work correctly with new implementation of UploadedFile. 14. upload system --> acceptedFiles state variable re-centralized to uploadparent. upload system will now automatically parse inbound files as part of Dropzone logic instead of requiring manual button press to parse. * full-scale changes and code reformatting. * forgot to update UploadFireAzure component reference in uploadparent.tsx * removing throw new error statement here * saving changes -- datagridcommons testing is ongoing. Integration of quadratpersonnel junction table into quadrats datagrid instance is still in progress, but broad-scale reformatting and changes made across the board. * QuadratPersonnel junction table has been successfully integrated into quadrats data grid! separate set of API endpoints has been created to handle this -- still need to go through and remove any unnecessary code that hasn't yet been pulled, but otherwise functionality is sound * confirmation dialogs added to generic datagrid functions before saving or deleting, and confirmation dialog added to personnelautocompletemultiselect component to ensure that user has to confirm before adding or deleting personnel * Adding function to clarify dialog text when personnel are being added/removed * package versions updated. Had to do some reworking to make sure that code is compliant with updated versions * promise.all seems to be breaking things. Removing and trying again. * removing hash access. seems to be breaking (throwing 500) * forgot to add azure catalog schema to github environmental variable. adding jump loop to make sure that user is redirected to dashboard if trying to access login page while already logged in. * hashing was breaking because catalog environmental schema reference was not added to github * never mind hashing still doesn't work * adding secret to new env being generated in live site * fixing breakage in live site * the changes to the yml file were not actually included because I was in the frontend/ dir, not ForestGEO/ * params null check added * I think I've worked the problem with the census datagrid * continuing tinkering with the census grid columns -- changing system to just show current date instead of null or invalid date * reverting this change super fast -- restoring null formatting * removing excess columns n stuff from utilized grid columns, accidentally disabled pool monitor export so restoring that super quick * formatting. adding null startdate formatting b/c mpala census selection bugs out the app (census 1 has a null start date) * forgot extra condition in login system. adding some extra filtering to ensure that attributes, which delete via string code values, are correctly accounted for when interacting with the datagrid * saving changes. Opening PR here shortly. * addressing feedback from testers. Adding broad-spec formatting to remove unused imports, etc. * debugging validation scripts -- seeing date conversions not being processed properly from parsing to upload, so adding additional user information panels and custom handling to account for that. * 1. removing date parsing from processcensus.tsx -- parsing has been moved to file upload stage 2. Measurements Summary Grid Columns' measurementDate column updated to type string. Moment-based date check added and conversion to date and then to datestring added. 3. date-fns typescript package added 4. upload parent compartmentalized into new separate modal component. All datagrid views updated to use this in place of existing manual modal creation 5. uploadparentmodal enhanced to restrict users from exiting modal by any means other than pressing the X button. 6. uploadstart stage updated to simplify and reduce complexity. components moved to center and back/finalize button adjusted to be more visually appealing. * 1. restructuring upload system -- added segments subfolder to highlight active parts of the upload system and separate them from the administrative parent component uploadparent.tsx 2. added modal system to compartmentalize upload parent system into dedicated state-managed variable system * beginning Jest test unit test process -- some implementations complete * partial test cases added for generic datagrid and macros * reorganizing file structure to increase visibility and adding API route and datagridhelpers.ts handling to show validation history (future-proofing, not currently in use) * saving changes. Will resume dev shortly * updated .gitignore pulled in from main * manually implemented validation routes replaced with dynamic routes with slug type * deprecated API routes removed * running cleanup command on codebase -- removing unused imports, reorganizing formatting, etc * adding tests and formatting * Bug fixes: 1. datagridcommons (and validation procedures) -- multiple rows with validation errors were not being highlighted (only one at a time). After investigating, it was determined that the bug was in the structuring of the validation procedures, which was overwriting or removing past validation errors for a given coremeasurementID. It seems to be working now. 2. sidebar -- sidebar-independent scrolling implemented to ensure that main page stretching does not occur. 3. login -- automated slideshow of background images was only running once. interval system and reset implemented to ensure that it runs continuously when the login page is open. 4. measurementssummary -- API route updated to remove duplicate validation error querying, which was also causing issues with the row error display. Output was also not being parsed correctly, leading to problems with pagination, now corrected. 5. styleddatagrid -- border outlining was not rendering properly, corrected New features/functionality: 1. stemtreedetails -- page created in measurementshub, custom view and route implemented, and configuration added to generic datagrid. helper functions added as needed, and RDS macros implemented. 2. dynamic routing system implemented for validation procedures -- all manual routes have been collapsed into one route 3. default limits -- species table updated to include default max/min DBH & HOM values that are now queried in place of using hardcoded values. 4. subspecies -- page created in properties hub. Currently unpopulated, but CRUD system will need testing to ensure correct function. * reformatting to ensure that existing datagrids work while adding restructuring for stemTreeDetails * 1. removing default dialog prompt 2. removing azure storage container text 3. removing endless login page slide loop (set to 30s) 4. removing endless icon loop (set to 30s) * printing system fixed. css bug in rainbow icon resolved. context references added directly to datagridcommons instead of passing it in via parameter. * next version * beginning process of integrating new subquadrats table into system. as part of this, refactoring all config macros into dedicated files and functions to reduce complexity of each file. * removing data files * I thought I'd removed sample data and stuff already but it's been added back now for some reason. removing it again * Project-wide changes: 1. Documentation application Writerside added and populated with table definitions to use as reference point for RDS type updates later on. 2. Extraneous /fetchall API endpoints removed to reduce clutter 3. File-specific API endpoints centralized into new /filehandlers folder 4. CoreMeasurements API endpoint moved into /fixeddata folder. References updated. 5. Plot and Census contexts were previously being drilled via props into upload segments. Drilling replaced with direct calls to the context instead. 6. Formatting applied across frontend/ directory. 7. RDSDefinitions folder fully populated with respective RDS type definitions and GridColDef datagrid column fields. 8. NPM module critical warning fixed with npm audit fix --force. * applying eslint changes across project. Replaced fetchall endpoints with dynamic routing. Added subquadrat processing to fixeddata API set and finished integrating subquadrat retrieval into layout and context files. * removing package-lock.json and forcing update to latest versions universally * fixing validation bug, adding subquadrat ID row to measurements, fixing measurements upload system. Beginning integration of subquadrats into quadrats page instead of having a dedicated subquadrats page (will reduce confusion, since subquadrats are not technically needed for SI server upload) * full-scope changes. Beginning dynamic API routing system construction. Saving changes before removing existing/outdated APIs and moving their function to the dynamic system. New system will only require 1 API route, which will vastly improve project readability. * slug api routing system seems to be somewhat stable now. It's been modified to be compatible with all data types so far. Plot creation interface has been added as well and tested to confirm that creation works properly. * Large-scale changes: 1. Created and implemented vitest unit tests. Vitest was selected on account of its higher efficiency than Jest, and compatibility with NextJS's newer versions (current version is set to 14.2). However, end-to-end testing still needs to be implemented. Testing showed Cypress is not compatible with NextJS v12+ yet, and Playwright proved to be a little too complicated to execute within the scope of the current development window. This will need to be phased in down the road, I think. 2. Upload system was updated to integrate with the updated version schema. Systemwide changes were required due to the degree of differentiation between the prior schema and the current version, focusing particularly on the processcensus.tsx file and its supporting functions. 3. Sidebar received large function shift -- was lifted to allow users to create, edit, or remove plots. Additional confirmation dialog added for deletion system. Form setup used for plot addition or editing was enhanced and customized with individual validations -- coordinate and dimension values are now required to be > 0, and area measurements are now automatically calculated, with the field being disabled to user input accordingly. Sidebar was also lifted to enhance the function of the census selection modal. Additional dialogs and button functions incorporated to allow users to "open", "reopen", or "close" a census directly from the sidebar. Census closure dialog was enhanced to require users to submit a date after the listed start date in order to successfully close the census. Census deletion was added, but summarily deleted, with an alert to inform the user that census deletion is not available added. 4. IDB integration was updated to resolve issues with implementation of manual reset. System will now dynamically upgrade DB version to automatically clear stored browser cookies. Please note that this does not affect function in any way and is a pseudo-upgrade solely to clear the browser's cache. 5. Datagrid was repaired following testing that showed that modification of the attributes datagrid was not working correctly, and integration of the dynamic fixeddata routing handler was not correctly performed. Deep-scope changes added to facilitate this. 6. Measurements Summary View page was strongly enhanced to add features to 1) highlight rows that are pending validation but have not failed any validations, 2) enhance the function of the measurements upload systemn as follows: previously, it was possible to directly upload measurements files without uploading requisite fixed data (quadrats, personnel, species, etc). This has been removed and replaced with a detailed dialog that polls each of these fixed data tables and returns a detailed message to the user using progress bars and tooltips to inform them of which tables are missing population. 7. Site navigation system was reworked to incorproate new icons and adapted to the new website structure. Stemtreedetails view page was removed and replaced with the other new views. * rework continued. Saving changes before attempting integration of a collapsible sidebar element. * attempting full-scope breakdown of sidebar file into interlocking subcomponents -- hoping to reduce this file size since it's crossed 1500 lines and also retrieve functionality I can reuse elsewhere. * never mind, lift has been scheduled for future release due to LOE time constraints * okay, large-scale changes seem to be working. quadratpersonnel datagrid has been separated into its own datagrid. Data validity checks have been automated and incorporated. Queries have been updated and repaired as needed. Insertion/deletion test has been done for quadrats, personnel, quadratpersonnel. Need to get the others done. * removing progress dialog's independent API calls and replacing with connection to existing Context data validity * 1. upload system updated integrated into new schema 2. view system integrated into query system 3. data validity checks confirmed and usage expanded throughout application 4. datagrid systems expanded into independent functions 5. quadratpersonnel datagrid completed 6. subquadrats datagrid completed 7. upload system enhanced to perform NULL insertion when insertion value exceeds table column limit 8. upload system enhanced to perform species division into genus and species when a two-word species is inserted and no genus is inserted. upload system-customized datagrid has been updated to provide clear instruction on this. 9. container client bug corrected -- turns out it doesn't like capitalized letters 10. alltaxonomiesview connected in place of species and upload reworked 11. views moved to fixeddatainput folder 12. basic joyride (incomplete) implemented and integrated into layout, sidebar, and contexts 13. validationerrordisplay bug corrected 14. PENDING -- need to implement species limit datagrid view * Documentation changes -- other changes still pending, but removed deprecated documentation and integrated complete instructions on handling migration from ctfsweb flat file to new forestgeo schema. * core changes: 1. Adding more test cases and reattempting foundation API test structure. 2. Removing core data provider from app structure -- no longer useful due to size constraints 3. Reworking app's server-side, client-side, and dynamic structure. Improved speed of website somewhat now that load is not fully concentrated on client-side 4. Removed references to dayjs and replaced all with moment -- ended up being a little more useful 5. Wrapped renderSwitch() elements in Box -- found that text was being cut off at bottom edge and this seemed to fix it. 6. CORE CHANGE: Census structuring has been updated to new type OrgCensusRDS (organized). Census approach and treatment has fundamentally change, requiring app-wide reworking. Restructure based on the following determinations: -- Censes are organized by PlotCensusNumber, not CensusID -- censusID seems to be used purely to associate a date range to a given PlotCensusNumber -- rows in the census table should not be directly updated or overwritten in the event that new information or corrected information needs to be inserted. In these cases, a new census row should be created based on the date range of the new changes, and the new changes should be incorporated there. -- once closed, censuses (by censusID) should not be reopened (i.e., row w/ censusID = 1 has start date and end date defined ==> closed. censusID 1 cannot then have its end date set to NULL to indicate reopening.) -- API calls and all pertinent references have been reworked to search along plotCensusNumber. HOWEVER, validation procedures and other quality control systems still use censusID, as each measurement is still going to be associated with a specific date range (censusID) -- SQL-side tables' implementation of census has NOT CHANGED. Census reorganization changes are intended purely to make understanding data easier. 7. Skeletons folder deleted (unused) 8. App has been refactored to better handle subquadrats -- it seems that plots can use EITHER subquadrats or just quadrats (from a stored data perspective). With this in mind, the system has been reworked to allow users to add subquadrats information in the event that it is needed. On the admin-side, a toggle switch has been incorporated to achieve this. 9. After issues with IDB synchronization and database connectivity issues, IDB usage has been temporarily disabled. As such, the app now leans purely on SQL interactions to perform operations. 10. App has been reworked to use UNDEFINED client-side and NULL server-side. Due to issues with how typescript interprets undefined and null, it became less complicated to rework the system to use udefined. Mapping systems have been updated to utilize this. 11. QuadratPersonnel datagrid has been created. Attempted use of Autocomplete components has been deprecated and replaced with a simple dropdown selection system. The autocomplete API system was becoming increasingly convoluted and would have required building in a cache to save recently selected/updated items, which is out-of-scope currently. 12. SQL views have been integrated into app datagrid handling/processing. In line with this, all views have been updated server-side to attach all joined tables' ID columns. A standardized system has been implemented to review changes made to the view and submit UPDATE statements to the corresponding table, thus allowing for an interactive SQL view. 13. CORE CHANGE: Sidebar has been simplified drastically to remove all user selection modals and instead directly show Select components. Function has been confirmed. 14. Sidebar has been updated to add census interaction buttons at bottom of sidebar in place of a census datagrid. This allows census interaction restrictions to be preserved while also allowing censuses to be reopened/closed/new census started. 15. prevalidation system -- datavalidityprovider.tsx has been added in order to poll core tables (fixed data tables) that are required in order for new census.csv format forms to be submitted. Tooltips and alert badges have also been incorporated. QuadratPersonnel has also been restricted and will not be unlocked until at least 1 quadrat and 1 person has been added to each respective table. 16. Date convention has been standardized to UTC across application. User warnings have been added accordingly. 17. Census open/close/reopen modals have been updated to show most recent open/close/reopen date as a reference point to ensure that the correct date is attached. * minor edit to ensure utc applied * need to build out and update documentation as contract end is approaching. Will attempt to host connected Azure site for documentation alongside existing live site to better facilitate onboarding. * saving changes -- might be removing this shortly. * test * resetting -- seems like this isn't going to work * removing storybook installation -- needs to be a future development task rather than something to be done immediately * adding missing .pem file * resolving merge issues * partial save -- should not be merged * documentation changes * workspace docs file * resolving package.json error * basic postvalidation route setup and added * partial changes made, saving. * saving changes -- need to resolve mysql connection state being set to closed, which is automatically kicking any incoming connections on live site * Adding additional emergency settings to ensure that connection is not automatically set to closed after mysql server is exited. Further incorporating graceful shutdown mechanism to ensure that in the rare case that leftover connections remain, they are properly closed. * Add or update the Azure App Service build and deployment workflow config * Remove the Azure App Service build and deployment workflow config * attempting to add development livesite job to test without needing to commit to main * build failure -- testing * wildcards and next cache steps added. Wildcards will ensure that all future versions will automatically build to the development website (must follow established convention) * received failed login error -- added dedicated login step * login step seems to be causing new issues -- commenting out and attempting build again * accidentally deleted main build step, restoring and commenting out current nonfunctional steps for later * accidentally using wrong publish profile reference * not sure why, but next-auth login is not being permitted on development site for some reason * trying more stuff to see if I can get the development-based authentication to work * need to fix base nextauth url * accidentally made the change backwards * temp -- saving changes. tried updating packages and that broke everything so had to rewind it all. TESTING.md file committed to explain user testing process for future use. * minor formatting changes, adding enhanced shortcut implementation to datagrids, fully fleshing out reentry data modal system * adding framework for icon customization as versions change * renaming branch * acacia version signature incorporated. Shell folder system and icon system incorporated. * icon updated to forestgeo-app central * saving changes. Mostly styling updates and reworkings to reduce code complexity. * Adding changelog and version explanation * first-round error export function created. Pending testing -- will update as output is tested * testing potential resolution to vitest get-stream issue * extensive core schema changes. Triggers and changelog system implemented across the board to address need for non-duplicative update system -- users will be iteratively updating rows as they complete a census. B/c of this, makign rows immutable will cause too much duplication. In place of this, the changelog system has been implemented. * incorporating SQL structure. Minor changes made to other authentication API and datagrids updated to correctly integrate with schema changes. Gitignore files updated accordingly. * PlotRDS type was updated to remove usesSubquadrats, but summary page was not updated to correctly work with this change, now repaired. * decided to re-add the usesSubquadrats property to the PlotRDS type to avoid cascading issues. Value has been defaulted to false. Additionally updating API endpoints for Personnel table to use RoleID instead of Role * migration script has been updated to target the sinharaja data source (no exposed data). The connection closed state issue is persisting, despite debugging attempt. To resolve this: 1. a manual inactivity timer has been implemented to automatically close the pool after 1 hour of inactivity. 2. A new user, `deployment`, has been created to act as a dedicated user for the live site. This will ensure that my connections to the database do not cause issues for any active site users. 3. For added redundancy, the connectTimeout parameter has been added to the sqlConfig object. * accidentally forgot to incorporate methods to handle pool reinitialization and update, resolving * changelog: 1. added resetschema.sql function to reset testing schema 2. formatting changes and consolidating some additional data 3. datagrid column arrays are in process of being moved into separate .tsx file so that Typography formatting can be applied to headers when needed (if column names are too large, etc) 4. Updated personnel fixeddata and cmprevalidation API endpoints to correctly work with updated PersonnelRDS type/table 5. Playwright initialization added and setup. No tests currently added, but future proofing for use. 6. RolloverModal added to handle rollover process when creating a new census. Allows users to selectively choose personnel OR quadrats to rollover from one OR more past censuses (where data is present). 7. Instead of discarding testing component file, was instead converted into the RolloverStemsModal. Its function has not been enabled but has been integrated. Allows users to additionally rollover stems data if needed ONLY if quadrats data is already being rolled over. Still needs to be fully tested and repaired (assuming bugs are present) 8. area selection options macro added now that dedicated selection array for area measurements added SQL-side. * 1. added resetschema.sql function to reset testing schema 2. formatting changes and consolidating some additional data 3. datagrid column arrays are in process of being moved into separate .tsx file so that Typography formatting can be applied to headers when needed (if column names are too large, etc) 4. Updated personnel fixeddata and cmprevalidation API endpoints to correctly work with updated PersonnelRDS type/table 5. Playwright initialization added and setup. No tests currently added, but future proofing for use. 6. RolloverModal added to handle rollover process when creating a new census. Allows users to selectively choose personnel OR quadrats to rollover from one OR more past censuses (where data is present). 7. Instead of discarding testing component file, was instead converted into the RolloverStemsModal. Its function has not been enabled but has been integrated. Allows users to additionally rollover stems data if needed ONLY if quadrats data is already being rolled over. Still needs to be fully tested and repaired (assuming bugs are present) 8. area selection options macro added now that dedicated selection array for area measurements added SQL-side. * skipping rollovermodal unit tests -- they're not up-to-date and don't properly test the modal. * - refactoring changes and centralizing features. - Github feedback form 1st iteration has been incorporated, but disabled pending approval of Github PAT. - Creating first-iteration ViewFullTable grid. - Removing census-lock from upload functionality. Now that the census function has been deprecated it doesn't need to be active - renamed and updated github secrets in next.config.js file * Updating modules and fixing a small emotioncache error. * saving changes -- pushing to dev * re-enabling feedback form * github feedback modal is throwing environmental errors, but only when building to dev site. Assessing. * Found the issue -- needed to update workflow to incorporate new env variables needed to operate git API call. * vitest bug found, determined to be thrown by potentially outdated version of execa. Manually adding latest version as dev dependency. * Adding default error pages to all routes. Will later customize to each page. No other configuration needed, NextJS will automatically wrap all adjacent page files with the error.tsx file * minor additions made here to clarify which error page is which * 1. view system has been updated to use materialized views instead of existing view queries (view runtime delay exceeded acceptable limits) 2. batch processing flag system implemented and removed, replaced with app-side monitoring instead of trigger-based monitoring. After the user performs updates to dependent tables, the app will place calls to manually refresh materialized view tables behind the scenes 2a. Currently, only the API is implemented. The implementation of a background load is pending. 3. All pages in application were given default error.tsx files. NextJS automatically wraps all page.tsx files with the (same-level) error.tsx file, ensuring that error handling is managed by NextJS instead of app 4. Progress Dialog system removed from the View Data page. This is already being handled by the prevalidation system. 5. fixeddata API route adapted to include calls to the materialized view tables 6. refreshviews API route created. Accepts dynamic routing segments to specify view and schema to target 7. Datagrid columns updated: 7a. View definitions across all schemas updated. Please see sqlscripting/ files for specific updates 7b. View Data datagrid columns updated to add customized handling for DBH and HOM values. These columns, instead of sitting adjacent to their respective unit columns, have been added together. 7c. id columns added where needed -- even though they're not necessarily needed, the datagrids will not work properly if they're not provided. 7d. AllTaxonomiesView grid has been updated to remove reference table column joins. This table doesn't seem to be historically used, and can be added later if needed. 8. Partial culprit for extended VIew Data page load times was also repeated calls to fetchPaginatedData at the same time. Component was debugged to ensure that it's only called once, and debounce function was also added. 9. Full-scale RDS type updates: 9a. Instead of manual implementation of each Result type, requiring two points of update whenever schema changes are made, generic utility functions implemented to automate implementation of Result types based on RDS type definitions 9b. Manual implementation of mapping functions replaced with single generic mapper system that automates mapping to and from RDS type. 10. PoolMonitor class updated to use `chalk` module to provide more comprehensive server-side logging using colors. 11. middleware.ts file updated to use new properties link name 12. acacia version name and debug setting added to build command * fixing links in middleware. Attempted to upgrade next-auth to v5 but reset after cascading errors. Adding no-lint setting to next build b/c of missing support for eslint v9 (current version) and various missing type dependencies. * minor updates through npm-check-updates * partial commit - saving changes to ensure that resetting will not remove too much progress. * Prettier and ESLint successfully integrated into project. Package.json command updated to add function. * 1. Applying prettier changes. 2. Amending Tooltip function in Dashboard's Chip to disappear when the pulse animation is active. 3. Amending datagrid tooltips to disable interactivity with the text itself, ensuring that user can move from one button to the next without accidentally holding the tooltip open. * 1. minor enhancements to the measurements-oriented datagrid to display viewfulltable. 2. global formatting and adjustments * 1. minor enhancements to the measurements-oriented datagrid to display viewfulltable. 2. global formatting and adjustments * saving changes. Dashboard system reconfigured and validations page added. * Full-scale formatting. Baseline dashboard has been completed. Baseline validations view (template) has been added. summary statistics route (postvalidations) has been tentatively added, but needs refining * Accidentally deleted validation API routes, restoring. Formatting and adding datagrid for cmverrors to the validations page. * Saving changes. Continuing reworking of validation CRUD system and beginning application of feedback changes. * Removing row pending/failed validations grids from validations page. minor adjustments to the dashboard page. * missed a file for some reason * Accidentally messed up structuring of datamapper file (bugs introduced), resolved. Applied prettier to project. * partial save. resolving baseline changes for validation CRUD system and preparing for further development. * Breaking changes being saved. Partial completion of core updates to validation system and upload interaction system. Additional commits will be submitted to resolve build errors caused here. * Broken utils functions have been corrected. Personnel upload has been confirmed in accordance with new schema structure. * Continuing correction process to upload system to ensure it's working correctly. Working on adding a wrapper function to encase use cases for the DataGridCommons to make applying it to tables easier -- currently, implementations for the attributes, personnel, quadrats tables, etc, require an extensive amount of duplication to build out and thus make modifications and updates trickier. Continuing structural shift of validation system to be app-side and modular to better facilitate CRUD maintenance system. * Revamping personnel datagrid to display role information along with roleID. Reworking validations functions to optimize. Structural updates to ensure that fetchall works with the roles table. * incorporating documentation sources. Beginning refactor and integration of walkthrough system to better facilitate pilot onboarding * Additional documentation started. Will continue expanding the full set of starting instructions. * New documentation stack completed. Pending review and feedback * first-attempt documentation github pages deployment * corrections and adding test/deployment steps for safety * separated build-and-deploy into dedicated jobs. attempted fix for docs build job * bug fixes for test-docs step * still breaking. trying to add fixes * artifacts directory creation is failing. * don't have direct permissions to touch github dir, using github_workspace env var instead * removing test-docs job. Let's see if this works better or worse * forgot to remove references to test-docs after deleting it * deploy-docs is failing. Trying to implement a fix * formatting error * trying a new approach * saving changes. partial updates made to validations system and processormacros * system changes. attempting to add roles interaction interface to personnel datagrid to facilitate manual role CRUD interactions. * restructuring. Centralizing functionality and expanding validation CRUD system. Restructuring core validation and upload process to be more modular. * continuing restructure of upload and validation process * saving changes. Upload structure has been confirmed and dynamic validation system is almost finished, pending next iteration of testing * adding updated package information * continuing to try and restructure the specieslimits system, the validations system, and the measurementssummary highlighting system. * vitest was causing build crashes, trying repair * vitest issues are persisting. * login errors -- nextauth is unable to find secret for some reason?? * nextjs is still throwing a prerender pages error at all of my website endpoints. trying to see if there's something else going on. * for some reason, login is failing. log stream is reporting SIGNIN_OAUTH_ERROR with invalid_request, app identifier is expected to be a GUID * comment typo * readding nextauth url endpoint. app seems to keep redirecting to localhost instead of what the correct link should be * nextauth route had the wrong environmental variables?? * Running out of options -- system has resolved into an OAuthSignInError when trying to log in * trying something new for a change. Maybe this'll work?? * still not sure what's going on. * Typos are still occurring. Still trying to figure out why the oauthsignin error is persisting. * okay, i think I managed to get the local authentication working again (in dev server). gonna start trying to see if I can get the actual github actions workflow to do it too. * Development website seems to be properly resolved. Not sure what the core issue was there, but moving the development site to a second dedicated app service seemed to do the trick. There was some kind of intersection with the usage of the development slot in the production instance that was causing issues. YML files were divided into dedicated production and development workflows. * full-scope changes. debounce added to layout files and selection systems. datagridcommons debugged for deep bugs found during add/editing row process. Additional stabilization steps need to be taken, but website is partially functional and testing environment has been prepared * continuing the stabilization effort. Deeper explanation of changes will be outlined in PR * overhaul. datagrid system revamped into a single-point interaction system that centralizes core function. Interaction testing passed attributes and personnel datagrids * Stabilization is almost completed. Testing has been completed for all supporting grids except for species, roles, and quadratpersonnel, none of which are immediately vital. * isolated implementation has been resolved to address all supporting data fields' manual entry * resolving build errors. * continuing the stabilization process. data baseline testing is confirmed and baseline validations have been confirmed to work. Additional stress testing is still needed, but for the time being it should work for piloting. * adding plotID parameter to postvalidation. Need to still integrate it fully into a data view. * minor visual change to enforce census date measurement rendering on different lines and keeping the sidebar from occupying half the screen * saving changelog changes and siteconfigs updates * adding console logging statements to debug login failures * created access-denied page to act as an endpoint for unallowed permissions access. shifted the email verification query into the authentication function itself. migration scripts adjusted and updated to correctly work with database. JWT interface extended in place of the Token in the next-auth.d.ts * 1. cleaning tests -- tests need to be reimplemented in full to account for new table changes. 2. postvalidation API endpoint refined into dynamic routing collection. 3. dedicated page to show post-validation statistics added to measurements hub 4. postvalidationqueries table created in schema. schema structure updated to reflect this 5. layout.tsx file updated to increase debounce (race conditions seem to be occurring). useEffects loop were restructured to ensure that on login, sites will correctly load. 6. error fallback pages received minor edits/changes * quick update to change the package.json file * formatting * following feedback -- removing reentry data modal from use. Component has been retained for application to other places in future. * adding template staging and drafting tables. updating package-lock. adding rds/result mappers for staging/draft tables * creating draft interaction datagrid for msv_draft. formatting/cleanup applied * integrating draft msv into fixeddata and testing render via isolated datagridcommons implementations * simple API endpoint to take signed in user's information and retrieve unique catalog UserID value for use. Pausing development of staging system to see if I can leverage existing UserDefinedFields column instead of needing to add a full new table * handleUpsertForSlices updated to correctly apply data propagation from one slice queryConfigSlice to the next. vitest test suite implemented and mocking used to test and verify propagation. Temporary RDS implementations for staging data added. * fixing view implementations to remove additional formatting changes in-view. unit tests for alltaxonomiesview and stemtaxonomiesview completed and working successfully * temp -- saving changes * removing deprecated tests. slight mods to poolmonitor. utils test case * Merge timekeeping.ts from main-rollover-modal-hotfix into development-baobab * testing solution to uploaded file view error (process env var isn't being found despite being correctly laoded in) -- adding declaration of env var to next.config.js * adding repair to fix broken uploaded file view azure storage connection * enhancing datamapper and updating deprecated modules * saving changes. Need to integrate updates from main in * debugging. minor changes to remove or disable features that shouldn't be accessible to the user yet since they're in development. Slightly rework of the monaco editing system to ensure it works with updated NextJS version. * build error -- resolving * Broken utils functions have been corrected. Personnel upload has been confirmed in accordance with new schema structure. * feature upgrades. multiline form system implemented and set up to act as bulk update forms. API endpoint set up to connect the form system to connect to the existing SQLLoad app system, needs testing, though * Restructure and implementation of the multiline form interaction are continuing. Saving changes to correct core quadrat schema structure failure. * merge conflict missed a line * requisite repairs to the quadrats processing system to make sure that the multiline handler was correctly interacting with the system. correcting modal close functions to also refresh datagrid after closing. * scrubbing old datagrid instances that aren't used anymore * continuing postvalidation construction. saving changes here to test a new display approach so that I can revert if need be. * first-iteration postvalidation UI is completed. need to apply final tuning and then will push to dev site. * postvalidation query testing is complete. integrated into sidebar successfully -- places check to ensure that > 0 coremeasurements before allowing access. Download/Run system checked. * Multiline Data Entry & Post Validation System (#179) * Broken utils functions have been corrected. Personnel upload has been confirmed in accordance with new schema structure. * feature upgrades. multiline form system implemented and set up to act as bulk update forms. API endpoint set up to connect the form system to connect to the existing SQLLoad app system, needs testing, though * Restructure and implementation of the multiline form interaction are continuing. Saving changes to correct core quadrat schema structure failure. * requisite repairs to the quadrats processing system to make sure that the multiline handler was correctly interacting with the system. correcting modal close functions to also refresh datagrid after closing. * scrubbing old datagrid instances that aren't used anymore * continuing postvalidation construction. saving changes here to test a new display approach so that I can revert if need be. * first-iteration postvalidation UI is completed. need to apply final tuning and then will push to dev site. * postvalidation query testing is complete. integrated into sidebar successfully -- places check to ensure that > 0 coremeasurements before allowing access. Download/Run system checked. * minor fix -- forgot to add missing function call in view full table grid * Removing old width formatting from form column declarations * applying linter changes * missed some sections that needed to be updated (interacting with quadrats --> census). resolved some additional errors that were occurring at various interaction points and repaired some functionality issues * version upgrades. mui materialui modules have been deprecated so needed to raise version levels to make sure things work * grid version has been deprecated. raising versions to grid2 * fixing version name * continuing to attempt bug fixes * adding default handling to address missing headers and fill them in with null values to make sure that file processing works without issue. * updates: incorporating download as form csv format button to retrieve data formatted to match input form standards, instead of default JSON format * fixing issues with file upload and processing system. bugs were found causing load issues and SQL overload issues, but have been temporarily resolved. refit is needed to update how the file set is processed to ensure that system will not collapse under weight of file set. * commenting out unnecessary part of file upload process due to massive delays being imposed, leading to webpage crashing. Will be restored later --------- Co-authored-by: Siddhesh Ambokar --- .github/workflows/dev-forestgeo-livesite.yml | 34 +- .github/workflows/main-forestgeo-livesite.yml | 7 +- frontend/CHANGELOG.md | 69 +- .../__tests__/alltaxonomiesview_uqc.test.tsx | 104 + .../__tests__/stemtaxonomiesview_uqc.test.tsx | 70 + frontend/app/(hub)/dashboard/page.tsx | 3 + .../measurementshub/postvalidation/page.tsx | 292 +- .../(hub)/measurementshub/summary/page.tsx | 2 +- .../measurementshub/validations/page.tsx | 1 - .../app/api/auth/[[...nextauth]]/route.ts | 8 - .../bulkcrud/[dataType]/[[...slugs]]/route.ts | 54 + .../catalog/[firstName]/[lastName]/route.ts | 26 + .../[dataType]/[[...slugs]]/route.ts | 12 + .../app/api/fetchall/[[...slugs]]/route.ts | 2 +- .../[dataType]/[[...slugs]]/route.ts | 18 +- .../[dataType]/[[...slugs]]/route.ts | 130 + frontend/app/api/postvalidation/route.ts | 2 +- .../[plotID]/[censusID]/[queryID]/route.ts | 61 +- frontend/app/api/sqlload/route.ts | 1 - .../procedures/[validationType]/route.ts | 1 - .../components/client/datagridcolumns.tsx | 977 +- frontend/components/client/formcolumns.tsx | 734 + .../components/client/githubfeedbackmodal.tsx | 8 +- .../components/client/postvalidationrow.tsx | 218 + .../alltaxonomiesviewdatagrid.tsx | 356 - .../applications/attributesdatagrid.tsx | 107 - .../isolatedalltaxonomiesdatagrid.tsx | 29 +- .../isolated/isolatedattributesdatagrid.tsx | 27 +- .../isolated/isolatedmsvstagingdatagrid.tsx | 105 + .../isolated/isolatedpersonneldatagrid.tsx | 25 +- .../isolated/isolatedquadratsdatagrid.tsx | 30 +- .../isolatedstemtaxonomiesviewdatagrid.tsx | 4 +- ...ummaryviewdatagrid.tsx => msvdatagrid.tsx} | 66 +- .../multiline/multilineattributesdatagrid.tsx | 31 + .../multilinemeasurementsdatagrid.tsx | 58 + .../applications/multiline/multilinemodal.tsx | 111 + .../multiline/multilinepersonneldatagrid.tsx | 31 + .../multiline/multilinequadratsdatagrid.tsx | 37 + .../multiline/multilinespeciesdatagrid.tsx | 34 + .../applications/personneldatagrid.tsx | 203 - .../applications/quadratpersonneldatagrid.tsx | 173 - .../applications/quadratsdatagrid.tsx | 145 - .../datagrids/applications/rolesdatagrid.tsx | 101 - .../stemtaxonomiesviewdatagrid.tsx | 131 - .../applications/viewfulltabledatagrid.tsx | 21 +- .../datagrids/isolateddatagridcommons.tsx | 208 +- .../isolatedmultilinedatagridcommons.tsx | 328 + .../datagrids/measurementscommons.tsx | 95 +- .../components/datagrids/reentrydatamodal.tsx | 11 +- .../components/processors/processcensus.tsx | 86 +- .../processors/processorhelperfunctions.tsx | 143 +- .../components/processors/processormacros.tsx | 4 +- .../components/processors/processquadrats.tsx | 13 +- frontend/components/sidebar.tsx | 28 +- .../uploadsystem/segments/uploadfireazure.tsx | 5 +- .../uploadsystem/segments/uploadfiresql.tsx | 108 +- .../segments/uploadparsefiles.tsx | 9 +- .../segments/uploadreviewfiles.tsx | 17 +- .../components/uploadsystem/uploadparent.tsx | 11 +- .../uploadsystemhelpers/uploadparentmodal.tsx | 1 - frontend/config/datagridhelpers.ts | 44 +- frontend/config/datamapper.ts | 38 +- frontend/config/db.ts | 4 +- frontend/config/macros.ts | 15 - frontend/config/macros/azurestorage.ts | 6 +- frontend/config/macros/formdetails.ts | 6 +- frontend/config/macros/siteconfigs.ts | 9 +- frontend/config/poolmonitor.ts | 43 +- frontend/config/sqlrdsdefinitions/core.ts | 20 +- .../config/sqlrdsdefinitions/validations.ts | 13 + frontend/config/sqlrdsdefinitions/views.ts | 45 +- frontend/config/sqlrdsdefinitions/zones.ts | 1 + frontend/config/styleddatagrid.ts | 22 +- frontend/config/utils.ts | 26 +- frontend/documentation/.gitignore | 4 + frontend/documentation/Logging into App.md | 22 +- frontend/documentation/Selecting a Census.md | 6 +- frontend/documentation/Selecting a Site.md | 9 +- frontend/documentation/Using the Sidebar.md | 7 +- frontend/documentation/Welcome.md | 6 +- .../Database-Setup-and-CTFSWeb-Integration.md | 6 +- frontend/package-lock.json | 25986 ++++++++-------- frontend/package.json | 88 +- .../sqlscripting/migration_no_mapping.sql | 68 +- .../sqlscripting/postvalidation_queries.sql | 203 + frontend/sqlscripting/resetschema.sql | 28 + frontend/sqlscripting/storedprocedures.sql | 11 +- frontend/sqlscripting/tablestructures.sql | 15 +- frontend/styles/globalloadingindicator.tsx | 8 +- frontend/styles/versions/acaciaversion.tsx | 8 +- frontend/tailwind.config.js | 4 +- frontend/tsconfig.json | 31 +- frontend/vite.config.ts | 14 + 93 files changed, 16795 insertions(+), 15747 deletions(-) create mode 100644 frontend/__tests__/alltaxonomiesview_uqc.test.tsx create mode 100644 frontend/__tests__/stemtaxonomiesview_uqc.test.tsx create mode 100644 frontend/app/api/bulkcrud/[dataType]/[[...slugs]]/route.ts create mode 100644 frontend/app/api/catalog/[firstName]/[lastName]/route.ts create mode 100644 frontend/app/api/formdownload/[dataType]/[[...slugs]]/route.ts create mode 100644 frontend/components/client/formcolumns.tsx create mode 100644 frontend/components/client/postvalidationrow.tsx delete mode 100644 frontend/components/datagrids/applications/alltaxonomiesviewdatagrid.tsx delete mode 100644 frontend/components/datagrids/applications/attributesdatagrid.tsx create mode 100644 frontend/components/datagrids/applications/isolated/isolatedmsvstagingdatagrid.tsx rename frontend/components/datagrids/applications/{measurementssummaryviewdatagrid.tsx => msvdatagrid.tsx} (68%) create mode 100644 frontend/components/datagrids/applications/multiline/multilineattributesdatagrid.tsx create mode 100644 frontend/components/datagrids/applications/multiline/multilinemeasurementsdatagrid.tsx create mode 100644 frontend/components/datagrids/applications/multiline/multilinemodal.tsx create mode 100644 frontend/components/datagrids/applications/multiline/multilinepersonneldatagrid.tsx create mode 100644 frontend/components/datagrids/applications/multiline/multilinequadratsdatagrid.tsx create mode 100644 frontend/components/datagrids/applications/multiline/multilinespeciesdatagrid.tsx delete mode 100644 frontend/components/datagrids/applications/personneldatagrid.tsx delete mode 100644 frontend/components/datagrids/applications/quadratpersonneldatagrid.tsx delete mode 100644 frontend/components/datagrids/applications/quadratsdatagrid.tsx delete mode 100644 frontend/components/datagrids/applications/rolesdatagrid.tsx delete mode 100644 frontend/components/datagrids/applications/stemtaxonomiesviewdatagrid.tsx create mode 100644 frontend/components/datagrids/isolatedmultilinedatagridcommons.tsx create mode 100644 frontend/documentation/.gitignore create mode 100644 frontend/sqlscripting/postvalidation_queries.sql create mode 100644 frontend/sqlscripting/resetschema.sql create mode 100644 frontend/vite.config.ts diff --git a/.github/workflows/dev-forestgeo-livesite.yml b/.github/workflows/dev-forestgeo-livesite.yml index 3bfbc59b..a6b701b2 100644 --- a/.github/workflows/dev-forestgeo-livesite.yml +++ b/.github/workflows/dev-forestgeo-livesite.yml @@ -19,9 +19,9 @@ jobs: - uses: actions/checkout@v4 - name: Set up Node.js version - uses: actions/setup-node@v3 + uses: actions/setup-node@v4.0.4 with: - node-version: '18.x' + node-version: '20.x' - name: create env file (in frontend/ directory) -- development id: create-env-file-dev @@ -48,22 +48,22 @@ jobs: echo OWNER=${{ secrets.OWNER }} >> frontend/.env echo REPO=${{ secrets.REPO }} >> frontend/.env - - name: Cache node modules - uses: actions/cache@v2 - with: - path: frontend/node_modules - key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }} - restore-keys: | - ${{ runner.os }}-node- +# - name: Cache node modules +# uses: actions/cache@v2 +# with: +# path: frontend/node_modules +# key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }} +# restore-keys: | +# ${{ runner.os }}-node- - - name: Cache Next.js build - uses: actions/cache@v2 - with: - path: frontend/build/cache - key: ${{ runner.os }}-next-${{ hashFiles('**/package-lock.json') }}-${{ hashFiles('**/.next/cache') }} - restore-keys: | - ${{ runner.os }}-next- - ${{ runner.os }}-next-${{ hashFiles('**/package-lock.json') }} +# - name: Cache Next.js build +# uses: actions/cache@v2 +# with: +# path: frontend/build/cache +# key: ${{ runner.os }}-next-${{ hashFiles('**/package-lock.json') }}-${{ hashFiles('**/.next/cache') }} +# restore-keys: | +# ${{ runner.os }}-next- +# ${{ runner.os }}-next-${{ hashFiles('**/package-lock.json') }} - name: move into frontend --> npm install, build, and test run: | diff --git a/.github/workflows/main-forestgeo-livesite.yml b/.github/workflows/main-forestgeo-livesite.yml index 3da01339..86c5e5e2 100644 --- a/.github/workflows/main-forestgeo-livesite.yml +++ b/.github/workflows/main-forestgeo-livesite.yml @@ -13,7 +13,7 @@ jobs: build-app-production: if: github.ref == 'refs/heads/main' runs-on: ubuntu-latest - environment: development + environment: production steps: - uses: actions/checkout@v4 @@ -30,7 +30,6 @@ jobs: echo AZURE_AD_CLIENT_SECRET=${{ secrets.AZURE_AD_CLIENT_SECRET_PRODUCTION }} >> frontend/.env echo AZURE_AD_CLIENT_ID=${{ secrets.AZURE_AD_CLIENT_ID_PRODUCTION }} >> frontend/.env echo AZURE_AD_TENANT_ID=${{ secrets.AZURE_AD_TENANT_ID_PRODUCTION }} >> frontend/.env - echo NEXTAUTH_URL=${{ secrets.NEXTAUTH_URL_DEV }} >> frontend/.env echo NEXTAUTH_SECRET=${{ secrets.NEXTAUTH_SECRET }} >> frontend/.env echo AZURE_SQL_USER=${{ secrets.AZURE_SQL_USER }} >> frontend/.env echo AZURE_SQL_PASSWORD=${{ secrets.AZURE_SQL_PASSWORD }} >> frontend/.env @@ -86,7 +85,7 @@ jobs: deploy-app-production: needs: build-app-production runs-on: ubuntu-latest - environment: development + environment: production steps: - name: Download build artifact @@ -103,4 +102,4 @@ jobs: app-name: 'forestgeo-livesite' slot-name: 'Production' publish-profile: ${{ secrets.AZUREAPPSERVICE_PUBLISHPROFILE_PRODUCTION }} - package: frontend/build/standalone + package: frontend/build/standalone \ No newline at end of file diff --git a/frontend/CHANGELOG.md b/frontend/CHANGELOG.md index 63f4382e..c04d7c05 100644 --- a/frontend/CHANGELOG.md +++ b/frontend/CHANGELOG.md @@ -118,11 +118,11 @@ 1. FixedData cases' queries updated to correctly work with updated schemas 2. New tables/cases added: - 1. `personnelrole` - 2. `sitespecificvalidations` - 3. `roles` - 4. `measurementssummary` - 5. `viewfulltable` + 1. `personnelrole` + 2. `sitespecificvalidations` + 3. `roles` + 4. `measurementssummary` + 5. `viewfulltable` ###### POST @@ -142,18 +142,18 @@ 1. Postvalidation summary statistics calculation endpoint 2. Statistics queries: - 1. `number of records by quadrat` - 2. `all stem records by quadrat (count only)` - 3. `live stem records by quadrat (count only)` - 4. `tree records by quadrat (count only)` - 5. `number of dead or missing stems by census` - 6. `trees outside of plot limits` - 7. `stems with largest DBH/HOM measurements by species` - 8. `all trees that were recorded in last census that are NOT in current census` - 9. `number of new stems per quadrat per census` - 10. `quadrats with most and least new stems per census` - 11. `number of dead stems per quadrat per census` - 12. `number of dead stems per species per census` + 1. `number of records by quadrat` + 2. `all stem records by quadrat (count only)` + 3. `live stem records by quadrat (count only)` + 4. `tree records by quadrat (count only)` + 5. `number of dead or missing stems by census` + 6. `trees outside of plot limits` + 7. `stems with largest DBH/HOM measurements by species` + 8. `all trees that were recorded in last census that are NOT in current census` + 9. `number of new stems per quadrat per census` + 10. `quadrats with most and least new stems per census` + 11. `number of dead stems per quadrat per census` + 12. `number of dead stems per species per census` #### frontend/app/api/refreshviews/[view]/[schema]/route.ts @@ -220,7 +220,7 @@ 3. customized cell and edit cell rendering added 4. some exceptions exist -- for instances where specific additional handling is needed, column states are directly defined in the datagrid components themselves. - 1. `alltaxonomiesview` -- specieslimits column customized addition + 1. `alltaxonomiesview` -- specieslimits column customized addition #### GitHub Feedback Modal @@ -250,18 +250,18 @@ 1. The DataGridCommons generic datagrid instance has been replaced by the IsolatedDataGridCommons instance, which isolates as much information as possible to the generic instance rather than the existing DataGridCommons, which requires parameter drilling of all MUI X DataGrid parameters. Current datagrids using this new implementation are: - - `alltaxonomiesview` - - `attributes` - - `personnel` - - `quadratpersonnel` - - `quadrats` - - `roles` - - `stemtaxonomiesview` + - `alltaxonomiesview` + - `attributes` + - `personnel` + - `quadratpersonnel` + - `quadrats` + - `roles` + - `stemtaxonomiesview` 2. found that attempting to use typescript runtime utilities to create "default" initial states for each RDS type was causing cascading failures. Due to the way that runtime utility functions work, no data was actually reaching the datagrids importing those initial states - 1. replaced with manual definition of initial states -- planning on centralizing this to another place, similar to - the `datagridcolumns.tsx` file + 1. replaced with manual definition of initial states -- planning on centralizing this to another place, similar to + the `datagridcolumns.tsx` file 3. `measurementssummaryview` datagrid instance added as a replacement to the previously defined summary page #### Re-Entry Data Modal @@ -307,20 +307,19 @@ 7. materialized view reload has been adjusted to be optional. user should be able to continue the process even if one or more of the views fails. ---- +--- ### SQL Updates 1. Schema has been been updated -- new tables added: - 1. `roles` - outlines user roles - 2. `specieslimits` - allows setting min/max bounds on measurements - 3. `specimens` - recording specimen data (added on request by ForestGEO) - 4. `unifiedchangelog` - partitioned table that tracks all changes to all tables in schema. All tables have triggers - that automatically update the `unifiedchangelog` on every change - 5. `sitespecificvalidations` - for specific validations applicable only to the host site + 1. `roles` - outlines user roles + 2. `specieslimits` - allows setting min/max bounds on measurements + 3. `specimens` - recording specimen data (added on request by ForestGEO) + 4. `unifiedchangelog` - partitioned table that tracks all changes to all tables in schema. All tables have triggers + that automatically update the `unifiedchangelog` on every change + 5. `sitespecificvalidations` - for specific validations applicable only to the host site 2. validation stored procedures have been deprecated and removed, replaced with `validationprocedures` and `sitespecificvalidations` tables 3. migration script set has been completed and tested 4. trigger definitions have been recorded 5. view implementations have been updated - diff --git a/frontend/__tests__/alltaxonomiesview_uqc.test.tsx b/frontend/__tests__/alltaxonomiesview_uqc.test.tsx new file mode 100644 index 00000000..86c8dc70 --- /dev/null +++ b/frontend/__tests__/alltaxonomiesview_uqc.test.tsx @@ -0,0 +1,104 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest'; +import { getConn } from '@/components/processors/processormacros'; +import { PoolConnection } from 'mysql2/promise'; +import { AllTaxonomiesViewQueryConfig, handleUpsertForSlices } from '@/components/processors/processorhelperfunctions'; +import * as utils from '@/config/utils'; +import { AllTaxonomiesViewResult } from '@/config/sqlrdsdefinitions/views'; // Import utils module + +// Mock getConn and handleUpsert using vi.mock +vi.mock('@/components/processors/processormacros', () => ({ + getConn: vi.fn(), + runQuery: vi.fn() +})); + +vi.mock('@/config/utils', async () => { + // Import the actual utils module to access the original functions + const actualUtils = await vi.importActual('@/config/utils'); + + return { + ...actualUtils, // Keep all original utilities + handleUpsert: vi.fn() // Mock only handleUpsert + }; +}); + +describe('handleUpsertForSlices with AllTaxonomiesViewQueryConfig', () => { + let connection: PoolConnection; + + beforeEach(() => { + connection = { + beginTransaction: vi.fn(), + commit: vi.fn(), + rollback: vi.fn(), + release: vi.fn(), + query: vi.fn(), + execute: vi.fn() + } as unknown as PoolConnection; + + vi.mocked(getConn).mockResolvedValue(connection); + vi.mocked(utils.handleUpsert).mockResolvedValue(1); // Mock handleUpsert to return a consistent ID + + vi.clearAllMocks(); + }); + + it('should correctly propagate foreign keys across slices for AllTaxonomiesViewQueryConfig', async () => { + const newRow = { + Family: 'Fabaceae', + Genus: 'Acacia', + GenusAuthority: 'Willd.', + SpeciesCode: 'AC001', + SpeciesName: 'Acacia nilotica', + ValidCode: 'Y', + SubspeciesName: 'nilotica', + SpeciesAuthority: 'L.', + IDLevel: 2, + SubspeciesAuthority: 'DC.', + FieldFamily: 'Leguminosae', + Description: 'Thorny shrub' + }; + + const insertedIds = await handleUpsertForSlices(connection, 'schema_name', newRow, AllTaxonomiesViewQueryConfig); + + // Verify that handleUpsert was called 3 times (family, genus, species) + expect(utils.handleUpsert).toHaveBeenCalledTimes(3); + + // Verify that FamilyID, GenusID, and SpeciesID are propagated correctly + expect(utils.handleUpsert).toHaveBeenNthCalledWith(1, connection, 'schema_name', 'family', { Family: 'Fabaceae' }, 'FamilyID'); + expect(utils.handleUpsert).toHaveBeenNthCalledWith( + 2, + connection, + 'schema_name', + 'genus', + expect.objectContaining({ + FamilyID: 1, + Genus: 'Acacia', + GenusAuthority: 'Willd.' + }), + 'GenusID' + ); + expect(utils.handleUpsert).toHaveBeenNthCalledWith( + 3, + connection, + 'schema_name', + 'species', + expect.objectContaining({ + GenusID: 1, + SpeciesCode: 'AC001', + SpeciesName: 'Acacia nilotica', + ValidCode: 'Y', + SubspeciesName: 'nilotica', + SpeciesAuthority: 'L.', + IDLevel: 2, + SubspeciesAuthority: 'DC.', + FieldFamily: 'Leguminosae', + Description: 'Thorny shrub' + }), + 'SpeciesID' + ); + + expect(insertedIds).toEqual({ + family: 1, + genus: 1, + species: 1 + }); + }); +}); diff --git a/frontend/__tests__/stemtaxonomiesview_uqc.test.tsx b/frontend/__tests__/stemtaxonomiesview_uqc.test.tsx new file mode 100644 index 00000000..598d3706 --- /dev/null +++ b/frontend/__tests__/stemtaxonomiesview_uqc.test.tsx @@ -0,0 +1,70 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest'; +import { getConn } from '@/components/processors/processormacros'; +import { PoolConnection } from 'mysql2/promise'; +import { handleUpsertForSlices, StemTaxonomiesViewQueryConfig } from '@/components/processors/processorhelperfunctions'; +import * as utils from '@/config/utils'; // Import utils module + +// Mock getConn and handleUpsert using vi.mock +vi.mock('@/components/processors/processormacros', () => ({ + getConn: vi.fn(), + runQuery: vi.fn() +})); + +vi.mock('@/config/utils', async () => { + // Import the actual utils module to access the original functions + const actualUtils = await vi.importActual('@/config/utils'); + + return { + ...actualUtils, // Keep all original utilities + handleUpsert: vi.fn() // Mock only handleUpsert + }; +}); + +describe('handleUpsertForSlices', () => { + let connection: PoolConnection; + + beforeEach(() => { + connection = { + beginTransaction: vi.fn(), + commit: vi.fn(), + rollback: vi.fn(), + release: vi.fn(), + query: vi.fn(), + execute: vi.fn() + } as unknown as PoolConnection; + + vi.mocked(getConn).mockResolvedValue(connection); + vi.mocked(utils.handleUpsert).mockResolvedValue(1); // Mock handleUpsert to return a consistent ID + + vi.clearAllMocks(); + }); + + it('should correctly propagate foreign keys across slices', async () => { + const newRow = { + TreeTag: 'T001', + StemTag: 'S001', + SpeciesCode: 'SP001', + SpeciesName: 'Test Species' + }; + + const insertedIds = await handleUpsertForSlices(connection, 'schema_name', newRow, StemTaxonomiesViewQueryConfig); + + // Verify that handleUpsert was called 5 times (family, genus, species, trees, stems) + expect(utils.handleUpsert).toHaveBeenCalledTimes(5); + + // Verify that the FamilyID, GenusID, and SpeciesID are propagated correctly + expect(utils.handleUpsert).toHaveBeenNthCalledWith(1, connection, 'schema_name', 'family', { SpeciesCode: 'SP001' }, 'FamilyID'); + expect(utils.handleUpsert).toHaveBeenNthCalledWith(2, connection, 'schema_name', 'genus', { FamilyID: 1, Family: undefined, Genus: undefined }, 'GenusID'); + expect(utils.handleUpsert).toHaveBeenNthCalledWith(3, connection, 'schema_name', 'species', { GenusID: 1, SpeciesName: 'Test Species' }, 'SpeciesID'); + expect(utils.handleUpsert).toHaveBeenNthCalledWith(4, connection, 'schema_name', 'trees', { SpeciesID: 1, StemTag: 'S001' }, 'TreeID'); + expect(utils.handleUpsert).toHaveBeenNthCalledWith(5, connection, 'schema_name', 'stems', { TreeTag: 'T001', TreeID: 1 }, 'StemID'); + + expect(insertedIds).toEqual({ + family: 1, + genus: 1, + species: 1, + trees: 1, + stems: 1 + }); + }); +}); diff --git a/frontend/app/(hub)/dashboard/page.tsx b/frontend/app/(hub)/dashboard/page.tsx index 5343d85f..1037c43f 100644 --- a/frontend/app/(hub)/dashboard/page.tsx +++ b/frontend/app/(hub)/dashboard/page.tsx @@ -31,6 +31,7 @@ import { useEffect, useState } from 'react'; import { UnifiedChangelogRDS } from '@/config/sqlrdsdefinitions/core'; import moment from 'moment'; import Avatar from '@mui/joy/Avatar'; +import { useLoading } from '@/app/contexts/loadingprovider'; export default function DashboardPage() { const { triggerPulse, isPulsing } = useLockAnimation(); @@ -43,6 +44,8 @@ export default function DashboardPage() { const userRole = session?.user?.userStatus; const allowedSites = session?.user?.sites; + const { setLoading } = useLoading(); + const [changelogHistory, setChangelogHistory] = useState(Array(5)); const [isLoading, setIsLoading] = useState(false); diff --git a/frontend/app/(hub)/measurementshub/postvalidation/page.tsx b/frontend/app/(hub)/measurementshub/postvalidation/page.tsx index 86ecfb66..2e28040a 100644 --- a/frontend/app/(hub)/measurementshub/postvalidation/page.tsx +++ b/frontend/app/(hub)/measurementshub/postvalidation/page.tsx @@ -1,92 +1,246 @@ 'use client'; import { useOrgCensusContext, usePlotContext, useSiteContext } from '@/app/contexts/userselectionprovider'; -import { useEffect, useState } from 'react'; -import { Box, LinearProgress } from '@mui/joy'; - -interface PostValidations { - queryID: number; - queryName: string; - queryDescription: string; -} - -interface PostValidationResults { - count: number; - data: any; -} +import React, { useEffect, useState } from 'react'; +import { Box, Button, Checkbox, Table, Typography, useTheme } from '@mui/joy'; +import { PostValidationQueriesRDS } from '@/config/sqlrdsdefinitions/validations'; +import PostValidationRow from '@/components/client/postvalidationrow'; +import { Paper, TableBody, TableCell, TableContainer, TableHead, TableRow } from '@mui/material'; +import { Done } from '@mui/icons-material'; +import { useLoading } from '@/app/contexts/loadingprovider'; export default function PostValidationPage() { const currentSite = useSiteContext(); const currentPlot = usePlotContext(); const currentCensus = useOrgCensusContext(); - const [postValidations, setPostValidations] = useState([]); - const [validationResults, setValidationResults] = useState>({}); - const [loadingQueries, setLoadingQueries] = useState(false); + const [postValidations, setPostValidations] = useState([]); + const [expandedQuery, setExpandedQuery] = useState(null); + const [expandedResults, setExpandedResults] = useState(null); + const [selectedResults, setSelectedResults] = useState([]); + const replacements = { + schema: currentSite?.schemaName, + currentPlotID: currentPlot?.plotID, + currentCensusID: currentCensus?.dateRanges[0].censusID + }; + const { setLoading } = useLoading(); - // Fetch post-validation queries on first render - useEffect(() => { - async function loadQueries() { - try { - setLoadingQueries(true); - const response = await fetch(`/api/postvalidation?schema=${currentSite?.schemaName}`, { method: 'GET' }); - const data = await response.json(); - setPostValidations(data); - } catch (error) { - console.error('Error loading queries:', error); - } finally { - setLoadingQueries(false); - } + const enabledPostValidations = postValidations.filter(query => query.isEnabled); + const disabledPostValidations = postValidations.filter(query => !query.isEnabled); + + const theme = useTheme(); + const isDarkMode = theme.palette.mode === 'dark'; + + async function fetchValidationResults(postValidation: PostValidationQueriesRDS) { + if (!postValidation.queryID) return; + try { + await fetch( + `/api/postvalidationbyquery/${currentSite?.schemaName}/${currentPlot?.plotID}/${currentCensus?.dateRanges[0].censusID}/${postValidation.queryID}`, + { method: 'GET' } + ); + } catch (error: any) { + console.error(`Error fetching validation results for query ${postValidation.queryID}:`, error); + throw new Error(error); } + } - if (currentSite?.schemaName) { - loadQueries(); + async function loadPostValidations() { + try { + const response = await fetch(`/api/fetchall/postvalidationqueries?schema=${currentSite?.schemaName}`, { method: 'GET' }); + const data = await response.json(); + setPostValidations(data); + } catch (error) { + console.error('Error loading queries:', error); } - }, [currentSite?.schemaName]); + } - // Fetch validation results for each query - useEffect(() => { - async function fetchValidationResults(postValidation: PostValidations) { - try { - const response = await fetch( - `/api/postvalidationbyquery/${currentSite?.schemaName}/${currentPlot?.plotID}/${currentCensus?.dateRanges[0].censusID}/${postValidation.queryID}`, - { method: 'GET' } - ); - const data = await response.json(); - setValidationResults(prev => ({ - ...prev, - [postValidation.queryID]: data - })); - } catch (error) { - console.error(`Error fetching validation results for query ${postValidation.queryID}:`, error); - setValidationResults(prev => ({ - ...prev, - [postValidation.queryID]: null // Mark as failed if there was an error - })); - } + function saveResultsToFile() { + if (selectedResults.length === 0) { + alert('Please select at least one result to save.'); + return; + } + const blob = new Blob([JSON.stringify(selectedResults, null, 2)], { type: 'application/json' }); + const url = URL.createObjectURL(blob); + const a = document.createElement('a'); + a.href = url; + a.download = 'results.json'; + a.click(); + URL.revokeObjectURL(url); + } + + function printResults() { + if (selectedResults.length === 0) { + alert('Please select at least one result to print.'); + return; } + const printContent = selectedResults.map(result => JSON.stringify(result, null, 2)).join('\n\n'); + const printWindow = window.open('', '', 'width=600,height=400'); + printWindow?.document.write(`
${printContent}
`); + printWindow?.document.close(); + printWindow?.print(); + } - if (postValidations.length > 0 && currentPlot?.plotID && currentCensus?.dateRanges) { - postValidations.forEach(postValidation => { - fetchValidationResults(postValidation).then(r => console.log(r)); - }); + useEffect(() => { + setLoading(true); + loadPostValidations() + .catch(console.error) + .then(() => setLoading(false)); + }, []); + + const handleExpandClick = (queryID: number) => { + setExpandedQuery(expandedQuery === queryID ? null : queryID); + }; + + const handleExpandResultsClick = (queryID: number) => { + setExpandedResults(expandedResults === queryID ? null : queryID); + }; + + const handleSelectResult = (postVal: PostValidationQueriesRDS) => { + setSelectedResults(prev => (prev.includes(postVal) ? prev.filter(id => id !== postVal) : [...prev, postVal])); + }; + + const handleSelectAllChange = (event: React.ChangeEvent) => { + if (event.target.checked) { + // Select all: add all validations to selectedResults + setSelectedResults([...enabledPostValidations, ...disabledPostValidations]); + } else { + // Deselect all: clear selectedResults + setSelectedResults([]); } - }, [postValidations, currentPlot?.plotID, currentCensus?.dateRanges, currentSite?.schemaName]); + }; + + // Check if all items are selected + const isAllSelected = selectedResults.length === postValidations.length && postValidations.length > 0; return ( - - {loadingQueries ? ( - - ) : postValidations.length > 0 ? ( - - {postValidations.map(postValidation => ( - -
{postValidation.queryName}
- {validationResults[postValidation.queryID] ? : } - - ))} + + + These statistics can be used to analyze entered data. Please select and run, download, or print statistics as needed. + + + + + + + + {postValidations.length > 0 ? ( + + + + + + + + } + label={isAllSelected ? 'Deselect All' : 'Select All'} + checked={isAllSelected} + slotProps={{ + root: ({ checked, focusVisible }) => ({ + sx: !checked + ? { + '& svg': { opacity: focusVisible ? 1 : 0 }, + '&:hover svg': { + opacity: 1 + } + } + : undefined + }) + }} + onChange={e => handleSelectAllChange(e)} + /> + + Query Name + + Query Definition + + Description + Last Run At + Last Run Result + + + + + {enabledPostValidations.map(postValidation => ( + + ))} + + {disabledPostValidations.map(postValidation => ( + + ))} + +
+ ) : ( -
No validations available.
+ No validations available. )} ); diff --git a/frontend/app/(hub)/measurementshub/summary/page.tsx b/frontend/app/(hub)/measurementshub/summary/page.tsx index da8b5f8b..7b894282 100644 --- a/frontend/app/(hub)/measurementshub/summary/page.tsx +++ b/frontend/app/(hub)/measurementshub/summary/page.tsx @@ -1,4 +1,4 @@ -import MeasurementsSummaryViewDataGrid from '@/components/datagrids/applications/measurementssummaryviewdatagrid'; +import MeasurementsSummaryViewDataGrid from '@/components/datagrids/applications/msvdatagrid'; export default function SummaryPage() { return ; diff --git a/frontend/app/(hub)/measurementshub/validations/page.tsx b/frontend/app/(hub)/measurementshub/validations/page.tsx index 99afe371..8fd2c617 100644 --- a/frontend/app/(hub)/measurementshub/validations/page.tsx +++ b/frontend/app/(hub)/measurementshub/validations/page.tsx @@ -62,7 +62,6 @@ export default function ValidationsPage() { try { const response = await fetch('/api/validations/crud', { method: 'GET' }); const data = await response.json(); - console.log('data: ', data); setGlobalValidations(data); } catch (err) { console.error('Error fetching validations:', err); diff --git a/frontend/app/api/auth/[[...nextauth]]/route.ts b/frontend/app/api/auth/[[...nextauth]]/route.ts index 64a1f752..b19e1a03 100644 --- a/frontend/app/api/auth/[[...nextauth]]/route.ts +++ b/frontend/app/api/auth/[[...nextauth]]/route.ts @@ -21,33 +21,26 @@ const handler = NextAuth({ }, callbacks: { async signIn({ user, profile, email: signInEmail }) { - console.log('callback -- signin'); const azureProfile = profile as AzureADProfile; const userEmail = user.email || signInEmail || azureProfile.preferred_username; - console.log('user email: ', userEmail); if (typeof userEmail !== 'string') { console.error('User email is not a string:', userEmail); return false; // Email is not a valid string, abort sign-in } if (userEmail) { - console.log('getting connection'); let conn, emailVerified, userStatus; try { conn = await getConn(); - console.log('obtained'); const query = `SELECT UserStatus FROM catalog.users WHERE Email = '${userEmail}' LIMIT 1`; const results = await runQuery(conn, query); - console.log('results: ', results); // emailVerified is true if there is at least one result emailVerified = results.length > 0; - console.log('emailVerified: ', emailVerified); if (!emailVerified) { console.error('User email not found.'); return false; } userStatus = results[0].UserStatus; - console.log('userStatus: ', userStatus); } catch (e: any) { console.error('Error fetching user status:', e); throw new Error('Failed to fetch user status.'); @@ -66,7 +59,6 @@ const handler = NextAuth({ user.sites = allowedSites; user.allsites = allSites; - // console.log('all sites: ', user.allsites); } return true; }, diff --git a/frontend/app/api/bulkcrud/[dataType]/[[...slugs]]/route.ts b/frontend/app/api/bulkcrud/[dataType]/[[...slugs]]/route.ts new file mode 100644 index 00000000..9651d2f5 --- /dev/null +++ b/frontend/app/api/bulkcrud/[dataType]/[[...slugs]]/route.ts @@ -0,0 +1,54 @@ +// bulk data CRUD flow API endpoint -- intended to allow multiline interactions and bulk updates via datagrid +import { NextRequest, NextResponse } from 'next/server'; +import { FileRowSet } from '@/config/macros/formdetails'; +import { PoolConnection } from 'mysql2/promise'; +import { getConn, InsertUpdateProcessingProps } from '@/components/processors/processormacros'; +import { insertOrUpdate } from '@/components/processors/processorhelperfunctions'; +import { HTTPResponses } from '@/config/macros'; + +export async function POST(request: NextRequest, { params }: { params: { dataType: string; slugs?: string[] } }) { + const { dataType, slugs } = params; + if (!dataType || !slugs) { + return new NextResponse('No dataType or SLUGS provided', { status: HTTPResponses.INVALID_REQUEST }); + } + const [schema, plotIDParam, censusIDParam] = slugs; + const plotID = parseInt(plotIDParam); + const censusID = parseInt(censusIDParam); + console.log('params: schema: ', schema, ', plotID: ', plotID, ', censusID: ', censusID); + const rows: FileRowSet = await request.json(); + if (!rows) { + return new NextResponse('No rows provided', { status: 400 }); + } + console.log('rows produced: ', rows); + let conn: PoolConnection | null = null; + try { + conn = await getConn(); + for (const rowID in rows) { + const rowData = rows[rowID]; + console.log('rowData obtained: ', rowData); + const props: InsertUpdateProcessingProps = { + schema, + connection: conn, + formType: dataType, + rowData, + plotID, + censusID, + quadratID: undefined, + fullName: undefined + }; + console.log('assembled props: ', props); + await insertOrUpdate(props); + } + } catch (e: any) { + return new NextResponse( + JSON.stringify({ + responseMessage: `Failure in connecting to SQL with ${e.message}`, + error: e.message + }), + { status: HTTPResponses.INTERNAL_SERVER_ERROR } + ); + } finally { + if (conn) conn.release(); + } + return new NextResponse(JSON.stringify({ message: 'Insert to SQL successful' }), { status: HTTPResponses.OK }); +} diff --git a/frontend/app/api/catalog/[firstName]/[lastName]/route.ts b/frontend/app/api/catalog/[firstName]/[lastName]/route.ts new file mode 100644 index 00000000..220a2898 --- /dev/null +++ b/frontend/app/api/catalog/[firstName]/[lastName]/route.ts @@ -0,0 +1,26 @@ +import { NextRequest, NextResponse } from 'next/server'; +import { PoolConnection } from 'mysql2/promise'; +import { getConn, runQuery } from '@/components/processors/processormacros'; +import { HTTPResponses } from '@/config/macros'; + +export async function GET(_request: NextRequest, { params }: { params: { firstName: string; lastName: string } }) { + const { firstName, lastName } = params; + if (!firstName || !lastName) throw new Error('no first or last name provided!'); + + let conn: PoolConnection | null = null; + + try { + conn = await getConn(); + const query = `SELECT UserID FROM catalog.users WHERE FirstName = ? AND LastName = ?;`; + const results = await runQuery(conn, query, [firstName, lastName]); + if (results.length === 0) { + throw new Error('User not found'); + } + return new NextResponse(JSON.stringify(results[0].UserID), { status: HTTPResponses.OK }); + } catch (e: any) { + console.error('Error in GET request:', e.message); + return new NextResponse(JSON.stringify({ error: e.message }), { status: HTTPResponses.INTERNAL_SERVER_ERROR }); + } finally { + if (conn) conn.release(); + } +} diff --git a/frontend/app/api/cmprevalidation/[dataType]/[[...slugs]]/route.ts b/frontend/app/api/cmprevalidation/[dataType]/[[...slugs]]/route.ts index 23a2f19f..963accc9 100644 --- a/frontend/app/api/cmprevalidation/[dataType]/[[...slugs]]/route.ts +++ b/frontend/app/api/cmprevalidation/[dataType]/[[...slugs]]/route.ts @@ -56,6 +56,18 @@ export async function GET(_request: NextRequest, { params }: { params: { dataTyp status: HTTPResponses.PRECONDITION_VALIDATION_FAILURE }); break; + case 'postvalidation': + const pvQuery = `SELECT 1 FROM ${schema}.coremeasurements cm + JOIN ${schema}.census c ON C.CensusID = cm.CensusID + JOIN ${schema}.plots p ON p.PlotID = c.PlotID + WHERE p.PlotID = ${plotID} AND c.PlotCensusNumber = ${plotCensusNumber} LIMIT 1`; + const pvResults = await runQuery(connection, pvQuery); + if (connection) connection.release(); + if (pvResults.length === 0) + return new NextResponse(null, { + status: HTTPResponses.PRECONDITION_VALIDATION_FAILURE + }); + break; // case 'subquadrats': // const subquadratsQuery = `SELECT 1 // FROM ${schema}.${params.dataType} s diff --git a/frontend/app/api/fetchall/[[...slugs]]/route.ts b/frontend/app/api/fetchall/[[...slugs]]/route.ts index 42daf7c6..bbaf8c6f 100644 --- a/frontend/app/api/fetchall/[[...slugs]]/route.ts +++ b/frontend/app/api/fetchall/[[...slugs]]/route.ts @@ -14,7 +14,7 @@ const buildQuery = (schema: string, fetchType: string, plotID?: string, plotCens ${schema}.quadrats q ON p.PlotID = q.PlotID GROUP BY p.PlotID ${plotID && plotID !== 'undefined' && !isNaN(parseInt(plotID)) ? `HAVING p.PlotID = ${plotID}` : ''}`; - } else if (fetchType === 'roles') { + } else if (fetchType === 'roles' || fetchType === 'attributes') { return `SELECT * FROM ${schema}.${fetchType}`; } else if (fetchType === 'quadrats') { diff --git a/frontend/app/api/fixeddata/[dataType]/[[...slugs]]/route.ts b/frontend/app/api/fixeddata/[dataType]/[[...slugs]]/route.ts index f8cd1329..0e37856f 100644 --- a/frontend/app/api/fixeddata/[dataType]/[[...slugs]]/route.ts +++ b/frontend/app/api/fixeddata/[dataType]/[[...slugs]]/route.ts @@ -64,9 +64,9 @@ export async function GET( break; case 'personnel': paginatedQuery = ` - SELECT SQL_CALC_FOUND_ROWS q.* - FROM ${schema}.${params.dataType} q - JOIN ${schema}.census c ON q.CensusID = c.CensusID + SELECT SQL_CALC_FOUND_ROWS p.* + FROM ${schema}.${params.dataType} p + JOIN ${schema}.census c ON p.CensusID = c.CensusID WHERE c.PlotID = ? AND c.PlotCensusNumber = ? LIMIT ?, ?;`; queryParams.push(plotID, plotCensusNumber, page * pageSize, pageSize); @@ -100,17 +100,18 @@ export async function GET( queryParams.push(plotID, plotCensusNumber, page * pageSize, pageSize); break; case 'measurementssummary': + case 'measurementssummary_staging': case 'measurementssummaryview': case 'viewfulltable': case 'viewfulltableview': paginatedQuery = ` - SELECT SQL_CALC_FOUND_ROWS q.* - FROM ${schema}.${params.dataType} q - JOIN ${schema}.census c ON q.PlotID = c.PlotID AND q.CensusID = c.CensusID - WHERE q.PlotID = ? + SELECT SQL_CALC_FOUND_ROWS vft.* + FROM ${schema}.${params.dataType} vft + JOIN ${schema}.census c ON vft.PlotID = c.PlotID AND vft.CensusID = c.CensusID + WHERE vft.PlotID = ? AND c.PlotID = ? AND c.PlotCensusNumber = ? - ORDER BY q.MeasurementDate ASC LIMIT ?, ?;`; + ORDER BY vft.MeasurementDate ASC LIMIT ?, ?;`; queryParams.push(plotID, plotID, plotCensusNumber, page * pageSize, pageSize); break; // case 'subquadrats': @@ -364,7 +365,6 @@ export async function DELETE(request: NextRequest, { params }: { params: { dataT let conn: PoolConnection | null = null; const demappedGridID = gridID.charAt(0).toUpperCase() + gridID.substring(1); const { newRow } = await request.json(); - console.log('newrow: ', newRow); try { conn = await getConn(); await conn.beginTransaction(); diff --git a/frontend/app/api/formdownload/[dataType]/[[...slugs]]/route.ts b/frontend/app/api/formdownload/[dataType]/[[...slugs]]/route.ts new file mode 100644 index 00000000..0f3b83d6 --- /dev/null +++ b/frontend/app/api/formdownload/[dataType]/[[...slugs]]/route.ts @@ -0,0 +1,130 @@ +import { NextRequest, NextResponse } from 'next/server'; +import { PoolConnection } from 'mysql2/promise'; +import { getConn, runQuery } from '@/components/processors/processormacros'; +import MapperFactory from '@/config/datamapper'; +import { AttributesRDS } from '@/config/sqlrdsdefinitions/core'; +import { HTTPResponses } from '@/config/macros'; + +export async function GET(_request: NextRequest, { params }: { params: { dataType: string; slugs?: string[] } }) { + const { dataType, slugs } = params; + if (!dataType || !slugs) throw new Error('data type or slugs not provided'); + const [schema, plotIDParam, censusIDParam] = slugs; + if (!schema) throw new Error('no schema provided'); + + const plotID = plotIDParam ? parseInt(plotIDParam) : undefined; + const censusID = censusIDParam ? parseInt(censusIDParam) : undefined; + + let conn: PoolConnection | null = null; + let query: string = ''; + let results: any[] = []; + let mappedResults: any[] = []; + let formMappedResults: any[] = []; + try { + conn = await getConn(); + switch (dataType) { + case 'attributes': + query = `SELECT * FROM ${schema}.attributes`; + results = await runQuery(conn, query); + mappedResults = MapperFactory.getMapper('attributes').mapData(results); + formMappedResults = mappedResults.map((row: AttributesRDS) => ({ + code: row.code, + description: row.description, + status: row.status + })); + return new NextResponse(JSON.stringify(formMappedResults), { status: HTTPResponses.OK }); + case 'personnel': + query = `SELECT p.FirstName AS FirstName, p.LastName AS LastName, r.RoleName AS RoleName, r.RoleDescription AS RoleDescription + FROM ${schema}.personnel p + LEFT JOIN ${schema}.roles r ON p.RoleID = r.RoleID + LEFT JOIN ${schema}.census c ON c.CensusID = p.CensusID + WHERE c.PlotID = ? AND p.CensusID = ?`; + results = await runQuery(conn, query, [plotID, censusID]); + formMappedResults = results.map((row: any) => ({ + firstname: row.FirstName, + lastname: row.LastName, + role: row.RoleName, + roledescription: row.RoleDescription + })); + return new NextResponse(JSON.stringify(formMappedResults), { status: HTTPResponses.OK }); + case 'species': + query = `SELECT DISTINCT s.SpeciesCode AS SpeciesCode, f.Family AS Family, + g.Genus AS Genus, s.SpeciesName AS SpeciesName, s.SubspeciesName AS SubspeciesName, + s.IDLevel AS IDLevel, s.SpeciesAuthority AS SpeciesAuthority, s.SubspeciesAuthority AS SubspeciesAuthority + FROM ${schema}.species s + JOIN ${schema}.genus g ON g.GenusID = s.GenusID + JOIN ${schema}.family f ON f.FamilyID = g.FamilyID + JOIN ${schema}.trees t ON t.SpeciesID = s.SpeciesID + JOIN ${schema}.stems st ON st.TreeID = t.TreeID + JOIN ${schema}.quadrats q ON q.QuadratID = st.QuadratID + JOIN ${schema}.censusquadrat cq ON cq.QuadratID = q.QuadratID + WHERE q.PlotID = ? AND cq.CensusID = ?`; + results = await runQuery(conn, query, [plotID, censusID]); + formMappedResults = results.map((row: any) => ({ + spcode: row.SpeciesCode, + family: row.Family, + genus: row.Genus, + species: row.SpeciesName, + subspecies: row.SubspeciesName, + idlevel: row.IDLevel, + authority: row.SpeciesAuthority, + subspeciesauthority: row.SubspeciesAuthority + })); + return new NextResponse(JSON.stringify(formMappedResults), { status: HTTPResponses.OK }); + case 'quadrats': + query = `SELECT * FROM ${schema}.quadrats q + JOIN ${schema}.censusquadrat cq ON cq.QuadratID = q.QuadratID + WHERE q.PlotID = ? AND cq.CensusID = ?`; + results = await runQuery(conn, query, [plotID, censusID]); + formMappedResults = results.map((row: any) => ({ + quadrat: row.QuadratName, + startx: row.StartX, + starty: row.StartY, + coordinateunit: row.CoordinateUnits, + dimx: row.DimensionX, + dimy: row.DimensionY, + dimensionunit: row.DimensionUnits, + area: row.Area, + areaunit: row.AreaUnits, + quadratshape: row.QuadratShape + })); + return new NextResponse(JSON.stringify(formMappedResults), { status: HTTPResponses.OK }); + case 'measurements': + query = `SELECT st.StemTag AS StemTag, t.TreeTag AS TreeTag, s.SpeciesCode AS SpeciesCode, q.QuadratName AS QuadratName, + q.StartX AS StartX, q.StartY AS StartY, q.CoordinateUnits AS CoordinateUnits, cm.MeasuredDBH AS MeasuredDBH, cm.DBHUnit AS DBHUnit, + cm.MeasuredHOM AS MeasuredHOM, cm.HOMUnit AS HOMUnit, cm.MeasurementDate AS MeasurementDate, + (SELECT GROUP_CONCAT(ca.Code SEPARATOR '; ') + FROM ${schema}.cmattributes ca + WHERE ca.CoreMeasurementID = cm.CoreMeasurementID) AS Codes + FROM ${schema}.coremeasurements cm + JOIN ${schema}.stems st ON st.StemID = cm.StemID + JOIN ${schema}.trees t ON t.TreeID = st.TreeID + JOIN ${schema}.quadrats q ON q.QuadratID = st.QuadratID + JOIN ${schema}.censusquadrat cq ON cq.QuadratID = q.QuadratID + JOIN ${schema}.species s ON s.SpeciesID = t.SpeciesID + WHERE q.PlotID = ? AND cq.CensusID = ?`; + results = await runQuery(conn, query, [plotID, censusID]); + formMappedResults = results.map((row: any) => ({ + tag: row.TreeTag, + stemtag: row.StemTag, + spcode: row.SpeciesCode, + quadrat: row.QuadratName, + lx: row.StartX, + ly: row.StartY, + coordinateunit: row.CoordinateUnits, + dbh: row.MeasuredDBH, + dbhunit: row.DBHUnit, + hom: row.MeasuredHOM, + homunit: row.HOMUnit, + date: row.MeasurementDate, + codes: row.Codes + })); + return new NextResponse(JSON.stringify(formMappedResults), { status: HTTPResponses.OK }); + default: + throw new Error('incorrect data type passed in'); + } + } catch (e: any) { + throw new Error(e); + } finally { + if (conn) conn.release(); + } +} diff --git a/frontend/app/api/postvalidation/route.ts b/frontend/app/api/postvalidation/route.ts index 0df42b31..e1ee5424 100644 --- a/frontend/app/api/postvalidation/route.ts +++ b/frontend/app/api/postvalidation/route.ts @@ -58,7 +58,7 @@ export async function GET(request: NextRequest) { // JOIN ${schema}.stems s ON s.TreeID = t.TreeID // JOIN ${schema}.quadrats q ON q.QuadratID = s.QuadratID // WHERE q.CensusID = ${currentCensusID} AND q.PlotID = ${currentPlotID};`, -// countNumDeadMissingByCensus: `SELECT cm.CensusID, COUNT(s.StemID) AS DeadOrMissingStems +// countNumDeadMissingByCensus: `SELECT s.StemID, COUNT(s.StemID) AS DeadOrMissingStems // FROM ${schema}.stems s // JOIN ${schema}.cmattributes cma ON s.StemID = cma.CoreMeasurementID // JOIN ${schema}.attributes a ON cma.Code = a.Code diff --git a/frontend/app/api/postvalidationbyquery/[schema]/[plotID]/[censusID]/[queryID]/route.ts b/frontend/app/api/postvalidationbyquery/[schema]/[plotID]/[censusID]/[queryID]/route.ts index 90a100dc..e40acc17 100644 --- a/frontend/app/api/postvalidationbyquery/[schema]/[plotID]/[censusID]/[queryID]/route.ts +++ b/frontend/app/api/postvalidationbyquery/[schema]/[plotID]/[censusID]/[queryID]/route.ts @@ -1,36 +1,55 @@ import { NextRequest, NextResponse } from 'next/server'; import { HTTPResponses } from '@/config/macros'; import { getConn, runQuery } from '@/components/processors/processormacros'; +import moment from 'moment'; export async function GET(_request: NextRequest, { params }: { params: { schema: string; plotID: string; censusID: string; queryID: string } }) { const { schema } = params; const plotID = parseInt(params.plotID); const censusID = parseInt(params.censusID); const queryID = parseInt(params.queryID); + if (!schema || !plotID || !censusID || !queryID) { return new NextResponse('Missing parameters', { status: HTTPResponses.INVALID_REQUEST }); } + const conn = await getConn(); - const query = `SELECT QueryDefinition FROM ${schema}.postvalidationqueries WHERE QueryID = ${queryID}`; - const results = await runQuery(conn, query); - if (results.length === 0) { - return new NextResponse('Query not found', { status: HTTPResponses.NOT_FOUND }); - } - const replacements = { - schema: schema, - currentPlotID: plotID, - currentCensusID: censusID - }; - const formattedQuery = results[0].QueryDefinition.replace(/\${(.*?)}/g, (_match: any, p1: string) => replacements[p1 as keyof typeof replacements]); - const queryResults = await runQuery(conn, formattedQuery); - if (queryResults.length === 0) { - return new NextResponse('Query returned no results', { status: HTTPResponses.NOT_FOUND }); + try { + const query = `SELECT QueryDefinition FROM ${schema}.postvalidationqueries WHERE QueryID = ${queryID}`; + const results = await runQuery(conn, query); + + if (results.length === 0) return new NextResponse('Query not found', { status: HTTPResponses.NOT_FOUND }); + + const replacements = { + schema: schema, + currentPlotID: plotID, + currentCensusID: censusID + }; + const formattedQuery = results[0].QueryDefinition.replace(/\${(.*?)}/g, (_match: any, p1: string) => replacements[p1 as keyof typeof replacements]); + + const queryResults = await runQuery(conn, formattedQuery); + + if (queryResults.length === 0) throw new Error('failure'); + + const currentTime = moment().format('YYYY-MM-DD HH:mm:ss'); + const successResults = JSON.stringify(queryResults); + const successUpdate = `UPDATE ${schema}.postvalidationqueries + SET LastRunAt = ?, LastRunResult = ?, LastRunStatus = 'success' + WHERE QueryID = ${queryID}`; + await runQuery(conn, successUpdate, [currentTime, successResults]); + + return new NextResponse(null, { status: HTTPResponses.OK }); + } catch (e: any) { + if (e.message === 'failure') { + const currentTime = moment().format('YYYY-MM-DD HH:mm:ss'); + const failureUpdate = `UPDATE ${schema}.postvalidationqueries + SET LastRunAt = ?, LastRunStatus = 'failure' + WHERE QueryID = ${queryID}`; + await runQuery(conn, failureUpdate, [currentTime]); + return new NextResponse(null, { status: HTTPResponses.OK }); // if the query itself fails, that isn't a good enough reason to return a crash. It should just be logged. + } + return new NextResponse('Internal Server Error', { status: HTTPResponses.INTERNAL_SERVER_ERROR }); + } finally { + if (conn) conn.release(); } - return new NextResponse( - JSON.stringify({ - count: queryResults.length, - data: queryResults - }), - { status: HTTPResponses.OK } - ); } diff --git a/frontend/app/api/sqlload/route.ts b/frontend/app/api/sqlload/route.ts index 3e82e070..b448ae36 100644 --- a/frontend/app/api/sqlload/route.ts +++ b/frontend/app/api/sqlload/route.ts @@ -39,7 +39,6 @@ export async function POST(request: NextRequest) { let connection: PoolConnection | null = null; // Use PoolConnection type try { - const i = 0; connection = await getConn(); } catch (error) { if (error instanceof Error) { diff --git a/frontend/app/api/validations/procedures/[validationType]/route.ts b/frontend/app/api/validations/procedures/[validationType]/route.ts index 7606a619..43f23c52 100644 --- a/frontend/app/api/validations/procedures/[validationType]/route.ts +++ b/frontend/app/api/validations/procedures/[validationType]/route.ts @@ -5,7 +5,6 @@ import { HTTPResponses } from '@/config/macros'; export async function POST(request: NextRequest, { params }: { params: { validationProcedureName: string } }) { try { const { schema, validationProcedureID, cursorQuery, p_CensusID, p_PlotID, minDBH, maxDBH, minHOM, maxHOM } = await request.json(); - console.log('data: ', schema, validationProcedureID, cursorQuery, p_CensusID, p_PlotID, minDBH, maxDBH, minHOM, maxHOM); // Execute the validation procedure using the provided inputs const validationResponse = await runValidation(validationProcedureID, params.validationProcedureName, schema, cursorQuery, { diff --git a/frontend/components/client/datagridcolumns.tsx b/frontend/components/client/datagridcolumns.tsx index e5cc4cd3..39a724d7 100644 --- a/frontend/components/client/datagridcolumns.tsx +++ b/frontend/components/client/datagridcolumns.tsx @@ -1,16 +1,11 @@ import { areaSelectionOptions, unitSelectionOptions } from '@/config/macros'; -import { Accordion, AccordionDetails, AccordionGroup, AccordionSummary, Box, FormHelperText, Input, Option, Select, Stack, Typography } from '@mui/joy'; +import { Box, FormHelperText, Input, Option, Select, Stack, Typography } from '@mui/joy'; import { GridColDef, GridRenderEditCellParams, useGridApiRef } from '@mui/x-data-grid'; import React, { useEffect, useState } from 'react'; -import Avatar from '@mui/joy/Avatar'; -import { ExpandMore } from '@mui/icons-material'; -import { useSession } from 'next-auth/react'; -import CodeMirror from '@uiw/react-codemirror'; -import { sql } from '@codemirror/lang-sql'; import { AttributeStatusOptions } from '@/config/sqlrdsdefinitions/core'; export const formatHeader = (word1: string, word2: string) => ( - + {word1} @@ -79,7 +74,6 @@ export const quadratGridColumns: GridColDef[] = [ headerName: 'Coordinate Units', headerClassName: 'header', flex: 1, - // renderHeader: () => formatHeader('Coordinate', 'Units'), align: 'right', headerAlign: 'right', editable: true, @@ -104,7 +98,6 @@ export const quadratGridColumns: GridColDef[] = [ headerName: 'Area Unit', headerClassName: 'header', flex: 1, - // renderHeader: () => formatHeader('Area', 'Unit'), align: 'right', headerAlign: 'right', editable: true, @@ -550,7 +543,7 @@ export const MeasurementsSummaryViewGridColumns: GridColDef[] = [ editable: true }, { - field: 'stemLocalX', + field: 'localX', headerName: 'X', headerAlign: 'left', headerClassName: 'header', @@ -564,7 +557,7 @@ export const MeasurementsSummaryViewGridColumns: GridColDef[] = [ editable: true }, { - field: 'stemLocalY', + field: 'localY', headerName: 'Y', headerAlign: 'left', headerClassName: 'header', @@ -578,7 +571,7 @@ export const MeasurementsSummaryViewGridColumns: GridColDef[] = [ editable: true }, { - field: 'stemUnits', + field: 'coordinateUnits', headerName: 'Stem Units', headerClassName: 'header', flex: 0.4, @@ -596,30 +589,9 @@ export const MeasurementsSummaryViewGridColumns: GridColDef[] = [ flex: 0.8, align: 'right', editable: true, - // type: 'number', - // valueFormatter: (value: any) => { - // return Number(value).toFixed(2); - // } renderCell: renderDBHCell, renderEditCell: renderEditDBHCell - // valueFormatter: (params: any) => { - // const value = params.row.measuredDBH ? Number(params.row.measuredDBH).toFixed(2) : 'null'; - // const units = params.row.dbhUnits || ''; - // return `${value} ${units}`; - // } }, - // { - // field: 'dbhUnits', - // headerName: 'DBH Units', - // headerClassName: 'header', - // flex: 0.4, - // maxWidth: 65, - // renderHeader: () => formatHeader('DBH', 'Units'), - // align: 'center', - // editable: true, - // type: 'singleSelect', - // valueOptions: unitSelectionOptions - // }, { field: 'measuredHOM', headerName: 'HOM', @@ -628,30 +600,9 @@ export const MeasurementsSummaryViewGridColumns: GridColDef[] = [ align: 'right', headerAlign: 'left', editable: true, - // type: 'number', - // valueFormatter: (value: any) => { - // return Number(value).toFixed(2); - // } renderCell: renderHOMCell, renderEditCell: renderEditHOMCell - // valueFormatter: (params: any) => { - // const value = params.row.measuredDBH ? Number(params.row.measuredDBH).toFixed(2) : 'null'; - // const units = params.row.dbhUnits || ''; - // return `${value} ${units}`; - // } }, - // { - // field: 'homUnits', - // headerName: 'HOM Units', - // headerClassName: 'header', - // flex: 0.4, - // maxWidth: 65, - // renderHeader: () => formatHeader('HOM', 'Units'), - // align: 'center', - // editable: true, - // type: 'singleSelect', - // valueOptions: unitSelectionOptions - // }, { field: 'description', headerName: 'Description', @@ -663,7 +614,7 @@ export const MeasurementsSummaryViewGridColumns: GridColDef[] = [ { field: 'attributes', headerName: 'Attributes', headerClassName: 'header', flex: 1, align: 'left', editable: true } ]; -export const CensusGridColumns: GridColDef[] = [ +export const StemGridColumns: GridColDef[] = [ { field: 'id', headerName: '#', @@ -674,164 +625,69 @@ export const CensusGridColumns: GridColDef[] = [ editable: false }, { - field: 'censusID', - headerName: 'ID', - type: 'number', + field: 'stemTag', + headerName: 'Stem Tag', headerClassName: 'header', flex: 1, align: 'left', - editable: false + type: 'string', + editable: true }, { - field: 'plotCensusNumber', - headerName: 'PlotCensusNumber', - type: 'number', + field: 'localX', + headerName: 'Plot X', headerClassName: 'header', flex: 1, align: 'left', - editable: false + type: 'number', + valueFormatter: (value: any) => { + return Number(value).toFixed(2); + }, + editable: true }, { - field: 'startDate', - headerName: 'Starting', + field: 'localY', + headerName: 'Plot Y', headerClassName: 'header', flex: 1, align: 'left', - type: 'date', - editable: true, - valueFormatter: (params: any) => { - if (params) { - return new Date(params).toDateString(); - } else return 'null'; - } + type: 'number', + valueFormatter: (value: any) => { + return Number(value).toFixed(2); + }, + editable: true }, { - field: 'endDate', - headerName: 'Ending', + field: 'coordinateUnits', + headerName: 'Unit', headerClassName: 'header', - type: 'date', flex: 1, align: 'left', - editable: true, - valueFormatter: (params: any) => { - if (params) { - return new Date(params).toDateString(); - } else return 'null'; - } - }, - { - field: 'description', - headerName: 'Description', - headerClassName: 'header', - flex: 1, - type: 'string', + type: 'singleSelect', + valueOptions: unitSelectionOptions, editable: true - } -]; - -export const ValidationErrorGridColumns: GridColDef[] = [ - { - field: 'id', - headerName: '#', - headerClassName: 'header', - flex: 0.3, - align: 'right', - headerAlign: 'right', - editable: false - }, - { field: 'validationErrorID', headerName: 'ValidationErrorID', headerClassName: 'header', flex: 1, align: 'left' }, - { - field: 'validationErrorDescription', - headerName: 'ValidationErrorDescription', - headerClassName: 'header', - flex: 1, - align: 'left' - } -]; - -export const CoreMeasurementsGridColumns: GridColDef[] = [ - { - field: 'id', - headerName: 'ID', - headerClassName: 'header', - flex: 0.3, - align: 'right', - headerAlign: 'right', - editable: false - }, - { - field: 'coreMeasurementID', - headerName: '#', - headerAlign: 'left', - headerClassName: 'header', - flex: 0.25, - align: 'left' }, { - field: 'censusID', - headerName: 'Census ID', - headerAlign: 'left', + field: 'moved', + headerName: 'Moved', headerClassName: 'header', flex: 1, align: 'left', + type: 'boolean', editable: true }, { - field: 'stemID', - headerName: 'Stem ID', - headerAlign: 'left', + field: 'stemDescription', + headerName: 'StemDescription', headerClassName: 'header', flex: 1, align: 'left', + type: 'string', editable: true - }, - { - field: 'measuredDBH', - headerName: 'DBH', - headerClassName: 'header', - flex: 0.8, - align: 'right', - editable: true, - renderCell: renderDBHCell, - renderEditCell: renderEditDBHCell - }, - { - field: 'dbhUnits', - headerName: 'DBH Units', - headerClassName: 'header', - flex: 0.4, - maxWidth: 65, - renderHeader: () => formatHeader('DBH', 'Units'), - align: 'center', - editable: true, - type: 'singleSelect', - valueOptions: unitSelectionOptions - }, - { - field: 'measuredHOM', - headerName: 'HOM', - headerClassName: 'header', - flex: 0.5, - align: 'right', - headerAlign: 'left', - editable: true, - renderCell: renderHOMCell, - renderEditCell: renderEditHOMCell - }, - { - field: 'homUnits', - headerName: 'HOM Units', - headerClassName: 'header', - maxWidth: 65, - renderHeader: () => formatHeader('HOM', 'Units'), - align: 'center', - editable: true, - type: 'singleSelect', - valueOptions: unitSelectionOptions } ]; -export const SubquadratGridColumns: GridColDef[] = [ +export const SpeciesLimitsGridColumns: GridColDef[] = [ { field: 'id', headerName: '#', @@ -841,75 +697,67 @@ export const SubquadratGridColumns: GridColDef[] = [ headerAlign: 'right', editable: false }, - { field: 'ordering', headerName: 'Order', headerClassName: 'header', flex: 1, align: 'left', editable: false }, { - field: 'subquadratName', - headerName: 'Name', + field: 'speciesLimitID', + headerName: '#', headerClassName: 'header', - flex: 1, + flex: 0.3, align: 'left', - type: 'string', - editable: true + headerAlign: 'left', + editable: false }, - { field: 'quadratID', headerName: 'Quadrat', headerClassName: 'header', flex: 1, align: 'left', editable: false }, { - field: 'dimensionX', - headerName: 'X-Dimension', + field: 'speciesID', + headerName: 'SpeciesID', headerClassName: 'header', - flex: 1, + flex: 0.3, align: 'left', - type: 'number', - editable: true + headerAlign: 'left', + editable: false }, { - field: 'dimensionY', - headerName: 'Y-Dimension', - headerClassName: 'header', - flex: 1, + field: 'limitType', + headerName: 'LimitType', + renderHeader: () => formatHeader('Limit', 'Type'), + flex: 0.5, align: 'left', - type: 'number', - valueFormatter: (value: any) => { - return Number(value).toFixed(2); - }, + headerAlign: 'left', + type: 'singleSelect', + valueOptions: ['DBH', 'HOM'], editable: true }, { - field: 'qX', - headerName: 'X', - headerClassName: 'header', - flex: 1, + field: 'lowerBound', + headerName: 'LowerBound', + renderHeader: () => formatHeader('Lower', 'Limit'), + flex: 0.5, align: 'left', + headerAlign: 'left', type: 'number', - valueFormatter: (value: any) => { - return Number(value).toFixed(2); - }, editable: true }, { - field: 'qY', - headerName: 'Y', - headerClassName: 'header', - flex: 1, + field: 'upperBound', + headerName: 'UpperBound', + renderHeader: () => formatHeader('Upper', 'Limit'), + flex: 0.5, align: 'left', + headerAlign: 'left', type: 'number', - valueFormatter: (value: any) => { - return Number(value).toFixed(2); - }, editable: true }, { field: 'unit', headerName: 'Units', headerClassName: 'header', - flex: 1, + flex: 0.3, align: 'left', type: 'singleSelect', - valueOptions: unitSelectionOptions, - editable: true + valueOptions: unitSelectionOptions } ]; -export const StemGridColumns: GridColDef[] = [ +export const RolesGridColumns: GridColDef[] = [ { field: 'id', headerName: '#', @@ -920,523 +768,27 @@ export const StemGridColumns: GridColDef[] = [ editable: false }, { - field: 'stemTag', - headerName: 'Stem Tag', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'string', - editable: true - }, - { - field: 'localX', - headerName: 'Plot X', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'number', - valueFormatter: (value: any) => { - return Number(value).toFixed(2); - }, - editable: true - }, - { - field: 'localY', - headerName: 'Plot Y', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'number', - valueFormatter: (value: any) => { - return Number(value).toFixed(2); - }, - editable: true - }, - { - field: 'coordinateUnits', - headerName: 'Unit', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'singleSelect', - valueOptions: unitSelectionOptions, - editable: true - }, - { - field: 'moved', - headerName: 'Moved', + field: 'roleID', + headerName: '#', headerClassName: 'header', - flex: 1, - align: 'left', - type: 'boolean', - editable: true + flex: 0.2, + align: 'right', + headerAlign: 'right', + editable: false }, + { field: 'roleName', headerName: 'Role', headerClassName: 'header', flex: 1, align: 'left', editable: true }, { - field: 'stemDescription', - headerName: 'StemDescription', + field: 'roleDescription', + headerName: 'Description', headerClassName: 'header', flex: 1, align: 'left', - type: 'string', editable: true } ]; - -export const SpeciesInventoryGridColumns: GridColDef[] = [ - { - field: 'id', - headerName: '#', - headerClassName: 'header', - flex: 0.3, - align: 'right', - headerAlign: 'right', - editable: false - }, - { field: 'speciesInventoryID', headerName: 'SpeciesInventoryID', headerClassName: 'header', flex: 1, align: 'left' }, - { field: 'censusID', headerName: 'CensusID', headerClassName: 'header', flex: 1, align: 'left' }, - { field: 'plotID', headerName: 'PlotID', headerClassName: 'header', flex: 1, align: 'left' }, - { field: 'speciesID', headerName: 'SpeciesID', headerClassName: 'header', flex: 1, align: 'left' }, - { field: 'subSpeciesID', headerName: 'SubSpeciesID', headerClassName: 'header', flex: 1, align: 'left' } -]; - -export const SpeciesGridColumns: GridColDef[] = [ - { - field: 'id', - headerName: '#', - headerClassName: 'header', - flex: 0.3, - align: 'right', - headerAlign: 'right', - editable: false - }, - { - field: 'speciesCode', - headerName: 'SpCode', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'string', - editable: true, - maxWidth: 125 - }, - { - field: 'speciesName', - headerName: 'Species', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'string', - editable: true - }, - { - field: 'subspeciesName', - headerName: 'Subspecies', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'string', - editable: true - }, - { - field: 'idLevel', - headerName: 'IDLevel', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'string', - editable: true - }, - { - field: 'speciesAuthority', - headerName: 'SpeciesAuth', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'string', - editable: true - }, - { - field: 'subspeciesAuthority', - headerName: 'SubspeciesAuth', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'string', - editable: true - }, - { - field: 'fieldFamily', - headerName: 'FieldFamily', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'string', - editable: true - }, - { - field: 'description', - headerName: 'Description', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'string', - editable: true - }, - { - field: 'validCode', - headerName: 'Valid Code', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'string', - editable: true - } -]; - -export const SpeciesLimitsGridColumns: GridColDef[] = [ - { - field: 'id', - headerName: '#', - headerClassName: 'header', - flex: 0.3, - align: 'right', - headerAlign: 'right', - editable: false - }, - { - field: 'speciesLimitID', - headerName: '#', - headerClassName: 'header', - flex: 0.3, - align: 'left', - headerAlign: 'left', - editable: false - }, - { - field: 'speciesID', - headerName: 'SpeciesID', - headerClassName: 'header', - flex: 0.3, - align: 'left', - headerAlign: 'left', - editable: false - }, - { - field: 'limitType', - headerName: 'LimitType', - renderHeader: () => formatHeader('Limit', 'Type'), - flex: 0.5, - align: 'left', - headerAlign: 'left', - type: 'singleSelect', - valueOptions: ['DBH', 'HOM'], - editable: true - }, - { - field: 'lowerBound', - headerName: 'LowerBound', - renderHeader: () => formatHeader('Lower', 'Limit'), - flex: 0.5, - align: 'left', - headerAlign: 'left', - type: 'number', - editable: true - }, - { - field: 'upperBound', - headerName: 'UpperBound', - renderHeader: () => formatHeader('Upper', 'Limit'), - flex: 0.5, - align: 'left', - headerAlign: 'left', - type: 'number', - editable: true - }, - { - field: 'unit', - headerName: 'Units', - headerClassName: 'header', - flex: 0.3, - align: 'left', - type: 'singleSelect', - valueOptions: unitSelectionOptions - } -]; - -export const RolesGridColumns: GridColDef[] = [ - { - field: 'id', - headerName: '#', - headerClassName: 'header', - flex: 0.3, - align: 'right', - headerAlign: 'right', - editable: false - }, - { - field: 'roleID', - headerName: '#', - headerClassName: 'header', - flex: 0.2, - align: 'right', - headerAlign: 'right', - editable: false - }, - // { field: 'roleID', headerName: 'RoleID', headerClassName: 'header', flex: 1, align: 'left', editable: false }, - { field: 'roleName', headerName: 'Role', headerClassName: 'header', flex: 1, align: 'left', editable: true }, - { - field: 'roleDescription', - headerName: 'Description', - headerClassName: 'header', - flex: 1, - align: 'left', - editable: true - } -]; - -export const ReferenceGridColumns: GridColDef[] = [ - { - field: 'id', - headerName: '#', - headerClassName: 'header', - flex: 0.3, - align: 'right', - headerAlign: 'right', - editable: false - }, - { field: 'referenceID', headerName: 'ReferenceID', headerClassName: 'header', flex: 1, align: 'left' }, - { field: 'publicationTitle', headerName: 'PublicationTitle', headerClassName: 'header', flex: 1, align: 'left' }, - { field: 'fullReference', headerName: 'FullReference', headerClassName: 'header', flex: 1, align: 'left' }, - { - field: 'dateOfPublication', - headerName: 'DateOfPublication', - type: 'date', - headerClassName: 'header', - flex: 1, - align: 'left', - valueGetter: (params: any) => { - if (!params.value) return null; - return new Date(params.value); - } - } -]; - -export const PlotGridColumns: GridColDef[] = [ - { - field: 'id', - headerName: '#', - headerClassName: 'header', - flex: 0.3, - align: 'right', - headerAlign: 'right', - editable: false - }, - { field: 'plotID', headerName: 'PlotID', headerClassName: 'header', flex: 1, align: 'left', editable: false }, - { field: 'plotName', headerName: 'PlotName', headerClassName: 'header', flex: 1, align: 'left', editable: true }, - { - field: 'locationName', - headerName: 'LocationName', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'string', - editable: true - }, - { - field: 'countryName', - headerName: 'CountryName', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'string', - editable: true - }, - { - field: 'dimensionX', - headerName: 'DimX', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'number', - valueFormatter: (value: any) => { - return Number(value).toFixed(2); - }, - editable: true - }, - { - field: 'dimensionY', - headerName: 'DimY', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'number', - valueFormatter: (value: any) => { - return Number(value).toFixed(2); - }, - editable: true - }, - { - field: 'area', - headerName: 'Area', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'number', - valueFormatter: (value: any) => { - return Number(value).toFixed(2); - }, - editable: true - }, - { - field: 'globalX', - headerName: 'GlobalX', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'number', - valueFormatter: (value: any) => { - return Number(value).toFixed(2); - }, - editable: true - }, - { - field: 'globalY', - headerName: 'GlobalY', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'number', - valueFormatter: (value: any) => { - return Number(value).toFixed(2); - }, - editable: true - }, - { - field: 'globalZ', - headerName: 'GlobalZ', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'number', - valueFormatter: (value: any) => { - return Number(value).toFixed(2); - }, - editable: true - }, - { - field: 'unit', - headerName: 'Units', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'singleSelect', - valueOptions: unitSelectionOptions - }, - { - field: 'plotShape', - headerName: 'PlotShape', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'string', - editable: true - }, - { - field: 'plotDescription', - headerName: 'PlotDescription', - headerClassName: 'header', - flex: 1, - align: 'left', - type: 'string', - editable: true - } -]; - -export const GenusGridColumns: GridColDef[] = [ - { - field: 'id', - headerName: '#', - headerClassName: 'header', - flex: 0.3, - align: 'right', - headerAlign: 'right', - editable: false - }, - { field: 'genusID', headerName: 'GenusID', headerClassName: 'header', flex: 1, align: 'left', editable: false }, - { field: 'familyID', headerName: 'FamilyID', headerClassName: 'header', flex: 1, align: 'left', editable: false }, - { field: 'genus', headerName: 'GenusName', headerClassName: 'header', flex: 1, align: 'left', editable: true }, - { - field: 'referenceID', - headerName: 'ReferenceID', - headerClassName: 'header', - flex: 1, - align: 'left', - editable: false - }, - { - field: 'genusAuthority', - headerName: 'Authority', - headerClassName: 'header', - flex: 1, - align: 'left', - editable: true - } -]; - -export const FamilyGridColumns: GridColDef[] = [ - { - field: 'id', - headerName: '#', - headerClassName: 'header', - flex: 0.3, - align: 'right', - headerAlign: 'right', - editable: false - }, - { field: 'familyID', headerName: 'FamilyID', headerClassName: 'header', flex: 1, align: 'left', editable: false }, - { field: 'family', headerName: 'Family', headerClassName: 'header', flex: 1, align: 'left', editable: false }, - { - field: 'referenceID', - headerName: 'ReferenceID', - headerClassName: 'header', - flex: 1, - align: 'left', - editable: false - } -]; -export const CMVErrorGridColumns: GridColDef[] = [ - { - field: 'id', - headerName: '#', - headerClassName: 'header', - flex: 0.3, - align: 'right', - headerAlign: 'right', - editable: false - }, - { field: 'cmvErrorID', headerName: 'CMVErrorID', headerClassName: 'header', flex: 1, align: 'left' }, - { field: 'coreMeasurementID', headerName: 'CoreMeasurementID', headerClassName: 'header', flex: 1, align: 'left' }, - { field: 'validationErrorID', headerName: 'ValidationErrorID', headerClassName: 'header', flex: 1, align: 'left' } -]; - -export const CMAttributeGridColumns: GridColDef[] = [ - { - field: 'id', - headerName: '#', - headerClassName: 'header', - flex: 0.3, - align: 'right', - headerAlign: 'right', - editable: false - }, - { field: 'cmaID', headerName: 'CMAID', headerClassName: 'header', flex: 1, align: 'left' }, - { field: 'coreMeasurementID', headerName: 'CoreMeasurementID', headerClassName: 'header', flex: 1, align: 'left' }, - { field: 'code', headerName: 'Code', headerClassName: 'header', flex: 1, align: 'left' } -]; - -// Combine the column definitions -const combineColumns = (primary: GridColDef[], secondary: GridColDef[]): GridColDef[] => { - const combined = [...primary]; +// Combine the column definitions +const combineColumns = (primary: GridColDef[], secondary: GridColDef[]): GridColDef[] => { + const combined = [...primary]; secondary.forEach(secondaryColumn => { const primaryColumnIndex = primary.findIndex(primaryColumn => primaryColumn.field === secondaryColumn.field); @@ -1476,177 +828,4 @@ export const ViewFullTableGridColumns = rawColumns.map(column => { return column; }); -export const ValidationProceduresGridColumns: GridColDef[] = [ - { field: 'id', headerName: 'ID', headerClassName: 'header' }, - { field: 'validationID', headerName: '#', headerClassName: 'header' }, - { - field: 'procedureName', - headerName: 'Procedure', - headerClassName: 'header', - type: 'string', - editable: true, - flex: 1, - renderCell: (params: GridRenderEditCellParams) => { - const value = params.row.procedureName.replace(/(DBH|HOM)([A-Z])/g, '$1 $2').replace(/([a-z])([A-Z])/g, '$1 $2'); - return {value}; - } - }, - { - field: 'description', - headerName: 'Description', - headerClassName: 'header', - type: 'string', - editable: true, - flex: 1, - renderCell: (params: GridRenderEditCellParams) => { - return {params.row.description}; - } - }, - { - field: 'definition', - headerName: 'SQL Implementation', - headerClassName: 'header', - type: 'string', - editable: true, - flex: 1, - renderCell: (params: GridRenderEditCellParams) => { - const { data: session } = useSession(); - let isEditing = false; - if (typeof params.id === 'string') { - isEditing = params.rowModesModel[parseInt(params.id)]?.mode === 'edit'; - } - const isAdmin = session?.user?.userStatus === 'db admin' || session?.user?.userStatus === 'global'; - - if (isEditing && isAdmin) { - return ( - { - // Update the grid row with the new value from CodeMirror - params.api.updateRows([{ ...params.row, definition: value }]); - }} - /> - ); - } - - return ( - - - - - - - {params.row.description} - - - {params.row.definition} - - - - ); - } - }, - { - field: 'createdAt', - headerName: 'Created At', - renderHeader: () => formatHeader('Created', 'At'), - type: 'date', - headerClassName: 'header', - headerAlign: 'center', - valueGetter: (params: any) => { - if (!params || !params.value) return null; - return new Date(params.value); - }, - editable: true, - flex: 0.4 - }, - { - field: 'updatedAt', - headerName: 'Updated At', - renderHeader: () => formatHeader('Updated', 'At'), - type: 'date', - headerClassName: 'header', - headerAlign: 'center', - valueGetter: (params: any) => { - if (!params || !params.value) return null; - return new Date(params.value); - }, - editable: true, - flex: 0.4 - }, - { field: 'isEnabled', headerName: 'Active?', headerClassName: 'header', type: 'boolean', editable: true, flex: 0.2 } -]; - -export const SiteSpecificValidationsGridColumns: GridColDef[] = [ - { field: 'id', headerName: 'ID', headerClassName: 'header' }, - { field: 'validationProcedureID', headerName: '#', headerClassName: 'header' }, - { - field: 'name', - headerName: 'Procedure', - headerClassName: 'header', - type: 'string', - editable: true, - flex: 1, - renderCell: (params: GridRenderEditCellParams) => { - const value = params.row.procedureName.replace(/(DBH|HOM)([A-Z])/g, '$1 $2').replace(/([a-z])([A-Z])/g, '$1 $2'); - return {value}; - } - }, - { - field: 'description', - headerName: 'Description', - headerClassName: 'header', - type: 'string', - editable: true, - flex: 1, - renderCell: (params: GridRenderEditCellParams) => { - return {params.row.description}; - } - }, - { - field: 'definition', - headerName: 'SQL Implementation', - headerClassName: 'header', - type: 'string', - editable: true, - flex: 1, - renderCell: (params: GridRenderEditCellParams) => { - return ( - - - - - - - {params.row.description} - - - {params.row.description} - - - - ); - } - }, - { field: 'isEnabled', headerName: 'Active?', headerClassName: 'header', type: 'boolean', editable: true, flex: 0.2 } -]; +// FORM GRID COLUMNS diff --git a/frontend/components/client/formcolumns.tsx b/frontend/components/client/formcolumns.tsx new file mode 100644 index 00000000..254beb53 --- /dev/null +++ b/frontend/components/client/formcolumns.tsx @@ -0,0 +1,734 @@ +'use client'; + +import { GridColDef, GridRenderEditCellParams, useGridApiContext } from '@mui/x-data-grid'; +import { areaSelectionOptions, unitSelectionOptions } from '@/config/macros'; +import { formatHeader } from '@/components/client/datagridcolumns'; +import moment from 'moment/moment'; +import { Box, Input, Tooltip } from '@mui/joy'; +import { DatePicker } from '@mui/x-date-pickers'; +import React, { useEffect, useRef, useState } from 'react'; +import useEnhancedEffect from '@mui/utils/useEnhancedEffect'; +import { AttributeStatusOptions } from '@/config/sqlrdsdefinitions/core'; +import { styled } from '@mui/joy/styles'; +import { CheckCircleOutlined } from '@mui/icons-material'; + +export const renderDatePicker = (params: GridRenderEditCellParams) => { + const convertedValue = params.row.date ? moment(params.row.date, 'YYYY-MM-DD') : null; + if (!convertedValue) return <>; + + return ( + + + + ); +}; + +export const renderEditDatePicker = (params: GridRenderEditCellParams) => { + const apiRef = useGridApiContext(); + const { id, row } = params; + + return ( + + { + apiRef.current.setEditCellValue({ id, field: 'date', value: newValue ? newValue.format('YYYY-MM-DD') : null }); + }} + /> + + ); +}; + +const getClosestAreaUnit = (input: string): string | null => { + const normalizedInput = input.trim().toLowerCase(); + + // Define threshold for acceptable "closeness" (tune this value) + const threshold = 2; + + let closestUnit: string | null = null; + let minDistance = Infinity; + + for (const option of areaSelectionOptions) { + const distance = levenshteinDistance(normalizedInput, option); + if (distance < minDistance && distance <= threshold) { + minDistance = distance; + closestUnit = option; + } + } + + // Return the closest match if within the acceptable threshold, otherwise return null + return closestUnit; +}; + +export const EditUnitsCell = (params: GridRenderEditCellParams & { fieldName: string; isArea: boolean }) => { + const apiRef = useGridApiContext(); + const { id, fieldName, hasFocus, isArea } = params; + const [value, setValue] = useState(params.row[fieldName]); + const [error, setError] = useState(false); + const ref = useRef(null); + + useEnhancedEffect(() => { + if (hasFocus && ref.current) { + const input = ref.current.querySelector(`input[value="${value}"]`); + input?.focus(); + } + }, [hasFocus, value]); + + useEffect(() => { + if (!(apiRef.current.getCellMode(id, fieldName) === 'edit')) { + apiRef.current.startCellEditMode({ id, field: fieldName }); + } + }, [apiRef, id, fieldName]); + + useEffect(() => { + setError(!(isArea ? getClosestAreaUnit(value) : getClosestUnit(value))); + }, [value]); + + const handleCommit = () => { + const isValid = isArea ? getClosestAreaUnit(value) : getClosestUnit(value); + + if (!isValid) { + apiRef.current.setEditCellValue({ + id, + field: fieldName, + value: '' + }); + return; + } + + apiRef.current.stopCellEditMode({ id, field: fieldName }); + }; + + return ( + + setValue(e.target.value)} + onBlur={() => { + apiRef.current.setEditCellValue({ + id, + field: fieldName, + value: (isArea ? getClosestAreaUnit(value) : getClosestUnit(value)) || value + }); + handleCommit(); + }} + onKeyDown={e => { + if (e.key === 'Enter') { + apiRef.current.setEditCellValue({ + id, + field: fieldName, + value: (isArea ? getClosestAreaUnit(value) : getClosestUnit(value)) || value + }); + handleCommit(); + } + }} + error={error} + /> + + ); +}; + +const getClosestUnit = (input: string): string | null => { + const normalizedInput = input.trim().toLowerCase(); + + // Define threshold for acceptable "closeness" (tune this value) + const threshold = 2; + + let closestUnit: string | null = null; + let minDistance = Infinity; + + for (const option of unitSelectionOptions) { + const distance = levenshteinDistance(normalizedInput, option); + if (distance < minDistance && distance <= threshold) { + minDistance = distance; + closestUnit = option; + } + } + + // Return the closest match if within the acceptable threshold, otherwise return null + return closestUnit; +}; + +function levenshteinDistance(a: string, b: string): number { + const matrix = Array.from({ length: a.length + 1 }, () => Array(b.length + 1).fill(0)); + + for (let i = 0; i <= a.length; i++) matrix[i][0] = i; + for (let j = 0; j <= b.length; j++) matrix[0][j] = j; + + for (let i = 1; i <= a.length; i++) { + for (let j = 1; j <= b.length; j++) { + const cost = a[i - 1] === b[j - 1] ? 0 : 1; + matrix[i][j] = Math.min( + matrix[i - 1][j] + 1, // Deletion + matrix[i][j - 1] + 1, // Insertion + matrix[i - 1][j - 1] + cost // Substitution + ); + } + } + return matrix[a.length][b.length]; +} + +function normalizeString(str: string): string { + return str.replace(/[\s-]+/g, '').toLowerCase(); +} + +const getClosestStatus = (input: string): string | null => { + const normalizedInput = normalizeString(input); + + // Define threshold for acceptable "closeness" (tune this value) + const threshold = 2; + + let closestStatus: string | null = null; + let minDistance = Infinity; + + for (const option of AttributeStatusOptions) { + const normalizedOption = normalizeString(option); + const distance = levenshteinDistance(normalizedInput, normalizedOption); + if (distance < minDistance && distance <= threshold) { + minDistance = distance; + closestStatus = option; // Return the original option, not the normalized one + } + } + + // Return the closest match if within the acceptable threshold, otherwise return null + return closestStatus; +}; + +const StyledInput = styled('input')({ + border: 'none', + minWidth: 0, + outline: 0, + padding: 0, + paddingTop: '1em', + flex: 1, + color: 'inherit', + backgroundColor: 'transparent', + fontFamily: 'inherit', + fontSize: 'inherit', + fontStyle: 'inherit', + fontWeight: 'inherit', + lineHeight: 'inherit', + textOverflow: 'ellipsis', + '&::placeholder': { + opacity: 0, + transition: '0.1s ease-out' + }, + '&:focus::placeholder': { + opacity: 1 + }, + '&:focus ~ label, &:not(:placeholder-shown) ~ label, &:-webkit-autofill ~ label': { + top: '0.5rem', + fontSize: '0.75rem' + }, + '&:focus ~ label': { + color: 'var(--Input-focusedHighlight)' + }, + '&:-webkit-autofill': { + alignSelf: 'stretch' + }, + '&:-webkit-autofill:not(* + &)': { + marginInlineStart: 'calc(-1 * var(--Input-paddingInline))', + paddingInlineStart: 'var(--Input-paddingInline)', + borderTopLeftRadius: 'calc(var(--Input-radius) - var(--variant-borderWidth, 0px))', + borderBottomLeftRadius: 'calc(var(--Input-radius) - var(--variant-borderWidth, 0px))' + } +}); + +const StyledLabel = styled('label')(({ theme }) => ({ + position: 'absolute', + lineHeight: 1, + top: 'calc((var(--Input-minHeight) - 1em) / 2)', + color: theme.vars.palette.text.tertiary, + fontWeight: theme.vars.fontWeight.md, + transition: 'all 150ms cubic-bezier(0.4, 0, 0.2, 1)' +})); + +const InnerInput = React.forwardRef< + HTMLInputElement, + React.JSX.IntrinsicElements['input'] & { + error?: boolean; + noInput?: boolean; + } +>(function InnerInput(props, ref) { + const { error, noInput, ...rest } = props; + const id = React.useId(); + + return ( + + + {noInput ? AttributeStatusOptions.join(', ') : error ? 'Invalid status' : 'Accepted!'} + + ); +}); + +const EditStatusCell = (params: GridRenderEditCellParams) => { + const apiRef = useGridApiContext(); + const { id, hasFocus } = params; + const [value, setValue] = React.useState(params.row['status']); + const [error, setError] = React.useState(false); + const ref = React.useRef(null); + + useEnhancedEffect(() => { + if (hasFocus && ref.current) { + const input = ref.current.querySelector(`input[value="${value}"]`); + input?.focus(); + } + }, [hasFocus, value]); + + React.useEffect(() => { + if (!(apiRef.current.getCellMode(id, 'status') === 'edit')) { + apiRef.current.startCellEditMode({ id, field: 'status' }); + } + }, [apiRef, id]); + + React.useEffect(() => { + setError(!getClosestStatus(value) && value !== ''); + }, [value]); + + const handleCommit = () => { + const correctedValue = getClosestStatus(value); + + console.log('handle commit: corrected value: ', correctedValue); + + apiRef.current.setEditCellValue({ + id, + field: 'status', + value: value + }); + + apiRef.current.stopCellEditMode({ id, field: 'status' }); + }; + + return ( + } + slots={{ input: InnerInput }} + slotProps={{ + input: { + placeholder: 'Enter status...', + type: 'text', + error, + noInput: value === '' + } + }} + sx={{ '--Input-minHeight': '56px', '--Input-radius': '6px' }} + onChange={e => setValue(e.target.value)} + onBlur={handleCommit} + onKeyDown={e => { + if (e.key === 'Enter' || e.key === 'Tab') { + console.log('on key down: enter || tab'); + handleCommit(); + } + }} + error={error} + /> + ); +}; + +export const AttributesFormGridColumns: GridColDef[] = [ + { + field: 'id', + headerName: '#', + headerClassName: 'header', + flex: 0.3, + headerAlign: 'right', + editable: false + }, + { + field: 'code', + headerName: 'Code', + headerClassName: 'header', + flex: 1, + editable: true + }, + { + field: 'description', + headerName: 'Description', + headerClassName: 'header', + flex: 1, + editable: true + }, + { + field: 'status', + headerName: 'Status', + headerClassName: 'header', + flex: 1, + editable: true + // This is temporarily being suspended -- it's a nice to have, not a need to have + // renderEditCell: params => + } +]; + +export const PersonnelFormGridColumns: GridColDef[] = [ + { + field: 'id', + headerName: '#', + headerClassName: 'header', + flex: 0.3, + align: 'right', + headerAlign: 'right', + editable: false + }, + { + field: 'firstname', + headerName: 'First Name', + headerClassName: 'header', + flex: 1, + align: 'left', + editable: true + }, + { + field: 'lastname', + headerName: 'Last Name', + headerClassName: 'header', + flex: 1, + align: 'left', + editable: true + }, + { + field: 'role', + headerName: 'Role', + headerClassName: 'header', + flex: 1, + align: 'left', + editable: true + }, + { + field: 'roledescription', + headerName: 'Role Description', + headerClassName: 'header', + flex: 1, + align: 'left', + editable: true + } +]; + +export const SpeciesFormGridColumns: GridColDef[] = [ + { + field: 'id', + headerName: '#', + headerClassName: 'header', + flex: 0.3, + align: 'right', + headerAlign: 'right', + editable: false + }, + { + field: 'spcode', + headerName: 'Species Code', + headerClassName: 'header', + flex: 1, + align: 'left', + editable: true + }, + { + field: 'family', + headerName: 'Family', + headerClassName: 'header', + flex: 1, + align: 'left', + editable: true + }, + { + field: 'genus', + headerName: 'Genus', + headerClassName: 'header', + flex: 1, + align: 'left', + editable: true + }, + { + field: 'species', + headerName: 'Species', + headerClassName: 'header', + flex: 1, + align: 'left', + editable: true + }, + { + field: 'subspecies', + headerName: 'Subspecies', + headerClassName: 'header', + flex: 1, + align: 'left', + editable: true + }, + { + field: 'idlevel', + headerName: 'ID Level', + headerClassName: 'header', + flex: 1, + align: 'left', + editable: true + }, + { + field: 'authority', + headerName: 'Authority', + headerClassName: 'header', + flex: 1, + align: 'left', + editable: true + }, + { + field: 'subspeciesauthority', + headerName: 'Subspecies Authority', + headerClassName: 'header', + flex: 1, + align: 'left', + editable: true + } +]; + +export const QuadratsFormGridColumns: GridColDef[] = [ + { + field: 'id', + headerName: '#', + headerClassName: 'header', + flex: 0.3, + align: 'right', + headerAlign: 'right', + editable: false + }, + { + field: 'quadrat', + headerName: 'Quadrat Name', + headerClassName: 'header', + flex: 1, + align: 'left', + editable: true + }, + { + field: 'startx', + headerName: 'StartX', + headerClassName: 'header', + flex: 1, + align: 'left', + type: 'number', + editable: true + }, + { + field: 'starty', + headerName: 'StartY', + headerClassName: 'header', + flex: 1, + align: 'left', + type: 'number', + editable: true + }, + { + field: 'coordinateunit', + headerName: 'Coordinate Units', + headerClassName: 'header', + align: 'left', + editable: true, + renderEditCell: params => + }, + { + field: 'dimx', + headerName: 'Dimension X', + headerClassName: 'header', + flex: 1, + align: 'left', + type: 'number', + editable: true + }, + { + field: 'dimy', + headerName: 'Dimension Y', + headerClassName: 'header', + flex: 1, + align: 'left', + type: 'number', + editable: true + }, + { + field: 'dimensionunit', + headerName: 'Dimension Units', + headerClassName: 'header', + flex: 0.3, + align: 'left', + editable: true, + renderEditCell: params => + }, + { + field: 'area', + headerName: 'Area', + headerClassName: 'header', + flex: 1, + align: 'left', + type: 'number', + editable: true + }, + { + field: 'areaunit', + headerName: 'Area Units', + headerClassName: 'header', + flex: 0.3, + align: 'left', + editable: true, + renderEditCell: params => + }, + { + field: 'quadratshape', + headerName: 'Quadrat Shape', + headerClassName: 'header', + flex: 1, + align: 'left', + editable: true + } +]; +/** + * [FormType.measurements]: [ + * { label: 'tag' }, + * { label: 'stemtag' }, + * { label: 'spcode' }, + * { label: 'quadrat' }, + * { label: 'lx' }, + * { label: 'ly' }, + * { label: 'coordinateunit' }, + * { label: 'dbh' }, + * { label: 'dbhunit' }, + * { label: 'hom' }, + * { label: 'homunit' }, + * { label: 'date' }, + * { label: 'codes' } + * ], + */ +export const MeasurementsFormGridColumns: GridColDef[] = [ + { + field: 'id', + headerName: '#', + headerClassName: 'header', + flex: 0.3, + align: 'right', + headerAlign: 'right', + editable: false + }, + { + field: 'tag', + headerName: 'Tree Tag', + headerClassName: 'header', + renderHeader: () => formatHeader('Tree', 'Tag'), + flex: 0.75, + align: 'center', + editable: true + }, + { + field: 'stemtag', + headerName: 'Stem Tag', + headerClassName: 'header', + renderHeader: () => formatHeader('Stem', 'Tag'), + flex: 0.75, + align: 'center', + editable: true + }, + { + field: 'spcode', + headerName: 'Species Code', + headerClassName: 'header', + renderHeader: () => formatHeader('Species', 'Code'), + flex: 0.75, + align: 'center', + editable: true + }, + { + field: 'quadrat', + headerName: 'Quadrat Name', + headerClassName: 'header', + renderHeader: () => formatHeader('Quadrat', 'Name'), + flex: 0.75, + align: 'center', + editable: true + }, + { + field: 'lx', + headerName: 'X', + headerClassName: 'header', + flex: 0.3, + align: 'center', + type: 'number', + editable: true + }, + { + field: 'ly', + headerName: 'Y', + headerClassName: 'header', + flex: 0.3, + align: 'center', + type: 'number', + editable: true + }, + { + field: 'coordinateunit', + headerName: '<= Units', + headerClassName: 'header', + // renderHeader: () => formatHeader('Coordinate', 'Units'), + flex: 0.5, + align: 'center', + editable: true, + renderEditCell: params => + }, + { + field: 'dbh', + headerName: 'DBH', + headerClassName: 'header', + flex: 0.75, + align: 'center', + type: 'number', + editable: true + }, + { + field: 'dbhunit', + headerName: '<= Units', + headerClassName: 'header', + // renderHeader: () => formatHeader('DBH', 'Units'), + flex: 0.5, + align: 'center', + editable: true, + renderEditCell: params => + }, + { + field: 'hom', + headerName: 'HOM', + headerClassName: 'header', + flex: 0.75, + align: 'center', + type: 'number', + editable: true + }, + { + field: 'homunit', + headerName: '<= Units', + headerClassName: 'header', + // renderHeader: () => formatHeader('HOM', 'Units'), + flex: 0.5, + align: 'center', + editable: true, + renderEditCell: params => + } +]; diff --git a/frontend/components/client/githubfeedbackmodal.tsx b/frontend/components/client/githubfeedbackmodal.tsx index 19d7dd6d..57d3751f 100644 --- a/frontend/components/client/githubfeedbackmodal.tsx +++ b/frontend/components/client/githubfeedbackmodal.tsx @@ -12,7 +12,6 @@ import { Divider, FormControl, FormLabel, - Grid, Input, LinearProgress, List, @@ -36,6 +35,7 @@ import { usePathname } from 'next/navigation'; import { useSession } from 'next-auth/react'; import ReactMarkdown from 'react-markdown'; import remarkGfm from 'remark-gfm'; +import Grid from '@mui/material/Grid2'; // this has been shelved -- it's a little too complicated for a first iteration. // saving it for a later version. @@ -185,7 +185,7 @@ ${pathname} - + {currentSite ? ( @@ -197,7 +197,7 @@ ${pathname} No site selected. )} - + {currentPlot ? ( Selected Plot: {currentPlot.plotName} @@ -207,7 +207,7 @@ ${pathname} No plot selected. )} - + {currentCensus ? ( Selected Census: {currentCensus.plotCensusNumber} diff --git a/frontend/components/client/postvalidationrow.tsx b/frontend/components/client/postvalidationrow.tsx new file mode 100644 index 00000000..4cf11919 --- /dev/null +++ b/frontend/components/client/postvalidationrow.tsx @@ -0,0 +1,218 @@ +'use client'; +import React from 'react'; +import { Box, Collapse, TableCell, TableRow, Typography } from '@mui/material'; +import ExpandMoreIcon from '@mui/icons-material/ExpandMore'; +import ExpandLessIcon from '@mui/icons-material/ExpandLess'; +import { PostValidationQueriesRDS } from '@/config/sqlrdsdefinitions/validations'; +import { Checkbox, IconButton, Textarea, Tooltip } from '@mui/joy'; +import { Done } from '@mui/icons-material'; +import dynamic from 'next/dynamic'; +import moment from 'moment/moment'; +import { darken } from '@mui/system'; + +interface PostValidationRowProps { + postValidation: PostValidationQueriesRDS; + selectedResults: PostValidationQueriesRDS[]; + expanded: boolean; + isDarkMode: boolean; + expandedQuery: number | null; + replacements: { schema: string | undefined; currentPlotID: number | undefined; currentCensusID: number | undefined }; + handleExpandClick: (queryID: number) => void; + handleExpandResultsClick: (queryID: number) => void; + handleSelectResult: (postValidation: PostValidationQueriesRDS) => void; +} + +const Editor = dynamic(() => import('@monaco-editor/react'), { ssr: false }); + +const PostValidationRow: React.FC = ({ + expandedQuery, + replacements, + postValidation, + expanded, + isDarkMode, + handleExpandClick, + handleExpandResultsClick, + handleSelectResult, + selectedResults +}) => { + const formattedResults = JSON.stringify(JSON.parse(postValidation.lastRunResult ?? '{}'), null, 2); + + const successColor = !isDarkMode ? 'rgba(54, 163, 46, 0.3)' : darken('rgba(54,163,46,0.6)', 0.7); + const failureColor = !isDarkMode ? 'rgba(255, 0, 0, 0.3)' : darken('rgba(255,0,0,0.6)', 0.7); + + return ( + <> + + + handleExpandResultsClick(postValidation.queryID!)} + > + {expanded ? : } + + + handleSelectResult(postValidation)} style={{ cursor: 'pointer', padding: '0', textAlign: 'center' }}> + + } + label={''} + checked={selectedResults.includes(postValidation)} + slotProps={{ + root: ({ checked, focusVisible }) => ({ + sx: !checked + ? { + '& svg': { opacity: focusVisible ? 1 : 0 }, + '&:hover svg': { + opacity: 1 + } + } + : undefined + }) + }} + onChange={e => e.stopPropagation()} + /> + + + + {postValidation.queryName} + + + + {expandedQuery === postValidation.queryID ? ( + + String(replacements[p1 as keyof typeof replacements] ?? '') + )} + options={{ + readOnly: true, + minimap: { enabled: false }, + scrollBeyondLastLine: false, + wordWrap: 'off', + lineNumbers: 'off' + }} + theme={isDarkMode ? 'vs-dark' : 'light'} + /> + ) : ( +