1492: Add bulk import from freinet #1541

ztefanie · 2024-07-23T08:21:28Z

Short description

Adding bulk import csv option from freinet csv format

Proposed changes

Added new route "import/?freinet=true"
Added project config setting to enable freinet import
Add function that adjusted freinet csvs to expected csv input format
Add button to import from freinet
Tests are missing, I will create a ticket for them if i will not manage to add some this week before my vacation

Side effects

Maybe standart csv import is somehow affected

Resolved issues

Fixes: #1492

f1sh1918

Added some comments.
In general it think we should have an error handling if the column names do not exist
If the column names in the header (f.e. vorname, nachname) do not match our format, i just get undefined name but can create cards.

administration/src/bp-modules/cards/ImportCardsFromFreinetController.tsx

f1sh1918 · 2024-07-23T11:04:39Z

administration/src/bp-modules/cards/ImportCardsFromFreinetController.tsx

+    csvHeader[csvHeader.length] = nameCoulmnName
+  }
+
+  const indexVorname = csvHeader.indexOf(FREINET_FIRSTNAME_COLUMN_HEADER)


maybe we should add some error handling if columns do not exists

administration/src/bp-modules/cards/ImportCardsFromFreinetController.tsx

# Conflicts: # administration/src/project-configs/bayern/config.ts # administration/src/project-configs/getProjectConfig.ts # administration/src/project-configs/nuernberg/config.ts # administration/src/project-configs/showcase/config.ts

f1sh1918 · 2024-08-19T11:48:07Z

@ztefanie
i used the import files in the mentioned issue. But i get all these empty lines, caused by ;

A second (optional) thing could be to set a null for the columnNames mail extension (my issue)
extensionColumnNames: ['Kartentyp', null, null], to not show the mail notification column
We could think about providing this as a feature that they could export the mail address of the user and send the mail notification also for bulk imports. But i think we didn't make a decision on this

ztefanie · 2024-08-20T06:11:23Z

@ztefanie i used the import files in the mentioned issue. But i get all these empty lines, caused by ;

A second (optional) thing could be to set a null for the columnNames mail extension (my issue) extensionColumnNames: ['Kartentyp', null, null], to not show the mail notification column We could think about providing this as a feature that they could export the mail address of the user and send the mail notification also for bulk imports. But i think we didn't make a decision on this

Yes both is as it should be.
In the example input are empty lines and they should be displayed like this.
About the email notification, we can do this if they request it at a later point.

f1sh1918

Alright works as expected 👍

For later testing i think it could help to also provide a working csv example.
Sorry i was a bit confused while testing it ^^

administration/src/Router.tsx

administration/src/bp-modules/cards/CreateCardsController.tsx

administration/src/bp-modules/cards/ImportCardsController.tsx

administration/src/bp-modules/cards/ImportCardsFromFreinetController.ts

steffenkleinle · 2024-08-21T13:05:52Z

administration/src/bp-modules/cards/ImportCardsFromFreinetController.ts

+const FREINET_LASTNAME_COLUMN_HEADER = 'nachname'
+const FREINET_CARDTYPE_COLUMN_HEADER = 'inhaber_ehrenamtskarte'
+
+const mergeFirstAndLastnameIntoNewColumn = (line: string[], csvHeader: string[], nameColumnName: string) => {


🙃 Is all of this done in place as no copies or anything are returned? Wouldn't it be a lot safer to work with copies? Looks kindof error prone to me.
E.g. if the header is adjusted in an early iteration, doesn't that have implications for later iterations?

Yes it does impact later iterations, but this is handled and works as expected. You are right it may be not the best implementation, but we are not planning to have this feature for a ever or a long time, it is just a workaround, we will remove this as soon as Freinet and entitlemtcard are connected.

administration/src/bp-modules/cards/ImportCardsFromFreinetController.ts

Co-authored-by: Steffen Kleinle <[email protected]>

steffenkleinle

Tested on firefox. I can open the CSV you linked in the issue (#1492 (comment)), but it looks like this:

So it checked that there are invalid values and therefore blocks creating the QR codes, but I am not able to change the names or anything? Am I doing something wrong?

If this is supposed to happen in another issue/PR, it works as expected!

Tests are missing, I will create a ticket for them if i will not manage to add some this week before my vacation

Did you manage to do this?

steffenkleinle · 2024-08-26T10:45:21Z

And perhaps you could add the csv here too? https://github.com/digitalfabrik/entitlementcard/tree/main/administration/resources/cards

f1sh1918 · 2024-08-26T11:41:54Z

Tested on firefox. I can open the CSV you linked in the issue (#1492 (comment)), but it looks like this:

So it checked that there are invalid values and therefore blocks creating the QR codes, but I am not able to change the names or anything? Am I doing something wrong?

If this is supposed to happen in another issue/PR, it works as expected!

Tests are missing, I will create a ticket for them if i will not manage to add some this week before my vacation

Did you manage to do this?

this is expected. just remove the ";" from the csv and create a card :) @steffenkleinle

…from-freinet # Conflicts: # administration/src/bp-modules/cards/ImportCardsInput.tsx

f1sh1918 · 2024-08-26T13:09:29Z

resolved merge conflicts and merge @ztefanie to put it in the next release

ztefanie requested review from steffenkleinle, f1sh1918 and seluianova as code owners July 23, 2024 08:21

ztefanie force-pushed the 1492-add-bulk-import-from-freinet branch from 550c2f6 to 3584167 Compare July 23, 2024 09:05

1492: Add bulk import from freinet

9c3b8f6

ztefanie force-pushed the 1492-add-bulk-import-from-freinet branch from 3584167 to 9c3b8f6 Compare July 23, 2024 09:06

f1sh1918 reviewed Jul 23, 2024

View reviewed changes

ztefanie added 2 commits August 19, 2024 09:55

1492: Add checks for missing columns

056a3d8

ztefanie requested a review from f1sh1918 August 19, 2024 08:05

1492: Add config prop to new koblenz config files

e66ac0e

f1sh1918 approved these changes Aug 20, 2024

View reviewed changes

Add freinet bulk import example csvs

a2ea2a4

steffenkleinle reviewed Aug 21, 2024

View reviewed changes

ztefanie and others added 2 commits August 26, 2024 08:29

Apply suggestions from code review

a7843ae

Co-authored-by: Steffen Kleinle <[email protected]>

1492: Small adjustments

4b56fe5

ztefanie requested a review from steffenkleinle August 26, 2024 07:15

steffenkleinle reviewed Aug 26, 2024

View reviewed changes

steffenkleinle approved these changes Aug 26, 2024

View reviewed changes

Merge remote-tracking branch 'origin/main' into 1492-add-bulk-import-…

6138c26

…from-freinet # Conflicts: # administration/src/bp-modules/cards/ImportCardsInput.tsx

f1sh1918 enabled auto-merge August 26, 2024 13:09

f1sh1918 merged commit 0e9791f into main Aug 26, 2024
1 check passed

f1sh1918 deleted the 1492-add-bulk-import-from-freinet branch August 26, 2024 13:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

1492: Add bulk import from freinet #1541

1492: Add bulk import from freinet #1541

ztefanie commented Jul 23, 2024 •

edited

Loading

f1sh1918 left a comment •

edited

Loading

f1sh1918 Jul 23, 2024

f1sh1918 commented Aug 19, 2024 •

edited

Loading

ztefanie commented Aug 20, 2024

f1sh1918 left a comment

steffenkleinle Aug 21, 2024

ztefanie Aug 26, 2024

steffenkleinle left a comment

steffenkleinle commented Aug 26, 2024

f1sh1918 commented Aug 26, 2024

f1sh1918 commented Aug 26, 2024

1492: Add bulk import from freinet #1541

1492: Add bulk import from freinet #1541

Conversation

ztefanie commented Jul 23, 2024 • edited Loading

Short description

Proposed changes

Side effects

Resolved issues

f1sh1918 left a comment • edited Loading

Choose a reason for hiding this comment

f1sh1918 Jul 23, 2024

Choose a reason for hiding this comment

f1sh1918 commented Aug 19, 2024 • edited Loading

ztefanie commented Aug 20, 2024

f1sh1918 left a comment

Choose a reason for hiding this comment

steffenkleinle Aug 21, 2024

Choose a reason for hiding this comment

ztefanie Aug 26, 2024

Choose a reason for hiding this comment

steffenkleinle left a comment

Choose a reason for hiding this comment

steffenkleinle commented Aug 26, 2024

f1sh1918 commented Aug 26, 2024

f1sh1918 commented Aug 26, 2024

ztefanie commented Jul 23, 2024 •

edited

Loading

f1sh1918 left a comment •

edited

Loading

f1sh1918 commented Aug 19, 2024 •

edited

Loading