[FEAT] Compile all content into one language json file from three separate files

[ngiangre]

⚠️ IMPORTANT: Please fill out this template to give us as much information as possible to consider/implement the feature.

Prerequisites

• check this box if you have completed the following:
  • Reviewed the contributing guidelines and support files
  • Reviewed the README file for the repository you are working in
  • Searched for relevant instructions on our Discord server
  • Searched the issues of the repository you are working in to make sure one was not already filed

Summary

Currently the education, health, and display text content for each language are three separate json files. For the front end end team to work with a particular language they need to have each language be one json file that's referenced in the locale chosen.

Motivation

This is to promote reference by the app to the proper, central language json file

Possible Alternatives

Keeping as is, which is a fragmented approach

Additional Context

This is part of the content continuous integration workflow being developed

[BrianHHough]

Hi @ngiangre JSON file is updated here: https://github.com/COVID-19-electronic-health-system/Corona-tracker/blob/master/Translations/appText.json

cc: @salvolpe @tesla809 @SomeMoosery

[ngiangre]

Extremely detailed @BrianHHough! We can definitely make use of this.

[salvolpe]

Seriously!! Awesome work @BrianHHough

[SomeMoosery]

Undid this change - there's a PR process these things need to go through and we can't commit right to master. Don't want to be dictatorial but this can seriously break things. Will need a PR relating to this issue just like everything else.

[ngiangre]

Yeah agreed thanks @SomeMoosery. But @salvolpe and I can work with this file if @BrianHHough sends it through discord. @SomeMoosery how can we make a json so that wee can reference tags to update components?

[SomeMoosery]

Yeah no prob! The file looks great, I agree - just need to make sure we're going through the git flow so things don't start to fall off track.

I'm also not sure what you mean - what are you trying to update? If you're trying to update components when some value changes that should be done with hooks in the React code.

[ngiangre]

Yeah don't want to obstruct the process at all.

We need to make sure the text is adequately translated dependent on the locale. We want to try to help you make your hooks with the proper content:

I should've first made an issue pertaining to this. @BrianHHough committed to master by accident but it's ok you live and learn.

I need to make a more formal issue and description, but I guess this will do.

The idea is to allow hooks to be made dependent on the locale chosen (a file with the adequate translation will be pointed to for the correct translation). Sorry I'm not sure how much I can explain here. @SomeMoosery what would be the best way to proceed from here?

[SomeMoosery]

Yeah, so it seems like what can be done from this end is that once we get a master doc for each language merged in, we can create a new issue for which we change the locations specified in i18next to point to these new JSON files, and we should be set 👍

[ngiangre]

Ok great. I made a PR that was approved which put language json files in docs/content but now I'm working on another that puts everything in that language from 3 to one json file with the structure needed by front end - I'm working with @pavel-ilin and @AdhamAH on that. But @SomeMoosery if you have any suggestion on a better structure for the json files in docs/content let me know!

[ngiangre]

Closing other issues to have this be the issue with all the information.

#623 (comment)

#359 (comment)

#530 (comment)

We need to determine the structure of each language's json file, where it should go, and how the content should be referenced for tagging in components to fully integrate our translations and have all available content for the coronatracker app.

[tesla809]

Can we make sure that the privacy policies and terms and conditions are also included as all content? If so, this can be within this issue.

[ngiangre]

Yes @tesla809

[AdhamAH]

@BrianHHough
I loved the file! It is clear and will be easy to use i18next
@ngiangre will you use this? or the examples you sent me?
As @SomeMoosery said as soon as the JSON ready we can start working on point the app to use i18next when needed

[ngiangre]

I'm working on it but would love help.

The issue is that we need to have the structure in our data model so However we make the keys is consistent, makes sense, and the values can be adapted in the future.
The data model is here:
https://docs.google.com/spreadsheets/d/1gifZGnj-ntIOXbZrRY4S5KEULoV64DCbi0pZSVWKsns/edit?usp=sharing

Right now only the health sheet follows the data model and has a field name to be included in the tag:
https://docs.google.com/spreadsheets/d/1tjeTH99QZSmyfG-TvX5seSYKa03yAHdMWrZt1z9521I/edit#gid=0

We need to fill in the parent/child keys and add a field name for education:
https://docs.google.com/spreadsheets/d/1OdasOk5y8w9JLv3VWmziV8XkznaYiW-OQzbQq5ZK26M/edit?usp=sharing

And Display Text is incomplete
https://docs.google.com/spreadsheets/d/1wn02oIks4avznMoJfxVHCo5221D9wwfu6yaWpi6QleA/edit?usp=sharing

I need help with the data model - I can handle this but @salvolpe can help and anyone else willing to help please reach out. And once the data model is done I can get the content from translations to populate the value via the python script src/python/pull_gsheets_data.py.

I work a lot with common data models so this is really important.

The goal is to allow for creating tags like demographics.age.question like for the question How old are you? which would be the value for the english translation. The content in the translations gsheets can be manipulated to be the value using src/python/pull_gsheets_data.py.

We need to do this for the education facts in the education gsheet as well as for the display text gsheet.

[pavel-ilin]

@BrianHHough I love your appText.json.

I would suggest modification of the survey section. I would like to store all questions in the array and add question type attribute to every question. This will help us not only with translation but also render all questions dynamically. We can iterate over the array and base on type of the question render components we need. This way we can move away from hardcoding every question.

Example:

I thin we can do the same with quizzes.

@ngiangre @AdhamAH @salvolpe @tesla809 @SomeMoosery What do you think?

[ngiangre]

Great! The structure is do-able. Can you make a sample @pavel-ilin for the education content and display text as well?

[ngiangre]

@pavel-ilin @AdhamAH or anyone on front end - to finish converting Sal and I need to understand what exactly is hard coded and how values are set to defaults so we can accommodate in our data model. Can someone ping me, @nickg, on discord or comment here?

[AdhamAH]

I really still don't fully understand what are you trying to do and why it looks so complicated @ngiangre
Anyway I hope this would help you
https://docs.google.com/document/d/1f8RQFBNfLKEPPp2v1eYf693zU9hK8oEKq8Lb0S2VbFI/edit?usp=sharing

[ngiangre]

This is great @AdhamAH! Let’s chat on discord so we can be on the same page.

[SomeMoosery]

I think just to keep things clear in terms of this issue. The end result here is that we just end up with "master" JSON files for each language, which will be stored in /public/locales. We use i18next's useTranlsation hook, which pulls the currently-selected language from this directory and populates it by using the t(...) function.

There is no hard and fast rule on what's hard-coded and what's not, as every component is different just by nature of building out a responsive web app. This simply requires someone going through each component and looking for where there is hardcoded text, and making sure that text is captured in the master JSON file for translations.

I saw Brian was breaking his WIP master JSON file appText.json in terms of components, which I think is a good idea. For example, onboardSection: { ... } would correspond to Onboarding.jsx, disclaimerSection: { ... } would correspond to Disclaimer.jsx - so on and so forth.

This will be tedious but having someone go through and report back where things are hardcoded and need translating wouldn't save any time, as someone then still has to go through and add those hardcoded portions into the master JSON file, and use the useTranslation hook's t(...) function to translate.

This will likely be an iterative process - we can always create more issues to reformat the master JSON files and subsequently update the translated values in the corresponding components. I think there is too much focus on forming a "data model" when what we really need is just a JSON file in public/locales that we then use to translate hard-coded text throughout all of our components.

I'm starting to fear this is becoming a bit over-engineered, as the principle behind i18next is that it's very easy to add new translations as we continue. It seems like things are taking longer due to the emphasis on finding a perfectly automated process, when we simply just need a JSON file in public/locales and to use t(...) in the corresponding components. At least, that's my understanding from the frontend side of things. Just my thoughts here.

[ngiangre]

Correct @SomeMoosery - one master json for each language that will be within public/locales/

My goal is to get a "MVP" available - we will have every field needed in the gsheets that can then be transformed into the correct json file via python scripting with the google api.

In order to get the right structure, we needed to know how you would call the tags which wasn't clear. Now it is so thanks! You're right we might not be able to get everything, but we want to get most atleast. The over-engineered part might be from detailing everything, so we'll not do that. But we did need to understand the right structure so it's usable. The data model is important so it's consistent - if not in a data model, then iterative processing/modification/adding languages will get really convoluted and 'makeshift' real fast.

So I'm glad we are having this discussion!

[SomeMoosery]

That's a very good point!

Sounds good, since I've been somewhat away for a day or two I was just hoping to add some clarity as somewhat of an "outside" perspective to this translations bit. Seems like it's on the right track though thanks to yours and everyone's work on it, and this is one of the best discussion on an issue I think we've had 😄

[ngiangre]

teamwork makes the dream work!

[ngiangre]

Done. Finishing work in #665