Skip to content

Structured JSON output + other minor fixes #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 15 commits into
base: master
Choose a base branch
from

Conversation

lfoppiano
Copy link

@lfoppiano lfoppiano commented Sep 29, 2023

In this PR I've added the JSON output that provide a structured output format:

{
  "question": {
    "title": "In which language should the questions be posted? Portuguese or English?",
    "body": "In which language should the questions be posted? Portuguese or English?\nIf a question is posted in a language different from the expected, what should we do? Flag to close or translate and edit?\n"
  },
  "answers": [
    {
      "id": "5",
      "body": "I think all contributions (questions, answers and comments) should be allowed in portuguese and\/or english.\nIn the case of a question or an answer, you can always edit it to add a translation. In the case of a comment you can not edit to translate it, but the comments that add value to a question or to an answer could be included in such question or answer, using both languages if appropriate.\n",
      "score": 13
    }
  ]
}

In fact this format is used a model and it's serialised in different ways depending on the output format.

There is also an additional flag --keep-sources that will not delete the 7z files in the dump directory

@lfoppiano lfoppiano changed the title Structured json output Structured JSON output + other minor fixes Sep 29, 2023
@lfoppiano
Copy link
Author

This change requires the latest version of https://github.com/lfoppiano/lm_dataformat

@lfoppiano lfoppiano marked this pull request as draft September 29, 2023 02:41
@lfoppiano lfoppiano marked this pull request as ready for review September 29, 2023 03:10
@lfoppiano
Copy link
Author

This requires updating lm_datasource as well, which currently is required in edit more

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant