[WIP] HumanOps - annotation #617
Conversation
|
ongoing work
|
…icient polling; fix double polling
| class HumanPreferenceAnnotationMapper(LabelStudioAnnotationMapper): | ||
| """Operator for human preference annotation using Label Studio.""" | ||
|
|
||
| DEFAULT_LABEL_CONFIG = """ |
There was a problem hiding this comment.
xml in code. could be a separate file if needed. supported via label_config_file param
There was a problem hiding this comment.
I recommend make this part binding to XxAnnotationMapper, to enable the feasibility of automatic adaptation in a ``MCP'' style (e.g., some agents that learn to route and modify the label_cfg given the HumanOP docstring)
There was a problem hiding this comment.
Can you elaborate? A sample would be great
|
getting below message: tracer only checks text_key field. we are actually adding more data. what is the best resolution? should we pack all the data into text_key as json dump? @HYLcool @yxdyc |
| class HumanPreferenceAnnotationMapper(LabelStudioAnnotationMapper): | ||
| """Operator for human preference annotation using Label Studio.""" | ||
|
|
||
| DEFAULT_LABEL_CONFIG = """ |
There was a problem hiding this comment.
I recommend make this part binding to XxAnnotationMapper, to enable the feasibility of automatic adaptation in a ``MCP'' style (e.g., some agents that learn to route and modify the label_cfg given the HumanOP docstring)
After discussion, we agree to packing all the data into text_key as json dump, and will make a quick PR to support this feature @HYLcool |
…ity and download logic for upgrading nltk
This feature is to add human annotation support for DJ. Included features:
still working on