Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data annotators rating #8027

Open
2 tasks done
chernov-a opened this issue Jun 13, 2024 · 0 comments
Open
2 tasks done

Data annotators rating #8027

chernov-a opened this issue Jun 13, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@chernov-a
Copy link
Contributor

Actions before raising this issue

  • I searched the existing issues and did not find anything similar.
  • I read/searched the docs

Is your feature request related to a problem? Please describe.

No response

Describe the solution you'd like

We utilize a fairly large annotation team for the needs of our projects, and the question of each annotator's performance is somewhat subjective. We would like to have specific statistics on each annotator's work, which could be available as a CSV file for download and analysis. In this regard, there is a proposal to develop something like an annotator rating system on the platform, which would be based on the following parameters:

  • Quality of completed tasks

  • Number of completed tasks (difference between assigned and completed tasks)

  • Average time to complete a task

  • Annotation speed

  • Number of annotated objects

  • Number of tasks accepted on the first (second and subsequent) attempts

It is implied that this rating would be formed at the level of a task or a project

Describe alternatives you've considered

No response

Additional context

No response

@chernov-a chernov-a added the enhancement New feature or request label Jun 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant