Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TruLens Docs (Feedback Functions) Graphic Improvement Suggestion #1198

Open
DrewGalbraith opened this issue Jun 12, 2024 · 1 comment
Open
Labels
documentation Improvements or additions to documentation enhancement New feature or request

Comments

@DrewGalbraith
Copy link

I'm not sure if this is the right place to suggest enhancements for the TruLens docs, but while this graphic is already helpful in explaining the surrounding text, it could be stronger.

Suggestion

The graphic would be clear if the 'Meaningful' axis was a finite scale. It should range from the least meaningful evaluation (random binary classification, i.e. hallucination/not hallucination) to the most meaningful possible evaluation (a representative ground truth), approximately scaling everything else in between. Additionally, the upper bound of 'Scalable' would likely be that same random classification; I don't see a lower bound for this axis. The point placement along axes wouldn't feel so arbitrary if this finite scaling was worked into the graphic.

Disclaimer/Justification

I understand the most important part of the graphic is the general relationship of each point to each other as opposed to their placement relative to the scales. I contend with that in two ways:

  1. Given the points aren't equidistant, especially along 'Scalable', there does seem to be an effort to communicate size of disparity as opposed to just general direction.
  2. Having 'Ground Truth Evals' sitting 2/3 along the image is unclear, suggesting there are even more informative/meaningful evaluations for a use-case not listed on this graphic. There aren't though, assuming ground truth is composed correctly.
@dosubot dosubot bot added the documentation Improvements or additions to documentation label Jun 12, 2024
@sfc-gh-jreini
Copy link
Contributor

Thanks for the feedback @DrewGalbraith - we'll take this into consideration

@sfc-gh-jreini sfc-gh-jreini added the enhancement New feature or request label Jun 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants