Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Descriptor points to a class that doesn't exist in the package nor any inherited packages #2

Open
ziy opened this issue Sep 25, 2012 · 1 comment

Comments

@ziy
Copy link
Member

ziy commented Sep 25, 2012

In the jdbc/eval/retrieval-aggregator-consumer.yaml, it refers to a class edu.cmu.lti.oaqa.framework.eval.retrieval.PassageRetrievalEvalConsumer, which doesn't exist in the package nor any inherited packages. In fact, the class exists in the baseqa, but jdbc-provider doesn't depend on baseqa.

Suggestions:

  1. As there is no common way to load gold-standard yet, we should move the task-specific descriptor to baseqa project. Once we have a standard procedure to deal with GS, then we can put a descriptor in jdbc-provider.
  2. Rename all the classes related to general evaluation metrics (P/R/F/etc.) to something like GeneralEval or BasicEval, instead of PassageEval as it is actually task independent.
    3, The name of the corresponding database schema might need to be changed.
@elmer-garduno
Copy link
Contributor

You are right, the reason that it works is that we have been using baseqa for all of our projects. Your suggestions are sound, lets try to implement them on the develop branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants