Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

no N/A label in relation classification #123

Open
yyzhuang1991 opened this issue Nov 29, 2021 · 0 comments
Open

no N/A label in relation classification #123

yyzhuang1991 opened this issue Nov 29, 2021 · 0 comments

Comments

@yyzhuang1991
Copy link

I noticed that in the datasets of relation classification, there is no entity pair of 'N/A' label (no relation between two entities). In the other words, entities without any relations are not considered in training and inference. For example, as I check in the dataset of SCIIE, I find only the six relations are considered: EVALUATE-FOR, PART-OF, USED-FOR, FEATURE-OF, CONJUNCTION, COMPARE, HYPONYM-OF. So basically in the evaluation, SciBERT takes a pair of entities that is assumed to to have an existing relation and classifies.

But obviously SCIIE have a lot of entity pairs that do not have any of the six relations, and they are not considered at inference time in this case. This is very different from other works that extract relations with gold entities, which do not assume a pair of input entities would have an existing relation beforehand. Specifically, they make inference for all possible pairs of entities, where the relation is classified over ( defined relations types + N/A relation) .

Could you clarify why N/A relation is not included in the training and the evaluation of SciBERT? Or am I missing something?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant