Multiple NER models with shared transformer #7412
Replies: 3 comments 1 reply
-
I'm trying to solve a similar problem. I updated the The training and dev data in this project example is just a JSON file of example sentences with the index/offset of each entity and its entity class. I would think a single base transformer would be specified in the |
Beta Was this translation helpful? Give feedback.
-
If you don't have merged/joint data with all your entity types to train from or you have overlapping entity types, then training one and freezing the transformer sounds like a sensible approach, since other options will degrade the performance on the first NER component. In terms of prediction, the built-in NER component is designed to preserve any existing entity spans, so it will modify tokens with unset or If you want each NER component to run on a clean slate, you'd want to write a custom component that gets inserted between NER components that copies the predicted entities to a |
Beta Was this translation helpful? Give feedback.
-
Hi,
Hope you can give me some pointers on how to properly handle my use case.
I'm currently using 4+ different NER models, which are currently trained as independent models, each with its own transformer. But actually, they are all used on the same domain and texts, so I'm thinking they should really be sharing the transformer layer.
Can you give me some general pointers on how should I approach the training? What I'm thinking is:
Is this the right approach?
A second question is: ok, I have trained my pipeline which now have multiple NERs, but how do I use it? How will the entity labels work if you have multiple NERs in your pipeline?
Thanks in advance for any kind of feedback.
Beta Was this translation helpful? Give feedback.
All reactions