How to combine NER with entity_ruler in an exclusive way?

[rsoika]

Hi,
I have a complex situation with text from invoices, where the statistical model (NER) not always finds all entities (mostly when the given text changes massive its internal structure).
I found out, that adding an entity_ruler I can fix this. For example I add a RegEx pattern to detect IBAN/BIC

        ....
        ruler = nlp.add_pipe("entity_ruler", after="ner")
        .....
        patterns = [{"label": "cdtr.iban", "pattern": [{"TEXT": {"REGEX": "[A-Z]{2}\d{2} ?\d{4} ?\d{4} ?\d{4} ?\d{4} ?[\d]{0,2}"}}]},
                    {"label": "cdtr.bic", "pattern": [{"TEXT": {"REGEX": "[A-Z0-9]{4}[A-Z]{2}[A-Z0-9]{2}(?:[A-Z0-9]{3})?"}}]}
                    ]
        ruler.add_patterns(patterns)

With this combination all BIC numbers, of course, are now detected correctly. Independently form the model quality.

But in many cases an invoice document may include several BIC numbers. The statistical model works after some training perfect on known invoice-types and so it knows for example to take the first BIC from a list. This is great and what I what to achieve.

But now with my added entity_ruler also all other BICs are always added to the result.

So my question is:

Is it possible to configure the pipes in an exclusive way, that the 'entity_ruler' only is applied if the NER did not yet found a match?

Thanks for any tips

===
Ralph

[polm]

There's no config option for this, but since pipeline components are just functions that run on a Doc, you can make a small custom component to wrap an EntityRuler like so:

# define ruler as usual
ruler = EntityRuler(...)
...

@Language.component("conditional_ruler")
def my_component(doc):
    ent_labels = [ent.label_ for ent in doc.ents]
    if "BIC" not in ent_labels:
        return ruler(doc)
   
   return doc

[rsoika]

Ok - yes I see. Thanks for this hint!

Just one more related question which is not obviously answered in the docu: The EntityRuler simply find matches and add them to the result. It is not the case, that the NER did use this as a hint for its own statistical computation - right?

[adrianeboyd]

The NER component predicts around any existing entities, going token by token through the doc from start to end. It can change unset or O tags into new entities, but doesn't modify any existing entities. Existing entities can affect its internal state when it makes predictions for O tokens, so it might not make the exact same prediction for an individual token as for a completely unannotated doc, but it's not exactly a hint because it wasn't trained at all on partially annotated docs. It may also predict a partial entity that is truncated unexpectedly when it runs into an existing entity in a following token. Does that make sense?

[rsoika]

Hi @adrianeboyd,
yes what you explain makes absolute sense and I now slowly suspect that I am making a big mistake in thinking.

As explained, I am training NER from invoice documents. I have 10 thousands of invoices for training - so I think this is a good trainingset, even if the invoices are only form round about 200 different companies (overfitting problem).

I train only a view very important entities inside my documents:

• invoice.number
• invoice.date
• invoice.total
• cdtr.name
• cdtr.iban
• cdtr.bic

But what I did so far is: I start from scratch with an empty model! I do not load the spaCy english language model. Now I think I'm making it unnecessary difficult for the NER component because SpaCy has absolutely no hints from pre-trained entities like PERSON, LOC or DATE which usually can of course be found in an invoice document.
Although my trained model is not too bad, I now take from your explanation that it would be much better to use an existing language model as a base, as this will help the NER component to recognize the position of my invoice entities much better?

Also if I use a Matcher to detect IBAN/BIC with regex before the NER and name this matches something different like 'general.iban' the NER component can learn that my 'invoice.iban' is always the 2nd match of a list of 3 different 'general.iban' entities found in an invoice document.
Or in an similar way, my 'invoice.date' is the entity following the 2nd 'DATE' entity on the document.

If you can confirm, that this is the case, which language model would you recommend for invoice recognition in general?