add multiple entity rulers with spacy 3

[lfoppiano]

Hi, my question is basically the same as here, but how to do it with spacy 3?

Additionally, does it make sense to have two entity rulers or it's better to mix pattern into a single one?

Thank you

[DuyguA]

What is the use case, would it be possible to provide some details? It might be the case that you don't need to Entity Rulers, but rather several patterns and add the patterns.

[lfoppiano]

The use case for this is that I create 2 set of patterns from two different sources and I use different "loader" from each source.
This first part outputs the patterns in a specific folder. You can see here the code of the loader..

I then have a service that, for each of these folders create a new entity-ruler. Initially i tried to run from_disk in sequence but at each invocation the entity-ruler is cleaned up. You can find the service here.

In the end, I solved it with the following solution:

@Language.factory("my_component")
def create_my_component(nlp, name):
    return EntityRuler(nlp)

[...]

ner = spacy.load("en_core_web_sm", disable=["parser", "textcat", "ner"])

        print("Loading space groups patterns...")
        entity_ruler = ner.add_pipe("entity_ruler")
        entity_ruler.from_disk(configuration['space-groups'])

        print("Loading crystal structure patterns...")
        er_crystal_structure = ner.add_pipe("my_component")
        er_crystal_structure.from_disk(configuration['crystal-structure'])

Indeed, probably two entity rulers are a bit overkilling, I agree.

[DuyguA]

OK, I examined the code a bit. The first pattern is related to crystal structures and second one is related to space groups, right? I googled a bit what space group is (concept in chemistry turned out to be same with mathematics 😉 ), semantically looks close enough to crystal structures. I'd add these 2 patterns to one entity ruler, these 2 entity groups sound semantically related enough to me. There's also loading from disk, that can be bit overkill 🙂 Overall, I'd use only one entity ruler, what do you think?

[lfoppiano]

I got your point, however, I prefer to use two entity rulers because the patterns are generated using two different logics which output on two different directories, therefore when I want to load them I'd be more comfortable with two entity rulers.

thanks anyway for your feedback 🙂

[kinghuang]

You can add as many entity rulers as you want.

nlp = spacy.load("en_core_web_sm")

spacy_groups_ruler = nlp.add_pipe("entity_ruler", "spacy_groups_ruler")
crystal_structure_ruler = nlp.add_pipe("entity_ruler", "crystal_structure_ruler")

The first argument to add_pipe is the factory name; the second is the component name. If you don't specify a custom component name, it takes the factory name as the default. Just make sure to give each entity ruler a distinct component name.

See Language.add_pipe for the full documentation.

[lfoppiano]

Thanks, I think this is the solution I was looking for. I overlooked it when looking at the documentation..

[polm]

For reference, there's an example project that includes multiple NER components here. NER components are slightly different, but this still covers:

1. How to add multiple copies of the same component
2. How ordering matters
3. How to do it in config or code

[lfoppiano]

Thanks! Good to know!