Explanation of the architecture of the model

[venti07]

Hello, I have trained my own NER model with spacy. Now I have to explain the architecture of the model and the training process in the context of a scientific paper. Unfortunately, I can't find any scientific literature on the subject and I can't figure out anything from the documentation. Does anyone know literature where the architecture as a whole is explained. Unfortunately, I am not an NLP expert, so I have a hard time with the explanations from the documentation. It would be nice if someone has literature on this or could explain the structure of the model and the components. I used the following config when training:

[nlp]
lang = "en"
pipeline = ["tok2vec","ner"]
batch_size = 1000

[components]

[components.tok2vec]
factory = "tok2vec"

[components.tok2vec.model]
@architectures = "spacy.Tok2Vec.v2"

[components.tok2vec.model.embed]
@architectures = "spacy.MultiHashEmbed.v2"
width = ${components.tok2vec.model.encode.width}
attrs = ["ORTH", "SHAPE"]
rows = [5000, 2500]
include_static_vectors = false

[components.tok2vec.model.encode]
@architectures = "spacy.MaxoutWindowEncoder.v2"
width = 96
depth = 4
window_size = 1
maxout_pieces = 3

[components.ner]
factory = "ner"

[components.ner.model]
@architectures = "spacy.TransitionBasedParser.v2"
state_type = "ner"
extra_state_tokens = false
hidden_width = 64
maxout_pieces = 2
use_upper = true
nO = null

[components.ner.model.tok2vec]
@architectures = "spacy.Tok2VecListener.v1"
width = ${components.tok2vec.model.encode.width}

[rmitsch]

Hi @venti07, check out this discussion. There's a video and slides that should be helpful.

[venti07]

HI @rmitsch,
I found the architecture of the Spacy NER model in this publication. The graphic is on page 7 on the left side. Could you tell me if this is the standard architecture of Spacy tok2vec?

https://www.researchgate.net/publication/351069835_TaxoNERD_deep_neural_models_for_the_recognition_of_taxonomic_entities_in_the_ecological_and_evolutionary_literature

Thanks in advance!

[rmitsch]

Yes, that looks correct.