Electra and T5 models in transformers?

[info2000]

Hi,
There's some plans to support Electra and T5 models into spacy-transformers?
Thanks

[polm]

It looks like both of those are on the HuggingFace Hub, so they may already work. I know there's been some work on using Electra with Japanese.

If there's some reason they don't work let us know, but we are not currently working on them in any particular way.

[info2000]

Thanks @polm
I asked because on this answer #6790 (comment)
@adrianeboyd told electra isn't into the supported models

[polm]

That was eight months ago and she says that the model can be used with the latest release candidate at the time. That version of the library has been released so Electra at least should be usable.

[info2000]

I tried with a bert based model mrm8488/bert-spanish-cased-finetuned-ner but don't get success
way 1: init_model.py converter but fails because calls to unknow transformers methods
way 2: pipe
config = { "model": { "@architectures": "spacy-transformers.TransformerModel.v1", "name": "mrm8488/bert-spanish-cased-finetuned-ner", "tokenizer_config": {"use_fast": False} } } nlp=spacy.load("es_dep_news_trf", exclude=["transformer" ]) nlp = nlp.add_pipe("transformer", config=config) for doc in nlp.pipe(text ): print([(ent.text, ent.label_) for ent in doc.ents])

Here returns "IndexError: list index out of range"

Way 3: load directly
nlp=spacy.load("/pathtomodel/bert-spanish-cased-finetuned-ner")
here returns "meta.json file don't found"

I want to load the model from the actual weights and used "as-this" (without train) on spacy
it's mandatory train with spacy-cli before to could use on spacy ?

[polm]

I don't understand the ways you attempted to load the model. If you want to use a Transformers model not provided by default you should follow the instructions here, I would recommend tweaking the config from the Quickstart following those instructions to get everything right.

it's mandatory train with spacy-cli before to could use on spacy ?

Transformers in spaCy can only be used as a feature source. Technically you don't have to train them to use them as just a feature source, but since you can't use existing NER heads or whatever it's not very useful.