Training own NER model from zero

[JorGit91]

Hi, I am new to Spacy and i am trying to build my own NER model. However, I do have some questions :

1. Can i train my NER so it also recognizes abbreviations/misspellings of the entity it is supposed to detect? For instance: New Yrk--> New York

2. Why is the german model so innacurate when the entities are lower case? What should i be aware of when training my NER model if i want it to be almost accurate in both lower and upper cases? and how could i update the german model so it is equally accurate in both cases?

3. Does the method nlp.pipe() that does the pipeleine faster affect accuracy?, if so how much? can i use the same for my own NER model or does this only work on the sandard NER Model?

4. Does the text upon i train my own NER model have to be consistent, i.e. does this text have to be about one same topic, or could it be loose sentences,containing the entity im interested in, that are unrelated between each other?

5. when training: do all the vectors, word embeddings , etc are provided by spacy? or do i have to do this manually?

6. My ner is identifying emails as names: how could i change this?

[adrianeboyd]

The general advice from the earlier thread #12570 still holds. Most of these are more general NLP questions, but a few spacy-specific pointers:

3. Does the method nlp.pipe() that does the pipeleine faster affect accuracy?, if so how much? can i use the same for my own NER model or does this only work on the sandard NER Model?

nlp vs. nlp.pipe should not affect the output or accuracy.

5. when training: do all the vectors, word embeddings , etc are provided by spacy? or do i have to do this manually?

There are vectors provided as part of the trained md/lg pipelines, but no vectors provided by default by the core spacy library itself. You can use spacy init vectors and the settings in your training config to use whichever static vectors you'd like.

[JorGit91]

Hi and thanks for the prompt answer!

Then if I want to train my owm custom model, say for detecting names of chemical compounds, do i need to pass only the data during the training process (and its labeling) or do i have to make all the vectors thing too?

and could i update the english model with people names in other languages. So let's say i want the english ner model to detect norwegian names like Martin Ødegård. will it be able to do it or i have to do my training/updating in the norwegian model?