Making own/new NER model from the scratch

[JorGit91]

Hi, I am new to Spacy and I'd like to know several things:

1. Can i make my own (custom) NER model in English (or any other language) from scratch (i.e. own entities, training data, etc)? or do I have to use models that have already been created?

2. If I make my new ner entity (say chemical/address detector) in any given language and would like to use it in addition to an existing model : will it detect chemical/address and Person, dates, Org at the same time ? or will I face the catastrophical memory loss problem?

3. My NER model needs to be able to identify chemicals/addresses : Do i have to train my data within sentences ? ,i.e. : "Alcohol is an organic compound" [0:6 'Chemical'] or "I live in Elm Street" [10:19 'Street'] or can I just train it as Alcohol [0:6 'Chemical'] and Elm Street" [0:9 'Street']?

Thanks in advance

[rmitsch]

Hi @JorGit91!

1. Yes, you can train your own models from scratch. One potential starting point for you is this NER tutorial (and, of course, the docs).

2. It is possible to base your new model on an existing one (see sourcing) and continue training with only examples of your new label. In theory the model might be able to recognize all the old and your new labels. In practice it often does lead to catastrophic forgetting, hence we recommend training your model from scratch. If you really want to extend a pretrained NER model by a new label, you should look into rehearsing (again, we don't recommend this).
   Alternatively you can also use two NER components in your pipeline - the pretrained one and one that identifies your new entities (e. g. chemicals). See here for an example.

3. You should train your model with full sentences/docs, as this is how the data will look like at inference time.

[JorGit91]

Hi @rmitsch!

Thanks for the answer!

I just want to make it clear that I want my model(s) to identify entities without a context/sentence in columns of data (csv, excel, json) : "John Obama", "Alcohol", " Developer" should be labeled as "People", "Chemical", "Job title". Will spacy still work or does the data have to be in a sentence/context in order to make a prediction(classification) ?

1. Is there any maximum/ amount entities that Spacy can optimally handle (i.e. perform NER)? I want to have a model up to 50 entities

2. Can i make an existing model to be more accurate?

3.Could you please provide me some links on how to train my custom model/ data that was used in order to train existing models?

[rmitsch]

The NER component might still work to some degree, but it's not the right tool for this job. The idea behind it is to leverage the sentence context to identify which parts of a text are entities and which entity label they should have. Textcat might be a better choice here, but even with Textcat this is problematic, as it's training a model based on a set of word embeddings and some of your target values might not appear in this set.

E. g. "John Obama" might not be part of the vocabulary, in which case you won't be able to fetch embeddings for this string. You could of course combine the embeddings for "John" and "Obama", which have a higher chance of being part of the vocabulary individually. You probably see where this is going though - the more uncommon terms your data contains, the more likely it is you can't retrieve embeddings for them, which in turn means that Textcat won't work properly as it relies on embeddings.

Regarding your questions:

1. There is no maximum number of entities. Consider that the higher the number of entities, the more training data you need.
2. You can continue training a model with more data using sourcing. Note that this is only recommended if you continue training with the same labels as the existing models.
3. Check out the spaCy docs on training your own model. There are also example projects on training NER and Textcat models.

[JorGit91]

What do you mean exactly with "might still work" or "it's not the right tool for this job." or "the more uncommon terms your data contains"? That the data (without context/sentence) will likely become more ambiguous for example? or that the NER component might not classify unseen data (that is not in context/sentence) at all? So far I have been able to tag 2400 Person names on that file, but , yes I did get names like Mercedes Colton or April Smith labeled as ORG and DATE respectively. Is this what you mean regarding context?

If what you meant is that ambiguity might be my biggest issue here, then I dont think so.
By the way all the entities I am aiming to classify are quite different from one another: Name, address, date, job title, gender, number, etc

[rmitsch]

"might still work" means NER training will terminate successfully and you will be able to use your model to infer entities from your data, but it's questionable how accurate the results will be (probably not very much).

"it's not the right tool for this job." means that your use case does is not what's commonly understood as NER, hence also not what spaCy's NER component was built for. The usual assumption is that NER is run on a natural language corpora corpus, i. e. full sentences with context surrounding potential entity mentions. If there's no context, NER models won't be very helpful, as they are designed to take advantage of this context.

"the more uncommon terms your data contains" means that you'll find common terms in spaCy's models' vocabulary, correspondingly you'll also be able to retrieve word embeddings for these terms. If a word is not in the model's vocabulary however, there is also no embedding for this term. Since both NER and Textcat model work with embeddings, they'll lack this source of information. If there is neither context nor a vector for the actual mention, they won't be able to learn and make informed decisions.

By the way all the entities I am aiming to classify are quite different from one another: Name, address, date, job title, gender, number, etc

The issue with your data is not that the categories aren't distinguishable enough, it's that you don't provide a lot of information to the model - there is no context and especially for names (or e. g. organisation names) there might not always be a word vector available.

[JorGit91]

Thanks for the response.

Last question : Would you be able to tell/guess which tool would be appropriate for this job?

[rmitsch]

Something like the example above gets classified pretty good

If it's working for you, great! In general the train data should resemble the test data as closely as possible.

[JorGit91]

Great. What about the question about vectors above?

[JorGit91]

..And can i see how existing models where trained? If so, could you pint out where?

[adrianeboyd]

The underlying issue is that you need context to disambiguate words like "April" as DATE vs. PERSON. People would need context to disambiguate this (like with the "John Smith" example) and NLP models also need context to disambiguate this.

For the default spacy models like NER, this context comes from the surrounding text. Word vectors might help a little in cases like "April Roberts", but the static word vector for "April" will be the same for all occurrences of "April", so what the NER model mainly needs is more context.

If you don't have context, then spacy's NER model is probably not the right choice and a rule-based approach with lists or patterns for known entity types might be a better approach. You can still use a spacy pipeline, but you might consider an entity ruler or span ruler instead of NER.

[JorGit91]

Hi @adrianeboyd !

If the main issue is ambiguity : wouldn't a costumed model (with only names in it for example) help with the issue if i only want to label/detect names? and wouldn't a sentence like "Hi, my name is April" would help when classifying v/s dates? or do we need a whole text to label a single word?

The main thing with my task is to be able to tell that a given string is a Name, so rule-based methods would be lengthy and not beneficial at all in the long run.

As pointed out above: I have put my data like "John Smith. myaccount@email.com. italian. 3.4.2023" and it has yielded very good results (full names get classified almost 96-98% of all of them) , except for those that are ambiguous: like DATE or ORG.

My whole point here with NER is that if it is able to detect/tagg names at all : i.e. if it has learned (i.e. that NER isn't a "list" based method ) to recognize peoples names and not that if a new name comes it wont be able to tell whether it is a name or not ? (of course some exotic names not but this is can be fixed by updating the method with new data).