Improving spacy NER performance

[baivabdash]

Hi,
I'm training an entity extraction (NER) model. I'm using spacy transformers with distilbert as the base model. The input texts for the task are pages from pdfs so you can imagine inputs to be large. Im getting a score of around 0.6. I was wondering what all hyperparameters can I tweak on the config file to get a better score?

Thanks in advance

[polm]

Totally depends on your task. What are your documents, what kinds of entities are you extracting?

[baivabdash]

@polm I am having financial documents in the form of pdfs. I'm using individual pages of it as input to train the model. The entities I'm trying to extract are details names, gender, age, etc. I am asking if you can give a hint on which all hyperparameters affect the accuracy of the model.
On those, I can perform experiments to observe empirically.
Thanks

[polm]

Uh, that's not really enough detail to help you much. Let me rephrase. What kinds of things are you having trouble with? For example, if you're working from OCR that might be causing issues that have nothing to do with hyperparameters.

If you really want to fiddle with hyperparameters specifically, maybe try different learning rates, or learning rate schedules, or optimizers. Besides that there's dropout. Otherwise... I guess you could try changing the base transformers model?

If you're at 60f1 there's probably something you can do that's more effective than adjusting hyperparameters, but it depends on the details of your problem - what do your sentences look like, do you have a class that's underperforming, etc.

[baivabdash]

Thank you @polm, I will try the things you mentioned. Actually, the model is performing fine. I am looking for ways to improve the performance slightly by fiddling with hyperparameters.