Packaging nlp with coreferee

[frimonkwork]

Hello, can someone help with thoughts on how I may able to package a nlp object installed coreferee(https://github.com/explosion/coreferee)

Environment
requirements.txt
spacy[cuda113,transformers,lookups]==3.4.0

Models

coreferee@git+https://github.com/explosion/coreferee.git@v1.3.0
coreferee-model-en@https://github.com/explosion/coreferee/raw/v1.3.0/models/coreferee_model_en.zip
en_core_web_sm@https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.4.0/en_core_web_sm-3.4.0-py3-none-any.whl
en_core_web_lg@https://github.com/explosion/spacy-models/releases/download/en_core_web_lg-3.4.0/en_core_web_lg-3.4.0-py3-none-any.whl
en_core_web_trf@https://github.com/explosion/spacy-models/releases/download/en_core_web_trf-3.4.0/en_core_web_trf-3.4.0-py3-none-any.whl
nl_core_news_lg@https://github.com/explosion/spacy-models/releases/download/nl_core_news_lg-3.4.0/nl_core_news_lg-3.4.0-py3-none-any.whl

writing nlp object to disk

` snlp = spacy.load("en_core_web_lg")
snlp.add_pipe('coreferee')
snlp.to_disk("./snlp")

packaging the nlp on the disk
python -m spacy package .\snlp\ .\temp\ --build wheel
Error with the above package command
ℹ Building package artifacts: wheel
Traceback (most recent call last):
File "C:\Users\user1\AppData\Local\miniforge3\envs\ts\lib\runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\user1\AppData\Local\miniforge3\envs\ts\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\Users\user1\AppData\Local\miniforge3\envs\ts\lib\site-packages\spacy_main.py", line 4, in
setup_cli()
File "C:\Users\user1\AppData\Local\miniforge3\envs\ts\lib\site-packages\spacy\cli_util.py", line 71, in setup_cli
command(prog_name=COMMAND)
File "C:\Users\user1\AppData\Local\miniforge3\envs\ts\lib\site-packages\click\core.py", line 1130, in call
return self.main(*args, **kwargs)
File "C:\Users\user1\AppData\Local\miniforge3\envs\ts\lib\site-packages\click\core.py", line 1055, in main
rv = self.invoke(ctx)
File "C:\Users\user1\AppData\Local\miniforge3\envs\ts\lib\site-packages\click\core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Users\user1\AppData\Local\miniforge3\envs\ts\lib\site-packages\click\core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\user1\AppData\Local\miniforge3\envs\ts\lib\site-packages\click\core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "C:\Users\user1\AppData\Local\miniforge3\envs\ts\lib\site-packages\typer\main.py", line 532, in wrapper
return callback(**use_params) # type: ignore
File "C:\Users\user1\AppData\Local\miniforge3\envs\ts\lib\site-packages\spacy\cli\package.py", line 49, in package_cli
package(
File "C:\Users\user1\AppData\Local\miniforge3\envs\ts\lib\site-packages\spacy\cli\package.py", line 105, in package
meta = get_meta(input_dir, meta)
File "C:\Users\user1\AppData\Local\miniforge3\envs\ts\lib\site-packages\spacy\cli\package.py", line 300, in get_meta
nlp = util.load_model_from_path(Path(model_path))
File "C:\Users\user1\AppData\Local\miniforge3\envs\ts\lib\site-packages\spacy\util.py", line 503, in load_model_from_path
nlp = load_model_from_config(
File "C:\Users\user1\AppData\Local\miniforge3\envs\ts\lib\site-packages\spacy\util.py", line 551, in load_model_from_config
nlp = lang_cls.from_config(
File "C:\Users\user1\AppData\Local\miniforge3\envs\ts\lib\site-packages\spacy\language.py", line 1812, in from_config
nlp.add_pipe(
File "C:\Users\user1\AppData\Local\miniforge3\envs\ts\lib\site-packages\spacy\language.py", line 795, in add_pipe
pipe_component = self.create_pipe(
File "C:\Users\user1\AppData\Local\miniforge3\envs\ts\lib\site-packages\spacy\language.py", line 655, in create_pipe
raise ValueError(err)
ValueError: [E002] Can't find factory for 'coreferee' for language English (en). This usually happens when spaCy calls nlp.create_pipe with a custom component name that's not registered on the current language class. If you're using a Transformer, make sure to install 'spacy-transformers'. If you're using a custom component, make sure you've added the decorator @Language.component (for function components) or @Language.factory (for class components).

Available factories: attribute_ruler, tok2vec, merge_noun_chunks, merge_entities, merge_subtokens, token_splitter, doc_cleaner, parser, beam_parser, lemmatizer, trainable_lemmatizer, entity_linker, ner, beam_ner, entity_ruler, tagger, morphologizer, senter, sentencizer, textcat, spancat, future_entity_ruler, span_ruler, textcat_multilabel, en.lemmatizer

[adrianeboyd]

It looks like there are a few small improvements needed in coreferee for loading of non-trf pipelines and entry points. See a few in-progress PRs:

• Enable packaging nlp pipelines containing Coreferee richardpaulhudson/coreferee#14
• Add entry point for coreferee richardpaulhudson/coreferee#15

For the en_core_web_trf-based model, I think that spacy package would currently work if you add the option spacy package -c code.py where code.py contains only:

import coreferee

It doesn't look like saving and loading pipelines will currently work for md/lg pipelines, so you'd need to wait for the next coreferee release.

[frimonkwork]

Hello, @adrianeboyd Thank you! Able to package only with the default name "en_core_web_trf". Is there a way to package coreferee with a custom package(I have components that am adding and constructing a new pipeline and coreferee is one of them).

[richardpaulhudson]

What happens when you try and package with another name? I don't know whether this is the issue, but please note that whatever name you are using will need to be configured in the Coreferee config file for the language you are using.

[frimonkwork]

I get an error saying coreferee can't be loaded with new package name (outside of spacy built-in packages)

[frimonkwork]

@richardpaulhudson I pip install coreferee using coreferee-model-en@https://github.com/explosion/coreferee/raw/v1.3.0/models/coreferee_model_en.zip. Does it mean configure new you in the config file and zip the whole coreferee_model_en then install the updated corefree_model_en.zip?

[frimonkwork]

@richardpaulhudson also I loaded en_core_web_trf and added my custom components then packaged with same name "en_core_web_trf". Then I force the package "en_core_web_trf"(with custom components) install(pip .. --force). I don't know this will cause any problems as am appending "en_core_web_trf". If this causes any issues, I would like to package with different name hence the ask for configuring new name in coreferee. not sure if changing the config in my local and repackage the coreferee with just new name added in the config is good enough?

[richardpaulhudson]

I don't see why this wouldn't work: I'd try it out and see if any issues arise.