Off by 1 error in Tokenizer #5043
-
My machine with SpaCy's git repo is currently packed, I'll contribute a patch when I set it up again, in a few weeks. How to reproduce the behaviour
Your EnvironmentInfo about spaCy
|
Beta Was this translation helpful? Give feedback.
Answered by
adrianeboyd
Feb 21, 2020
Replies: 1 comment
-
Hi, the tokenizer returns a
|
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
ines
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi, the tokenizer returns a
Doc
object rather than just a list of tokens. You can inspect the tokens like this and see that there are 6: