Sentence segmentation with 'en_core_web_trf' shows unexpected behavior

[igormorgado]

Segmentation with transformers is made in very uncommon places. Let me show some small examples. The text is the following

text = """David noticed he had put on a lot of weight recently.
He examined his habits to try and figure out the reason.
He realized he'd been eating too much fast food lately.
He stopped going to burger places and started a vegetarian diet.
After a few weeks, he started to feel much better.
"""

Processing with en_core_web_trf gives the following

nlp = spacy.load("en_core_web_trf")
doc = nlp(text)
for idx, sent in enumerate(doc.sents):
    print(f"{idx} '{sent}'")

Output

0 'David noticed he had put on a lot of weight recently.'
1 '
He examined his habits to try and figure out the reason.
He realized he'd been eating too much fast food lately.'
2 '
'
3 'He stopped going to burger places and started a vegetarian diet.'
4 '
After a few weeks, he started to feel much better.'
5 '
'

While using en_core_web_sm (or md or lg), gives

nlp = spacy.load("en_core_web_sm")
doc = nlp(text)
for idx, sent in enumerate(doc.sents):
    print(f"{idx} '{sent}'")

Output

0 'David noticed he had put on a lot of weight recently.
'
1 'He examined his habits to try and figure out the reason.
'
2 'He realized he'd been eating too much fast food lately.
'
3 'He stopped going to burger places and started a vegetarian diet.
'
4 'After a few weeks, he started to feel much better.'

As expected.

My versions are the following

spacy                      3.7.6
spacy-alignments           0.9.1
spacy-curated-transformers 0.2.2
spacy-experimental         0.6.4
spacy-legacy               3.0.12
spacy-llm                  0.7.2
spacy-loggers              1.0.5
spacy-lookups-data         1.0.5
spacy-transformers         1.3.5

How can I improve/correct the segmentation created by the transformer?