Throttle reads from standard input if output queue is full

[npat-efault]

Currently picocom never stops / delays reading from the standard input (terminal) if data are available. If the output queue is full, data read are discarded. Maybe it would be better if we could delay reading from stdin until the output queue partially drains. This could be useful particularly when the standard input is not a terminal and, possibly, it should be done only in this case...

[JoeMerten]

What's the way to reproduce that issue of discarding chars?
I tried out to copy&paste 20k text data into my (linux) console and observe that tty_q seems to be grow (realloced) accordingly. But don't see lost characters.

On TX terminal, were I paste the text data into:

New tty_q size: 512
New tty_q size: 1024
New tty_q size: 2048
New tty_q size: 4096
New tty_q size: 8192
New tty_q size: 16384
New tty_q size: 32768

[npat-efault]

Practically, only by setting a limit on the queue size (i.e. TTY_Q_SZ != 0) and trying to copy-paste something large.

But DO NOT try to fix it, as this (dropping input chars) is probably the CORRECT behavior...

That's why I marked this as "needs more thought". I opened the issue more as a note to self...

[npat-efault]

At least, it is the correct behavior, when the input is from a terminal, that is...

Plus, the fact that the output queue can grow as much as needed, makes the problem very hard to practically matter (unless you are using really huge files as input)...

[dleonard0]

If picocom stops reading stdin to apply backpressure because its output queue is full, then C-a sequences would be unavailable, making user recovery hard. e.g. when the back tty device is malfunctioning and not draining, stuck cts, etc.

Idea: If you really want to apply backpressure to stdin (by entering a state where you just don't read stdin), maybe enable a restricted ISIG termios mode on stdin, with the escape char C-a mapped to VINTR/SIGINT. Then some (complicated) logic to "wake up" a single stdin read on the signal. You'd accompany this with a command to dump the output queue, eg C-a C-o.

[npat-efault]

When standard input is a terminal, there is no real reason for picocom to throttle its reads from it, in order to apply back-pressure. In a huge copy-paste, for example, (the only case I can think of where it would be practically applicable), it normally shouldn't matter at which end of the pipe you buffer the data (yes, conceivably the two ends could be on different hosts, but let's not get into that). After all, copy-paste is not expected to be a fully reliable transfer method for huge amounts of data. Not being able to "see" the escape / command characters in time (as you say) is another very good reason not to do so. So when stdin is a tty you should always read from it, and either buffer or discard what you read. Anything else (like the complicated signaling dance you describe 😆 ) lies on the path to insanity! EDIT If you ever get it to work reliably, after great effort, it will be at a big loss of maintainability and possibly portability, and for very little practical gain.

When reading from a file,though, or if you have the escape character disabled, throttling your reads could be useful and practicable---and that's the real reason for this note / issue.

It hasn't come up, in practice, especially since I've put in the resize-able output buffer, but conceivably it could be useful.

[dleonard0]

I think the auto-sizing tty_q.sz is good enough for practical use. Like @JoeMerten I've tried rapidly pasting multiple 35kB text blocks (gnome-terminal and 2 x PL2303) and it works great.

Another crazy idea/workaround while I'm here, inspired by the vi trick of !!cat /dev/tty, is to abuse sx similarly (eg C-a C-s !cat /dev/tty) so that some other process will deal with blocking.

[npat-efault]

Yes the grow-able output buffer is good enough, at least for the present