[Feature Request] More navigation options in audio player

[tage64]

Hi, and thanks for this fantastic project!

I've been using Thorium with the NVDA screen reader to read e-books for a while. But I now decided to try and read some audio books as well. I noticed that the audio player lacks some important features for navigation. There are specificly two features I think about:

• Ability to rewind and fast forward the audio. It would preferably be a shortcut to jump something like 10 seconds backwards or forwards.
• Navigation on different levels in the book. I don't know exactly how EPUB works, but DAISY organizes chapters in a hierarchical structure. So that chapters for instance might be on level 1 and subsections on level 2. Currently in Thorium, one can only jump to the next or previous chapter on the "lowest" level (I.E. subsections). But it would be useful to be able to jump to the next chapter or next part of the book with a single shortcut. It is of course possible to do that from the table of contents, but it would be very convenient to do that directly from the audio player.

This navigation on different levels is a quite essential feature of the DAISY format and it seems to me, (at least when I look at the table of contents), that EPUB is structured in a similar (hierarchical) way. It would preferably be a global variable holding the current level of navigation, defaulting to lowest (I.E. smallest) level. This variable should be reset for each new session/window. I.E. if one reads a book, closes the window and opens the book again, the current level of navigation should be reset to its default value. There should be two buttons/shortcuts to jump to the next/previous (higher/lower) level of navigation. And two shortcuts to jump to the next/previous section on that level.

This system of quick navigation on different levels would also be useful in normal reader mode, (I.E. not only when reading audio books). It could also be extended to include pages and perhaps even thos 10-seconds jumps (I mentioned earlier) as levels. Then one could not only choose to navigate between chapters of different levels but also between pages with the same shortcuts.

What do you think about these ideas? If I speak for my own, these features I've just mentioned are what blocks me from using Thorium to read structured audio books effectively.

Best regards,
Tage

[danielweck]

Thank you very much for the detailed feedback.
What audiobook format(s) are you opening with Thorium? What version did you test with?

We currently support EPUB3 Media Overlays synchronised text-audio publications, which are not "audio books" per se, as the text component is quite prominent in the user experience, i.e. the text document takes center stage visually in the user interface, occupying most of the screen real estate, whilst the audio controls are positioned in the top toolbar.

The reason I mention this is because Thorium also supports "proper" audiobooks in Readium's own format or with the recently-created W3C standard. Here the user experience is audio-centric in the sense that the reading order is dictated by the sequence of audio resources, text is only layered on in the form of a table of contents (i.e. heading labels), as well as page lists and other landmarks. The user interface is very different in this case, with more typical audio player controls (including a linear time scroller / interactive slider) that take center stage in the middle of the window, including the cover image if there is one (note that this GUI is currently not very accessible to screen readers, and not localised properly like the rest of the application "chrome").

Now, recent builds of Thorium also support DAISY digital talking books (Daisy 2.02 and Daisy 3) in the main 3 variants: text-only, audio-only (with table of contents, page list and phrase-level navigation), and full text full audio.

Internally, text-only Daisy publications are converted to their EPUB equivalent so the reading experience is identical to EPUB.

Full-text full-audio Daisy books are also internally converted to EPUB3 Media Overlays, so we preserve the SMIL playback granularity as well as the higher-level TOC and page break navigation. We also preserve the skippability semantics, such as ignoring page numbers during playback.

Where things get a bit more confusing is with Daisy audio-only books. Currently Thorium internally implements support for phrase-level navigation by preserving the SMIL structure as an EPUB3 Media Overlays. In other words, despite the audio-centric format we offer a visual experience based on the text-centric user interface, resulting in displaying empty text blocks that represent Daisy phrases. Note that textual headings are shown when available, but of course most phrases do not have text at all so instead we display a blocky shape that gets highlighted in a synchronised manner, as audio playback occurs. The net benefit is that the standard EPUB3 Media Overlays controls situated in the toolbar above the main publication viewport can be used to trigger previous/next commands i.e. navigate the audio stream at the phrase level (something that the pure audiobook Readium and W3C formats do not currently replicate). Note that we support skippability here too (e.g. ignore page numbers during playback or previous/next navigation)

It is worth noting that under the hood, Thorium actually supports the lossy conversion of such Daisy audiobook to W3C/Readium audiobook, but we do not currently expose this feature in the user interface. We think that it is worth exposing the phrase-level playback structure, even though the GUI is currently a hack based on the EPUB3 Media Overlays user experience.

To conclude, there is definitely room for improvement, no doubt about that. Thank you for your feedback once again, let's see what we can do to enhance our accessible audiobook support.

[GeorgeKerscher]

I have seen the rewind and fast forward implemented in a variety of ways. The arrow keys with a modifier seems to be a good candidate. One approach is to press the arrow key with a modifier and it skips back or forward a certain length of time. I have also seen implemented a "scrubbing" effect, where you hear the audio. With audio tape players, this is easy, but in digital systems, this is difficult to do What I have heard is when holding down the arrow key, you hear some audio being played fast. If you are moving backward faster, the audio clip you hear sounds faster. Another approach is to hold down the arrow key and modifier for a short time and it says "back five seconds" and keep holding down and it says back 20 seconds, and continue to hold and it says back 1minute, back 5 minutes, back 10 minutes back 15 minutes, back 30 minutes, back one hour.

All of these approaches are possible and it is clear that this is needed implemented in one way or another.

[tage64]

I actually had a Daisy 2.02 book which I converted to EPUB3 with Daisy Pipeline because I didn't knew how to open Daisy books with Thorium. What file should I import in order to read a Daisy 2.02 book with Thorium? The entire book as a zip-file, the NCC.html or something else?

Regarding the rewind/fast forward feature, I think that the easiest solution is to have one command to jump back 10 seconds and one command to jump forward 10 seconds. Then if someone wants to jump many minutes or hours it'd be better to have a specific option to jump to an absolute time of the book. It's quite common when you listen to a book that you want to jump back 20 seconds or so to hear the last sentence again, but it's not that common that you want to jump back 45 minutes, then you usually want to jump to the previous chapter or so instead. Hence I propose there should be a quick shortcut to jump 10 seconds in any direction and not several minutes or hours.

[danielweck]

I am moving this meta issue to a GitHub discussion. The path forward will be to extract actionable issues from the discussion, so that development tasks can be assigned in the tracker.