Hop mode doesn't work with an android screenlocked

[bricewge]

When listening on an android phone with the screen locked the hope modes (return or stay) doesn't work. Up until an add appear on the radio channel for the first time it works fine, but it fail to switch to the next channel when the add is detected or recover from it.

My guess is that the player stop outputting sound from the webpage when switching channel and that android don't allow a webpage to start playing a sound when the device's screen is off.
Keeping the player up in-between switching channels would probably solve the issue.

Thanks a lot for adblockradio, I had been waiting for something like it forever! I'm eager to see it in an dedicated device/radio.

[TeChn4K]

Thanks for your report!

We know that we have some issue with the player minimized or backgrounded... Mute mode works great because sound volume is simply reduced during ads/talks.
We can try some tricks like the one you proposed, but it will be hard to switch from one radio stream to another without stop playing...

@dest4 have you made some experiments previously about this subject?

[dest4]

Thanks for your report. I have received this feedback several times, it should be our priority now.
If I understood @bricewge correctly, the trick would be to have two media players. One would stop the first media player only when the second one has started playing audio. We need to test this though, I am not sure it would work.

[dest4]

By the way we have a similar issue in iOS.

[bricewge]

If I understood @bricewge correctly, the trick would be to have two media players. One would stop the first media player only when the second one has started playing audio.

Not really, let me try to explain it more clearly. The native player UI, that allow you to control the media on the notification screen of your OS, should never close. So when switching radio channels, the webapp need to stitch the 2 audio stream, without closing the notification player. To get something like channel1 audio -(close channel1)-> no sound (but player still open) -(open channel2)-> channel2 audio. I'm not sure if it's the way to go though.

I took a look at the code in src/components/MediaElement.js but got scared since I don't know any React.

[TeChn4K]

So I started the investigations, and it's not very conclusive yet...

I created a test page to reproduce this issue, with a similar context that the actual player. It includes MediaElement library and it switch between two radio streams every 5s.

That allows me to try different situations, like :

• React states to switch between streams by recreating audio tag
• React states to switch between streams by using MediaElement API
• vanilla JS and MediaElement API
• Use of a WebWorker (instead of the UI) to run all setInterval

Android 9, Firefox, screen off and locked.

About the last, you can read more here : https://stackoverflow.com/questions/5927284/how-can-i-make-setinterval-also-work-when-a-tab-is-inactive-in-chrome
And I tried with this library: https://github.com/turuslan/HackTimer

All these experiments have the same common issue: it start to fail after ~5min. Playback simply stops. It will behave perfectly fine again after a screen wake up.

[TeChn4K]

Now this is my current reading : https://www.html5rocks.com/en/tutorials/webaudio/intro/
Looks interesting! Mix two streams, crossfade, and never stop playing.
But it will resolves nothing if backgrounded browsers don't let us execute any code at all... I will give a try to be in the picture.

[TeChn4K]

Before all, Web Audio API doesn't support live stream for now. I read somewhere this is a work in progress. For now we have to rely on an HTMLAudioElement (audio tag) and connect it to our AudioContext via createMediaElementSource().

It looks a little bit wonky to me. But anyway, it works, and it allows us right now to use this powerful new Web Audio API.

I developed a small sandbox to experiment crossfading between two sources. You can try it here: https://testaudio.techn4k.now.sh/

Both streams will play across a common AudioContext, so playback never stops when you crossfade or stop one stream.

Click the "auto crossfade" button to alternate between each source every 10s.
Stream1 is playing -> Start stream2 -> Crossfade to stream2 -> Stop stream1. U-turn and start over.
Lock your phone and tell me what happen!

[TeChn4K]

40min without any playback issue, sounds promising!

[TeChn4K]

Before to dive into this deep sea, I would like to be sure this proposal resolves this issue or at least greatly improve it.
Please let me know how my sandbox works on your screen offed device. See my comment above.

In addition to the rewrite of the player, I would like to uncouple playback management (mute/switch on ads) from UI animations. This will allow to disable animations when the application is backgrounded.
I don't know exactly how browsers decide to limit tab running but I'm convinced it's not only because of playback stopping/starting. The lighter, the better!

[bricewge]

I didn't reported my feedback yesterday because it was quite inconclusive.

On Bromite (my Android browser) no sound was coming out from the speakers (even when using the play buttons) but the notification player was there. It is probably an issue from the browser.
Then I tried my native android browser (org.lineageos.jelly), where I got some sound but no notification player and it seems to keep playing for some time.

I'll try other devices with other browsers and report again. But it already seems a good start!

[bricewge]

I tried on two other devices with google chrome, no sound was playing but the player notification was visible.

[TeChn4K]

@bricewge you're right it's not working on Chrome Android... Firefox only. I'm on it.

[TeChn4K]

Playback is now working on Chrome and Firefox for Android.
Sorry for the inconvenience..
https://testaudio.techn4k.now.sh/

[TeChn4K]

Crossfading is playing pretty well for me since 40min on Chrome Android.

I do not have an iOS device to give a try. Someone here own one?

[dest4]

Great news, congrats for the progress!
Here is a mini checklist:

• does your code work with streams without CORS?
• is your code compatible with HLS streams? (there aren't a lot, but a at least few)

[TeChn4K]

Thanks for your checklist!
I have no experience into this topic so it's better to ask even obvious questions :)

I think CORS question is answered by

const audio = new Audio();
audio.crossOrigin = "anonymous";

RadioNova stream was not playing without this preset.

I have never heard about HLS streams ... Have to dig into.

[dest4]

Here is a brief summary for both topics:

• HLS is a way to stream content with files of finite length, e.g. 10s audio, instead of the continuous HTTP request that never completes. It's been developed for video by Apple and features one or more playlists (*.m3u8) that have to be periodically refreshed and ultimately link to media files. The reference implementation is "hls.js" (https://video-dev.github.io/hls.js/stable/), but it seemed easier to just use the more abstracted MediaElement (https://www.mediaelementjs.com/).
  I occasionally handle HLS in my daytime job, I may help further if you need.
  You can test with this radio stream: http://www.radio-browser.info/gui/#!/history/9627eb5f-0601-11e8-ae97-52543be04c81 (Radio Capital, Italy)

• CORS is a standard to allow which "foreign" resource is available from JS. It's a HTTP header, typically Access-Allow-Control-Origin: *, that, when missing, forbids JS to access resources not served on its own domain (in our case, adblockradio.com). That means you can have HTML player that read CORS protected content on the HTML side, but it will fail as soon as you'll try to gain access to raw data in JS. Classic HTML media element is fine but web audio API is a grey area, it will work or fail for subtle reasons.
  CORS is what has prevented me to implement the analysis algorithm in the browser itself. To circumvent CORS, you need a native client or you need browser web extensions.
  You can try with this stream http://www.radio-browser.info/gui/#!/history/961b8569-0601-11e8-ae97-52543be04c81 (Basso Radio, Finland)
  To test a stream for the presence of this header, just to curl -I http://... (capital i, not L)

[TeChn4K]

Hi guys,

The previous version of the sandbox, fully using Web Audio API, was not able to play HLS streams... It's not natively supported. 😞

So I built the same sandbox with MediaElement.js, which allows to play HLS easily.

You can try it here : https://testaudio-git-mediaelement.techn4k.now.sh/
Channel 1 is playing an HLS stream.

I have not tried anything with crossfade and screen off for now. But I noticed the same cracking sound issue with volume slide up/down that on ABR (at least on Firefox).

Give me your feedback!

[bricewge]

I have a hard time reporting my various tests on the various prototypes because I find it very difficult to reproduce them. So take my reports with a grain of salt.

The second prototype version, when playing in background, with cross-fading stopped playing after 5min.

The latest prototype, with no cross-fading and no background, plays the channel1 (HLS) nicely whereas channel2 stops intermittently. Cross-fading on Linux Chromium make it trip on a promise rejection.

[dest4]

It seems HLS errors are not handled, so that as soon as a HLS error throws, HLS playback stops.

Also I think we should not hardcode delays to wait for playback start (2000 ms) but instead use the mediaelement events.

I have forked @TeChn4K and am working to further improve it. Will report back when ready

[dest4]

Here is my work:
https://github.com/TeChn4K/test_audio/pull/2/files
https://testaudio-zxwd99rrs.now.sh/

It works for a while on my screenlocked android (like for 10 minutes with a switch every 3 seconds). Then it stops, probably because I coded it to stop when a stream raises an error. Handling is still partial.

On the good side:

• the code does not use mediaelement anymore, barebones hls.js
• track switching does not have a hardcoded overlap duration. It relies on events.

[bricewge]

The last prototype lasted 30min easily on the devices I tired it on. The only minor issue (also present in the previous ones) is the pause button from the notification which isn't effective; after 3sec it start playing the other stream. So there is now way to pause the audio directly from the lock-screen.

Anyway, it's looking good, thanks for your work! I hope it'll make it in the webapp soon.

[TeChn4K]

Same for me, playing last proto for hours without major issue. Time to time some minor switch latencies with screen off.

@bricewge We will keep in mind lock screen play/pause button. This is a nice feature!

@dest4 I was thinking about to straightly use hls.js, without mediaelement.js, but no time to give it a try, glad you did it!

So, as the last proto is working good for everyone, I will continue on this track.
Next steps for me: code a small vanilla js player service using Web Audio API and hls.js which meets ABR requirements. After what I will be able to start integrate it to ABR web interface.

[TeChn4K]

I started to develop the standalone player.
You can try it here, with a minimal UI : https://testaudio.techn4k.now.sh/

Right now:

• Play & Pause (stream loading canceled, not just playback paused/muted)
• Switch stream (not crossfaded for now)
• Loading feedback
• Master volume (cracking sound issue resolved !)

Roadmap:

• Instant crossfading
• Progressive crossfading

You can give me feedback if you want but it's not finished yet! Keep your strength up, we will need it ;)

[TeChn4K]

New features for now:

• HLS capable
• Fade in/out on start/stop or stream switch
• Customizable fade delay
• Possibility to fully disable transitions to lighten player (background playback)

Missing:

• error management
• clean the API
• develop the demo to be able to test all features intensively

[dest4]

Great! :)

I have left technical comments in your commits.
Remark about user experience:

• when we switch, it is okay-ish in my opinion to have smooth transitions between streams, but the best would be to have a nice cut. If an ad is playing, I don't want to hear it for xxx ms more. In that scenario the point of this PR would be to make the player work even on screen locked, stopping the previous stream when the next one has started. Remember that when the tab is in the background you may have a poor reliability of timers, or you need a web worker for them to work properly (I am unsure about the reliability of timers in background when an audio stream is playing / when web sockets are active).
• when we stop, I would like the stream to stop immediately.

[TeChn4K]

Understood @dest4, I took some notes for the rest.

About the demo/sandbox, this is not the behavior that will be in the "next ABR". Its purpose is to test and approve the stream player as a service, not the UI itself ;)

Yes, background playback must be able to play & switch without any timers, and be as lightest as possible. So no smooth transition if we don't want to be "degraded" by browsers. As it stands for now in the production version, this is not possible at all because React/Player/Settings are intertwines and switch between streams involve too many treatments. Unfortunately we need to fully rewrite these core features...
The web worker timers technique has been invalidate in a previous sandbox. "Web worker timers" now behave just as "normal timers".