Skip to content

An update on the Android audio latency issue

December 2, 2012

With the advent of newer versions of Android (4.1, 4.2) and newer devices, the topic of latency has come a bit more into the mainstream of system development. Newer APIs in the Android SDK are providing means to enquire devices about low-latency parameters (buffer sizes and sampling rates).  But how does that translate in practical terms to latency reduction?

My tests with Csound

Well, I got hold of a Google Nexus 7 tab this week and went about measuring it. Using the latest Csound, whose OpenSL IO code is (I expect) optimal, I first measured touch-to-sound, and then round-trip latency. The first measurement was made by recording an audible mechanical ‘tap’ as I touched the screen and the sound coming out of the speakers. Granted, it is not as fine-grained as a video test, but it is not too far off the mark and would be particularly close to user experience as one tries to manipulate the UI. The second was done by using a microphone attached to a pair of headphones, which would pick up a click from its source and then from the tablet output.

Results were as follows:

  • best touch-to-sound latency: ~120ms
  • best round-trip latency: ~105ms

The parameters for Csound were -B1024 and -b128, which means, in this particular case, circular buffer and audio buffer sizes of 1024 and 128  frames, respectively. What I observed was that the size of the circular buffer was critical: anything less than about 1024 would bring either intermittent or constant dropouts, regardless of the audio buffer size. Also, smaller sizes would not translate to lower latencies (e.g.  low-latency but with dropouts), the lower limit never changed. This means that although the buffer sizes can be made very small, there will be considerable jitter in the OpenSL buffering and that the lower layers of buffering, which we do not have access to, are the ones ultimately responsible for the latency (we knew that, but this confirms it).

Although the round-trip figure is far away from low-latency, it is actually becoming more usable. When testing little processing apps (echos, reverb, flangers), actually it does not feel too bad. I would say it can be used in live-electronics settings that are not too strict in terms of live-processed synchronisation, and I plan to use it in a piano piece I have been asked to write, next year.

What is a bit disappointing is the touch latency, which appears to contribute more to the overall latency than the single-direction audio delay. This is an issue for performance systems using the tablet/phone touch interface, although it might be solved via external controllers  (but do we want to depend on these?).

Other reports

I have reported these measurements to the Android audio discussion list and was interested to hear other results posted there, which confirm my numbers in the case of the same hardware and Android version.

Amongst these reports, in particular, there was a meticulous measurement made with a video camera, graphics and audio. After a touch,  graphics and audio would be generated, so touch latency could be evaluated separately. This is another way to split the measurements, if you are not looking to measure round-trip delay. The hardware used was a low-cost tablet and Android 4.0. Results were as follows: touch-graphics delay was ~90ms; touch to audio, ~250ms. These measurements also line up quite well with my own experiences with the Galaxy Tab 10.1 and 4.0.

Conclusions

I can drawn some interesting conclusions from this, and as the movie goes, some are good, some are bad, and some are definitely ugly:

1. The good: things are improving and we have a situation now that for some applications, audio processing can be usable. They seem to be listening to us.

2. The bad: development is far too slow. Older devices with fine computing capabilities (like my Galaxy Tab 10.1) are being left behind.

3. The ugly: a comment in the Android audio list says it all, “I wished that those shiny tablet magazines did simple tests like this, and then published the results instead of irrelevant values like their thickness and number of processing cores”. In my opinion, with the plethora of magazines and online review sites that exist today, someone has to provide us with better information in this regard. As it is now, they are only patting hardware and software vendors on the back, helping to preserve the status quo.

Post-script

As an aside, I have noticed that in Android platform-14, the OpenSL ES headers are providing an alternative streaming audio interface, identified by  SL_IID_ANDROIDBUFFERQUEUESOURCE. Well, I thought, let’s give it a go, maybe this is a door for devices to support lower latency streams. It turns out that although it is a published interface, it has not actually found its way into the library binary, which is odd. Furthermore, in the release 8b of the NDK, there is a SL_IID_BUFFERQUEUE in the library (but that is not in the headers), which has now disappeared in the latest, 8c, release.

Anyway, the documents still seem to say that the Android Simple Buffer Queue interface is to be used, although this does not mean much, as the documentation can sometimes lag behind the released code. I have filed a ticket with Google regarding the absence of the buffer queue source interface, so it will be interesting to see how they respond to this.

About these ads

From → Android, Csound

9 Comments
  1. Thanks for sharing this Victor.

    It is always valuable to have some references for latency measurements :)

  2. liam permalink

    Looks like they submitted a patch.

    https://android-review.googlesource.com/#/c/47481/

    • yes, indeed. They were very quick off the mark. It will be interesting to try the new interface, when it comes out.

  3. Adriaaaaan permalink

    Bear in mind also that low latency is hardware specific. The n7 has over double the audio latency of the galaxy nexus. To me the gn has no perceivable audio latency (for piano/drum apps).

  4. Taras permalink

    Thanks for you blog. Could you please advice how to measure audio latency with using NDK and openSL?

    • The measurement mode for input and output round-trip is to use a separate recorder and check the time delay between a short impulse going in and
      coming out of the speaker. That will give you a total round-trip latency. For output only, unfortunately, the touch latency will be lumped
      in with the audio. For this you can use two methods, a more accurate one, which is to use a video recording with a high frame rate and
      then check the delay between the touch moment and the synthesised output. A more rough measure can be done by audio recording a tap and the
      resulting synthesised sound, and checking the delay.

  5. Hi Victor, could you let me know has the situation changed in the last 9 months? there must now be an accepted pathway to get somewhat acceptable results?

    Maybe its using the NDK? I hope that’s not the answer as I’m not too familiar with C programming…

Trackbacks & Pingbacks

  1. Low-latency audio playback on Android

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 30 other followers

%d bloggers like this: