Mix_PlayChannel Android lag |
Mix_PlayChannel Android lag |
David Olofson
Guest
|
On Thu, Apr 3, 2014 at 6:52 AM, LBandy wrote:
[...]
Android is notorious for awful audio latency... Haven't investigated it deeply, but by the looks of it, the API is fundamentally broken by design. (It doesn't technically need to be callback based - though all serious realtime audio APIs are - but it simply has to play by the same rules as a callback based API, to avoid intermediate buffering behind the scenes.) You might want to try some simple test using the SDL audio API directly, but I wouldn't think SDL_Mixer is part of this problem. Only the sample rate and buffer size should matter. (Higher sample rates and smaller buffers decrease latency.) Are you getting better latencies with apps that use the Android audio API directly? (Possible if they use other APIs, as those might bypass the problematic API that SDL is using. I suspect that would be of no use to SDL, as the APIs I'm aware of are not streaming APIs - but if you really have to, you might be able to use that instead of SDL_Mixer...) -- //David Olofson - Consultant, Developer, Artist, Open Source Advocate .--- Games, examples, libraries, scripting, sound, music, graphics ---. | http://consulting.olofson.net http://olofsonarcade.com | '---------------------------------------------------------------------' _______________________________________________ SDL mailing list http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org |
|||||||||||||
|
Mix_PlayChannel Android lag |
Eric Wing
Guest
|
On 4/3/14, David Olofson wrote:
I agree with David that a simple test should be done. But David is also correct that Android audio is famously terrible. I usually have a list of links I refer people to, but I misplaced it. I have been fighting Android audio for years now and I always get blamed, but it's completely Android's fault. Yes, their APIs are terrible. They have like 4. SoundPool, AudioTrack, MediaPlayer, and OpenSL ES. The first 3 are Java, the last one is C. SoundPool is a massive hack that historically had significantly lower latency, but the API is so nerfed, it wasn't really usable for any real world cases. But this is why people always blame me instead of Android...because you can make a toy app that doesn't exhibit the problem. (And then in 2.3.x, a very popular Android device had a bug that crashed apps that used SoundPool. And now I've seen a few cases that have high latency with SoundPool.) OpenSL ES is the only C API, and really is the future of Android audio. But it's still not there yet. I'm not willing to claim AP is the problem though. The real problem at least right now is much deeper. The problems go as deep as the kernel and the scheduler. There was actually a very good talk given by their audio guy at Google I/O. He went pretty hard core and built his own measurement equipment (using photodiode and oscilloscope) to measure latency. They did make some good progress in 4.1. However, they still have long ways to go. P.S. I have non-latency gripes about their OpenSL ES API extensions for loading/decoding. You must read from a file or URI. They don't have a read from memory system or anything that is fopen/fread/fseek signature compatible, so it is impossible to interface with SDL_RWops. -Eric -- Beginning iPhone Games Development http://playcontrol.net/iphonegamebook/ _______________________________________________ SDL mailing list http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org |
|||||||||||||||
|
Mix_PlayChannel Android lag |
Jonas Kulla
Guest
|
2014-04-03 13:28 GMT+02:00 Eric Wing:
This comes to mind: https://www.youtube.com/watch?v=d3kfEeMZ65c not sure how helpful it is to SDL though... |
|||||||||||||||||
|
LBandy
|
Thanks for the input guys!
I'm involved in various projects using different sound playback libraries. In another game where I'm using fmod, it gives a much better result, as the lag is nearly unnoticable. So I'm just curious, if there is a known method using the SDL library to play sounds in a way that could be acceptable, or should I use external libraries for this problem? Should I try SDL_Audio and expect some progress? |
|||||||||||
|
Mix_PlayChannel Android lag |
David Olofson
Guest
|
On Thu, Apr 3, 2014 at 10:01 PM, LBandy wrote:
Unless there's a bug in SDL_Sound, I don't think bypassing it is going to help. It's certainly possible there's a problem, but it doesn't seem likely to me, based on what I know about how it's implemented. (No time stamps or "sub-buffer" timing logic that can break, AFAIK.) Unfortunately, I suspect bypassing the SDL audio API isn't going to help either, as the problem is in Android itself. As I suspected, it might be possible to get lower latency for basic stuff with another Android API - but as Eric pointed out, that instead has even more serious issues on certain Android versions. So, in short, do try other options, but don't hope for miracles... :-/ -- //David Olofson - Consultant, Developer, Artist, Open Source Advocate .--- Games, examples, libraries, scripting, sound, music, graphics ---. | http://consulting.olofson.net http://olofsonarcade.com | '---------------------------------------------------------------------' _______________________________________________ SDL mailing list http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org |
|||||||||||||
|
Mix_PlayChannel Android lag |
Eric Wing
Guest
|
On 4/3/14, LBandy wrote:
I'm a little skeptical that you will see any major difference between SDL_Audio and SDL_mixer since the latter is built on the former. But if you try it, I would like to know your results. I know early on, fmod was affected by the same Android latency problems. Theoretically, you can only optimize around the areas that are not Android, so it could be they've streamlined their own data pipeline more than SDL_Audio. Apportable's OpenAL soft fork attempted to do just this, and made an integer/fixed-point modification to OpenAL (as well as implement an OpenSL ES backend). (The hardware floating point situation on Android is also a mes.) Apportable's fork is what I use (and tweak). It did improve things quite a bit from where they were with OpenAL Soft. And Android 4.1 helped improve things a lot more. But I still perceive the latency problem, though normal people complain a lot less than they used to. I would suggest trying ALmixer (my library). It is an SDL_mixer inspired API, but using OpenAL as the backend. I have special hooks to make it work better on Android and with OpenAL Soft. Here is a hello world program which you can experiment with the latency. (I really recommend Android 4.1+) https://bitbucket.org/ewing/hello-android-almixer Since the API is very close to SDL_mixer, if it works better for you, then it should be pretty easy to port to ALmixer. Thanks Eric -- Beginning iPhone Games Development http://playcontrol.net/iphonegamebook/ _______________________________________________ SDL mailing list http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org |
|||||||||||||
|
Mix_PlayChannel Android lag |
Jonny D
|
OP said that the sound lags 0.5 seconds. That's way more than just Android being awkward. I'm not getting that in my apps, but I don't think I can offer any suggestions right now beside the 1024 byte buffer size I use.
Jonny D |
|||||||||||
|
Mix_PlayChannel Android lag |
Eric Wing
Guest
|
On 4/3/14, Jonathan Dearborn wrote:
Pffft. 0.5 seconds is nothing on Android. I've seen Angry Birds have 3-5 seconds latency on Android devices. Believe me on this. Android audio is terrible. -Eric -- Beginning iPhone Games Development http://playcontrol.net/iphonegamebook/ _______________________________________________ SDL mailing list http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org |
|||||||||||||
|
Mix_PlayChannel Android lag |
Jonny D
|
That's ridiculous. And funny.
Jonny D On Thu, Apr 3, 2014 at 6:41 PM, Eric Wing wrote:
|
|||||||||||||||
|
Mix_PlayChannel Android lag |
Jared Maddox
Guest
|
And even if we were willing to use FUSE, 1) I'm going to guess that Android doesn't normally support it (at least for apps), and, 2) It would provide enough latency itself, right? _______________________________________________ SDL mailing list http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org |
|||||||||||||||
|
Mix_PlayChannel Android lag |
Eric Wing
Guest
|
Android doesn't support it as far as I know. I don't know enough about FUSE's permission requirements about whether it is compatible with Android's security model. But anything needs to be in userland and live in Android's app sandbox. My hunch is that FUSE won't work without rooting your device. But I don't think FUSE helps here. There are really 2 different problems which got entangled. The first problem is that all assets inside an apk are basically inside a zip, and the standard C ways of accessing a file are useless. You must use their special APIs to read anything (AssetManager). (And I have nightmare performance problem stories about this API too.) But the other problem is that you really want access to Android's native decoder system in my very strong opinion. You get multiple benefits if you use Android's decoder system: - Optimized for both battery life and performance on the platform - Protected under Google's patent umbrella protection for formats like AAC - Don't bloat your apk size with 3rd party codecs (and remember you need multiple versions for each architecture you support and some of these have complicated build systems that you have to port to Android...also remember that Android has a pathetic apk max size limit of like 50MB). - Also, so codecs are really hard to find commercial friendly, free 3rd party implementations for (AAC) Since I deal with both iOS and Android a lot, it turns out AAC is the best format for shipping. iOS doesn't support OGG. While you could bundle a Ogg Vorbis or Tremor decoder with iOS, Apple has optimized the heck out of Core Audio for both battery and performance (and even has on-board hardware decoders) and bundling Ogg yourself just doesn't reach that. And I saw my binary sizes go up. The next obvious format is MP3, but there are royalty obligations if you the app/game maker distribute an MP3. So that's out. AAC is the sequel to MP3 and doesn't make distributors pay royalties, so when looking at the end user experience of game makers and end-players, AAC looks like the best of bad options. (My world mostly deals with people who want to ship games, not fight the war against software patents.) Anyway, the only way to acces Android's native decoders is through their file or URI API. I think they did this partly because they conflated the AssetManager problem with the decoder API. (The abstraction leaked.)
This one I think you might be actually mitigate if you want to do the extra hoop jumping by prefetching data into memory if it was an issue. -Eric -- Beginning iPhone Games Development http://playcontrol.net/iphonegamebook/ _______________________________________________ SDL mailing list http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org |
|||||||||||||||
|
Mix_PlayChannel Android lag |
Jared Maddox
Guest
|
While I half-recall something about Android being ported to run on BSD or something (not that it matters, since FUSE has versions for more than just Linux), Android mostly runs on Linux, and Linux supports FUSE, so Android SUPPORTS FUSE... but FUSE requires a kernel module which Google apparently doesn't require, so... Vendor-dependent, in essence. Regardless, a brief search suggested that there are FUSE compiles for Android, so that part isn't the issue, the apparent lack of a requirement for it is the issue.
For at least some devices, yes. I suppose that we could try to figure out a test to see if FUSE is present, but would that really be worth the effort without a fall-back?
Wait, wait, wait... are you saying that the "best" API is only supported for resource files that you've stuffed into your package? Please tell me that I misread what you were trying to imply. If ordinary files ARE supported, and the API doesn't do something weird, then FUSE would almost guaranteed work IF you could run it... but if only packaged files are supported then it would be futile regardless.
I don't properly recall the email that I originally replied to, but from you mentioning this I'm guessing: the good-latency API doesn't interact with the Android audio decoder that you're talking about, does it?
Actually I think that sorta/kinda makes sense, what with cell-modem data-plan charges, but I'll concede that if I were implementing a packaging system I'd allow packages to provide files & folders to other packages, to be hard or soft linked into their "virtual package", so it also sorta doesn't make sense to me...
Unfortunately I think that this gets at the real fix that's required: someone needs to fix the Android implementation (from my perspective, preferable to a trivially pipeline-able memory-mapping-based system). That way we could get sensible latency on plenty of handsets in, oh, a few years. I'm thinking it's not going to happen from just us. _______________________________________________ SDL mailing list http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org |
|||||||||||||||||||||||||
|