Beats 1.7.5b and Android Fragmentation

So even though I wasn’t planning on it, I just published yet another minor update to Beats 1.x as Beats 1.7.5b. Changelog? Added Amazon Fire TV support (Beats-style only), add more speed multipliers, removed AdMob, and fixed a few crashes. Interesting, not-so-obvious question: What do all these changes have in common? Answer: They are all related to the problem of Android fragmentation. Actually, so was most/all of the changes in Beats 1.7.4b and 1.7.3b. So here’s a rant about why.

How fragmentation lead to Beats 1.7.5b

Lets look at the root causes of each of these changes and how they relate to Android fragmentation (we’ll get to that topic later in this post). Note that some of the exact version numbers and dates may be off since I’m recalling everything from memory, but the sediments and reasonings should be more or less accurate:

Amazon Fire TV

The Amazon Fire TV has two sources of input: the game controller and the remote. Note that a few years ago, Android did not support TVs, or non-touch/non-mouse devices for that matter. As such, Beats was developed with the assumption that a touchscreen was always available (in actuality, Beat 1.0a to 1.4b supported keyboard input, but when I revamped the graphic and touch input routines in Beats 1.5b, I dropped it because it was a pain maintaining and the Android emulator was getting more and more unbearable to test with). With the Amazon Fire TV (and supposedly more future mainstream TV devices running Android but without a touchscreen), that previously safe assumption was broken. Now while I could have just said “meh, no support, oh well”, one loyal user with an Amazon TV reached out and volunteered to help with testing. Plus, I was pretty curious myself despite having no plan on getting one.

So after doing some documentation reading and some ugly hacking around with the 2-year-old Beats source code, I got D-pad support working via Android emulator. Thank goodness I upgraded my computer a few months ago because I’m 99% sure my old laptop wouldn’t be able to handle the 4.4 Android emulator, even with Intel’s HAXM acceleration enabled. Of course this change doesn’t really add anything to the majority of my users (manufacturers have stopped releasing Android phones with D-pads/trackballs for quite some time) and I don’t plan on releasing Beats 1.7.x on the Amazon store (which is quite a lengthy, bureaucratic process), but it was nice to know about and just taught me to not be surprised when even the most basic of assumptions get broken in the Android world.

Speed multipliers

This is an interesting topic. When Beats were first being developed, there were no such thing as Android tablets. In fact, there wasn’t so much a mainstream consumer concept of a tablet at all (other than Wacom drawing tablets and extremely expensive pen tablets running Windows and used mainly by rich students/businessmen for taking notes). Thus, it was assumed (again, a relatively safe assumption back then) that phone physical screen sizes would more or less stay the same (maybe just increase in DPI). Given such, Beats’ speed multiplier was based purely on unscaled pixels (pixels per millisecond). The reasoning behind this was that, given two similar-sized devices held side-by-side, arrows would fall at the same pixel speed downwards (not a good reasoning today, but was the best choice back then). When Android tablets started popping up featuring Android 3.0, the speed multiplier algorithm was updated to multiply by the screen size category (Small, Normal, Large). And then Android introduced the concept of screen size vs screen DPI, and then it went downhill from there.

Suddenly, all sorts of combinations were popping up from various manufacturers. xxhdpi x Normal, mdpi x XLarge, hdpi x Large, etc.. Any previous assumption about the display dimensions and physical size had been broken. I could have easily changed the speed multiplier logic to use scaled pixels instead of unscaled pixels or done something related to the physical size of the device’s display, but the last time I made a speed-multiplier-related change (in Beats 1.6b I think?), users were confused and massively complained so I reverted the change. Recently, some of the new Samsung Galaxy Tabs released have huge 10.1″ screens and are running hdpi or even xhdpi resolutions, meaning unscaled pixels per millisecond would be slow. Not wanting to deal with the massive user confusion/complaints about changes to the fundamental mechanics of the game, I decided to just throw in a few speed multipliers. Next time around, I’ll know better during the designing phase.

AdMob

I added AdMob integration early on back before it had been purchased by Google. The initial reasons I added AdMob integration were: 1) it was really easy to do, 2) it helped me learn about integrating third party plugins and APIs, and 3) it kept Beats free and gave me a bit of pocket cash for all the work I put in (as much as I hated ads, I was a still a poor college student). When Google bought AdMob, they promised they would keep the service as is and maintain it. Which they did nicely for the next 4 years. Until very recently when they dropped the “upgrade to our new Google Mobile Ads SDK or else” bomb. And so I said why not and as part of my 1.7.3b update. But then in came the crash reports and user complaints. “App crashes on launch”, “app blackscreens with no response”, etc. Previously, AdMob was an independent library (a simple .jar that gets compiled in), but the new Google Mobile Ads plugin communicates with Google’s system services, which is supposed to be implemented and running on all Google-approved Android devices. Except it looks like it wasn’t, or at least not cleanly.

Looking through the crash traces, looks like a lot of random devices were getting permission or access errors relating to Google services, especially older devices. Searching around didn’t do much help, so I just threw the AdMob initialization code in a simple try-catch-swallow and that worked in Beats 1.7.4b. For the most part. Turns out, many Android 2.2 devices don’t even have such a kind of service, so I’d actually have to remove the references from the layout XMLs. Not wanting to deal with so much forking, I decided to just completely remove AdMob in the new Beats 1.7.5b. No more ads finally, yay! Though that does lead to the despairing question, why are people still using Android 2.2 phones… (more on that later in the Beats usage stats breakdown section).

Hardware acceleration

During the initial development of Beats 1.0a, Matt and I had a brief discussion on whether or not to use Android’s native Canvas drawing routines for our custom graphics or to use the (mostly incomplete at that time) OpenGL drawing routines. Given that this was during a hackathon and the documentation for Android’s OpenGL ES port was very, very lacking, we opted for native Canvas drawing on a simple blank View which we know would work. Far from optimal, but okay given the simple graphic demands of Beats. And it did for a good long while and stayed that way until Beats 1.5b. Sometime before 1.5b, manufacturers started releasing new devices with larger screens and I started noticing significant performance hits with drawing on a standard View (ie. under 30fps). Digging around the (very lacking) documentation, I discovered that some of the SDK samples instead used SurfaceView for their drawing. I tried it out and suddenly, bam, a solid 50+fps everywhere.

It was great and I thought that’d be the end of it. But then came Android 3.1 Honeycomb, where hardware-acceleration support for drawing was introduced (phones and tablets were starting to get dedicated GPUs, gasp!). OpenGL ES drawing would gain the most benefits but GPU acceleration was also promised to be ported over to standard views as well (I was also in no rush to yet again rewrite all the graphics-related code, which had become quite an unrecognizable mess by that time). Testing on the few tablets I could get my hands on (eventually bought a Samsung Galaxy Tab 7″ just for this), Beats still ran reasonably well (~45fps), so I figured I’d just go and do nothing, as I’ll eventually get GPU acceleration for free.

Turns out, this would never happen. I kinda got more and more surprised when I started hearing users report that they were getting low framerates on the Samsung Galaxy S4 (top-tier phone at that time with a powerful GPU), but I didn’t have access to any of those devices so didn’t really act upon it. Only recently when I migrated Beats’ source code to Github did I decide to take a closer look and was quite surprised. Indeed, Beats was struggling to run at 30fps on the Galaxy S4 even with multiple settings turned off, and there was definitely no CPU bottleneck anywhere. Profiling told me that the problem lied in the SurfaceHolder.lockCanvas() which was returning extremely delayed. I was expecting a slightly slower response due to the larger screen compared to my older smaller screened phones, but not on the magnitude of a few hundred milliseconds.

So once again, I dug through the documentation, only to discover that apparently Google had no plans on supporting hardware acceleration for SurfaceView. So funny enough, the old slower regular View, if hardware accelerated, is now faster than the non-accelerated SurfaceView on some devices. And so, in Beats 1.7.3b, I added a new setting which would check if hardware acceleration support was available on regular Views and if yes, use the old pre-Beats 1.5b code instead of SurfaceView.

That fixed things for most devices, but I suddenly started noticing a new never-before-seen crash: “java.lang.UnsupportedOperationException in android.view.GLES20Canvas.clipPath”. Turns out, the clipPath call used for Hold notes is not guaranteed to be supported with hardware acceleration up until Android 4.3 (API 18), and I definitely saw it working on some less-than-API-18 devices I tested on. Oh, yet another case of inconsistency/fragmentation across devices. And so in Beats 1.7.5b, I threw in a simple try-catch clipPath check in the hardware acceleration setting logic. Far from a proper, clean solution, but keep in mind Beats 1.7.5b was never planned to happen in the first place…

How bad is this Android fragmentation?

Simply put, it’s still pretty bad. As of the moment, we are currently on Android 4.4 (API 19), with Android 5.0 on the horizon. That’s an entire 19 API updates, covering 9 different OS names (ignoring Android 1.0 and 1.1). Beats was first started back in 2010 when Android 1.6 was still the most common Android OS version and Android 2.2 was just starting to appear on new phones. The initial release of Beats supported Android 1.5 (API 3) and up. The normal trend with version numbers of actively developed software is that of a steady upward trend, with most users on the latest versions and little or no users on the earlier versions. Except in reality, this is not the case.

OpenSignal did a great numerical analysis and report of the Android fragmentation issue using real data collected by their apps. Below are some screenshots of their July 2013 report, alongside some of Beats 1.7.x’s data from Localytics and Google Play.

Devices

“Don’t worry, lets just pick the top 5 most popular Android devices and we should be good!” Think again. A general good rule of thumb when it comes to software development is to go to where the users are. Build, target, and optimize for the most popular configurations such that the majority of users will have the best experience. Once you’ve chosen your targets, optimizing and testing becomes much easier. Unless your targets don’t actually represent the majority. Which happens to be the case for the Android ecosystem. Even totaling together the top 10 most popular Android devices will just barely cover a quarter of the market, with the top device (the Samsung Galaxy S3) not even accounting for a double digit percentage.

For Beats 1.7.x, it just barely reached 5%. You can try to be “smart” with your strategy all you want, but at the end of the day, there’s really nothing much you can do other than “just throw more money at it”. Unfortunately, for indie developers like myself, that isn’t an option. With only an LG Spectrum 2 in one hand (not-at-all popular, low-tier phone) and a Samsung Galaxy Tab 7.0 (a first generation Android tablet that struggles with the web browser) in the other, I can only blindly hope my changes and bugfixes work every time I push out a new release. Either way, if you were previously dreaming of a world where your app runs perfectly on all these hundreds of different devices, I’d recommend you give up now. I did at least, for now…

Manufacturers

For Android, brand usually implies system-app level changes and UX changes. Each manufacturer and carrier will do their best to push onto users their unique/modified version of every core app (think email, messenger, photo gallery, camera, etc.) that may not always adhere to the same expected contracts that the stock apps follow. As you can imagine, trying to call intents to and from other system apps can be a pain when you have no guarantee what the resulting behaviour will be (works, fails, or maybe even a crash).

But looking at the chart, it actually isn’t all that bad. In fact, brand fragmentation is the kind of fragmentation you’ll be most thankful of (not being as bad as the others). Over half the market runs Samsung hardware and Samsung hardware. That means you can expect some kind of standard set of hardware features for at least 50% of your users and also that 50% of your users are running some butchered version of Android bloated with TouchWiz. Well, 50% is better than <10% at least. Thankfully, Beats doesn’t really have any external requirements (i.e it’s more or less a standalone app that doesn’t interact much with other system apps) so I actually haven’t run into this issue very often. That said, just the fact that I don’t own a Samsung phone puts me in an annoying hard spot, especially when I see comments like “why does Beats suck so much on Samsung device XXX?” (which was the case with the Galaxy S4 and lead to the hardware acceleration change above).

OS Versions

 

OS fragmentation is arguably an Android developer’s worst nightmare. Most bugs/changes are due to changes/discrepancies between Android OS versions. One bug in one OS version might get fixed in the next version, and one crucial feature might only be available on the latest OS version. If you are developing for iOS, great! Over 90% of your users are running the lastest OS, so you can rest easy and target that, knowing that eventually those on older versions will upgrade (or just not care enough and won’t actively use your app anyway).

But Android is a completely different story. Over the past 5 years that Android has been in the market, There have been a number of distinct “eras” – Android 1.0/1.1 (no codename), Android 1.5/1.6 (Cupcake/Donut), Android 2.1/2.2 (Eclair/Froyo), Android 2.3.x (Gingerbread), Android 3.x (Honeycomb), and Android 4.x (Ice Cream Sandwich, Jelly Bean, KitKat). Each of these eras signified drastically different feature changes as well as hardware changes: Cupcake/Donut was a hybrid of old dumbphones with QWERTY keyboards and trackballs, Eclair/Froyo was the first real generation of touchscreen-only phones, Gingerbread was the start of real “smartphones”, Honeycomb was the introduction of tablets, and ICS/etc. was the start of the unification of phones, tablets and recently “phablets”.

So logically, everyone should be running 4.x right? Unfortunately, wrong. With Android 4.x came the introduction of minimum hardware requirements (finally, thank you Google!) and a list of minimum features requirements. Tablets were still relatively recent and met hardware requirements so most got 4.x upgrades. For phones however, instead of bumping up the standard, there was a long period where manufacturers opted to keep releasing new devices with older OS versions (most likely due to cost reasons) and at that time, Android 2.3.x was still the “golden” OS that worked and worked well. Not to mention, unlike iPhone users who happily throw money at the newest generation iPhone every year, Android users were more or less happy with what they had: a working phone running a solid OS that could run apps and, you know, make phone calls.

There was no reason to throw away perfectly fine hardware, and that’s what many users did. In fact, a good 20+% of them, who even today continue actively using their phone and play Beats 1.7.x. That percentage will never increase of course and will only go down over time, but as OpenSignal’s trend graph shows, it won’t be disappearing any time soon. Which sucks, because both my devices run 4.x either – Android 4.1.2 stock on the LG Spectrum 2 and CyanogenMod 9 (based on Android 4.0.4) on mthe Samsung Galaxy Tab 7.0. That aside, the fact that there are still Beats users running Android 1.5/1.6/2.1/2.2 worries me… If you are one of those people, please please consider upgrading your OS (or installing CyanogenMod if possible) or buying a new phone.

The result?

Now Android fragmentation has long been (and still is) a frustrating challenge to developers. And actually, it’s not just an issue for small indie apps like Beats that only get an update here and there whenever I happen to have the free time. It’s a major maintainability and costing issue for any company developing mobile apps for Android. Earlier this year, my team at work made the decision to raise the minimum OS requirement to Android 4.0.3 (API 15) because the cost of testing and backporting new features was far too great. And to be honest, even though the percentage of Android 2.x users out in the market is still a significant 20+%, it’s a declining percentage. Sure, one can throw more money at the problem and buy more devices or hire more developers to ensure all features work nice and smooth on that dying 4-year old Android phone running Froyo or Gingerbread, but is it really worth it? Why not just spend that effort focusing on adding more new features or whatnot? For an indie developer like myself who works on Beats alone and doesn’t even own any Android device under 4.0, it makes even less sense. The trade-off just isn’t worth it.

So does that mean I’ll be dropping Android 2.x support for Beats 1.7.x? No, simply because it already mostly works and I’d hate to be a jerk to that loyal 25% of my users. I recently dropped support for Android 1.6 (API 4) when upgraded to the newest set of Android SDK+tools package – there were so many errors and warnings that I honestly didn’t want to spend the next 2-3 hours debugging legacy code from Beats 1.2b or so. After release, I did pay extra attention to feedback but seemingly no one complained, so it turned out it was a safe drop (though that worry really never should have been there in the first place).

What this does mean, however, is that with Beats2, I’m going to start being a lot more strict on minimum requirements and have them set in stone so that legacy support issues and bugfixes like Beats 1.7.5b won’t have to happen in the future. For the time being, I’m still okay with supporting Android 2.3.4 and up (the Unity 3 game engine actually declares that it supports Android 2.0 and up but I’m skeptical), but if things turn out troublesome during the initial beta testing phase, expect me to bump that requirement up to Android 4.1.2. As a single indie developer working on a game during his spare time and (ambitiously) targeting multiple platforms (Android, Windows touch+mouse+keyboard and eventually iOS and Windows Phone), sometimes it’s just better to bite the bullet and fight against the market.

~Keripo

BeatsPortable: http://beatsportable.com/2014/04/beats-1-7-5b-and-android-fragmentation/