TV App Engagement : Beyond Download Counts

Those of us in the TV App space would like to understand the extent and trajectory of user interest in TV apps. A positive trajectory might indicate early excitement maturing into a user habit, upon which an industry can then exist. Unfortunately, app downloads (especially in a free app ecosystem) indicate awareness, but fall short of calibrating involvement. All of us who download apps are familiar with the “use once, forget forever” set in our Apps folders. More nuanced engagement studies around dwell time or biometrics are closely guarded secrets, revealed on an uneven basis across the App Ecosystem. The question is – can we do better than downloads in calibrating App Engagement across the TV App Ecosystem, using marketplace data that is commonly available for most apps?

One simple approach is to compute normalized ratings (i.e. reviews as a fraction of total downloads). The intuition is that people are more persuaded to take the trouble to rate/review an app if it is interesting. And if it is interesting, they probably use it more. Below, I’ve calculated a Figure of Merit (1000*#Reviews/#Downloads) and its associated behaviors.

A summary of the Figure of Merit data  (ignoring ‘tip of tail’ apps with less than 5000 downloads and less than 50 reviews) yields the following : 

    • Average Figure of Merit – 5.25 (i.e on average about 5 reviews per 1000 downloads)
    • Average TV App Rating – 4.1
    • Average App Ratings Count – 1250

The middle 80% of apps get between 2 and 20 reviews per 1000 downloads, with a distribution that looks something like the below.

FOM10to90

As one goes to either end of the distribution, the very popular apps get an order of magnitude more engagement (and conversely on the long tail)

FOM5to95

Overall, the use of this Figure of Merit (normalized ratings) is moderately useful, and generally correlate with intuitive notions of the quality of the Apps. Qualitative observations based on Figure of Merit data include :

  • Global Brand + App Strategy does well as a pair – 50% of apps in the top 25 on the FOM scale (and FOM numbers ranging from 15 to 200) are either global brands or brands with strong local presences (e.g.  media companies of note in Sweden, Vietnam and India).
  • Brand awareness doesn’t save bad apps –  There are a number of well known content brands that have put out apps without a discernable content strategy or user benefit. These apps garner downloads .. and user disappointment. 42  out of the 73  apps with over 250K downloads have below average ratings.  8 (of those 42) are from global media/entertainment companies, have over 250K downloads and a rating of well under 3 (compared to a median rating of 4.1). So, if you are a large and well known brand and put out an app that ‘snookers’ people – people will take the trouble to publicly call you out.
  • Sports has a natural engagement advantage Sports TV Apps score marginally higher on average ratings (4.25 vs 4.1) but about 25% higher in terms of the average Figure of Merit score. Thus people are more vocal (and generally more positive) about Sports Apps.

The State of TV (& 2nd Screen) Apps : Android

Thinking. Folks (including me) have been talking about the Appification of TV via second screen TV apps for sometime now. This expectation has led to a plethora of announced and implied corporate second screen strategies. No one seems to have put a number on how many of these announcements are backed up by concrete apps, and how those apps are doing. This is a ‘start small’ exercise towards putting some numbers to that picture.

The simple exercise here is to collect and quantify app data in the Android marketplace by doing a bit of marketplace scraping  and some R ‘data wrangling’ to understand  the Google Play TV App landscape.

The Data at First Blush. To my knowledge there are no public Java API’s to datamine Google Play, but thanks to the android market api, and some tediousness – one gets the following preliminary result. There are about 500 (543 being the exact result returned by the API) TV apps in Android. Their breakdown in terms of categories, and the relative distribution of app downloads is shown in the pie charts below.

Image

Some of the more interesting observations here are:

  • Equal distributions of apps self-categorizing as Media, Entertainment & Games. Games is the Rodney Dangerfield category – with the exception of Mark Suster, no one else in the community has called out Games as a category deserving of respect, and perhaps a disproportionate amount of investment.
  • A 10-15% representation of Sport? – both surprising and not so surprising. Sports Apps aren’t easy to write, but if written well they find a ready and enthusiastic audience.
  • Now to the downloads. If one thinks of 100K or less downloads as the ‘poverty line’ (i.e. no amount of cleverness can lead you to a lucrative $ number in terms of app monetization), about 66% of TV Apps live below the poverty line.

Speculation. So why is the data the way it is? A few theories:

  • Why so many games? Because independent publishers can create compelling (largely textual) experiences even without access to copyrighted TV related content — that is assuming a Twitter future that is still somewhat ‘open access’. 
  • Why the relative paucity of apps? 500 is a lot better than the 5 interactive TV applications that was the recent past, but a disproportionately small proportion of the app space. Why – because good apps need good content. On mobile, the content is the user + web services, for TV Apps – the content is (copyrighted) TV.
  • Why the paucity of downloads? Because a TV show is currently the best way of getting a TV app discovered, and not every app developer owns a TV show. Advances in the App Discovery space could go a long way in making the download picture less bleak. A product punch line from Apple’s acquisition of Chomp, and more activity around TV App containers could change the picture rather quickly.

Unfinished Business. There’s a bunch of stuff I haven’t covered here (left for future little experiments). The state of TV Apps on iOS. A monetization argument for why I consider 100K downloads the poverty line. Extent of replication of capability (e.g. TV guide) across geographies. App property variance across large studios vs small developers. And several other topics.

Fine Print. 

  1. The Android Market scraping isn’t foolproof – due to its limited query capability. True – however, 550 is a large enough sample size that I would posit it to mirror the actual TV app population in terms of statistical behavior, even if the actual population size is off by a bit.
  2. Google Play isn’t 100% of the Android market. It may not include the dark matter (other marketplaces, or direct downloads from large publishers). But for our purposes, it’s close enough.
  3. Why focus on downloads, after all downloads do not equal engagement? – It’s true that downloads may not imply dwell time. But lack of downloads is likely to imply lack of engagement with an App, which is the pertinent point here.