Category Archives: R

Installing R and Shiny on Cloud 9

I wanted to see if I could run an instance of RStudio’s Shiny server on Cloud 9 and after a bit of finagling with the right set of steps to get it set up I have an instance running. Here’s what I did starting from a workspace using the stock HTML template.

strefethen:~/workspace/ $ sudo sh -c 'echo "deb http://cran.rstudio.com/bin/linux/ubuntu trusty/" >> /etc/apt/sources.list
strefethen:~/workspace/ $ gpg --keyserver keyserver.ubuntu.com --recv-key E084DAB9
strefethen:~/workspace/ $ sudo apt-get update
strefethen:~/workspace/ $ sudo su - \
-c "R -e \"install.packages('shiny', repos='https://cran.rstudio.com/')\""
strefethen:~/workspace/ $ sudo su - -c "R -e \"install.packages('shiny', repos='https://cran.rstudio.com/')\""
strefethen:/etc/shiny-server $ sudo shiny-server
[2016-09-02 02:25:04.460] [INFO] shiny-server - Shiny Server v1.4.4.801 (Node.js v0.10.46)
[2016-09-02 02:25:04.463] [INFO] shiny-server - Using config file "/etc/shiny-server/shiny-server.conf"
[2016-09-02 02:25:04.535] [WARN] shiny-server - Running as root unnecessarily is a security risk! You could be running more securely as non-root.
[2016-09-02 02:25:04.539] [INFO] shiny-server - Starting listener on 0.0.0.0:8081

I need to look into the user “shiny” to see about fixing the above warning. Then edit /etc/shiny-server/shiny-server.conf and change the port from 3838 -> 8081 so Cloud 9 will server the content and start the server:

sudo shiny-server

Browse to http://<project-name>-<username>.c9users.io:8081/ and you should see:

Default Shiny app running on Cloud 9

From Google Sheets to an Automated Shiny Dashboard (Part I)

Over the past several months I’ve been building what’s become a large Google Sheet (which is now at the 2M cell limit) tracking Find&Save Cash Dash offers. Like a lot of data projects this one started out small but over many months grew to a point where it no longer makes sense to cut/paste SQL data into a Google Sheet for pivot tables and charts. I wanted a solution that could pull data from REST APIs, MSSQL and Postgres and present data in an internal dashboard. Unfortunately, Google Sheet’s JDBC support doesn’t include Postgres otherwise Google Apps Script might have been a viable choice for automating data collection notwithstanding size limitations. Finally, another issue with Sheets was that I couldn’t seem to get pivot tables to resize automatically as more data was added. So, 3 strikes and Google Sheets was out.

Continue reading From Google Sheets to an Automated Shiny Dashboard (Part I)

Plotting Weekly Mobile Retention from the Localytics API using R and ggplot2

Part of building mobile web apps is understanding the myriad of mobile analytics and in part visualizing the data to shed light on trends that my otherwise be difficult to see in tabular data or even a colorful cohort table. I’ve been building a dashboard using R, RStudio, Shiny, and Shiny Dashboards aggregating data from MSSQL, Postgres, Google Analytics, and Localytics.

Within the Product section of the dashboard I’ve included a retention chart and found some great articles at R-Blogger like this one. The retention data comes from the Localytics API which I discussed previously though getting the data into the proper format took a few steps. Let’s start with the data, here’s an example of the REST response from Localytics looks like for a weekly retention cohort:

{
"results": [
{
"birth_week": "2014-09-08",
"users": 1,
"week": "2014-12-29"
},
{
"birth_week": "2014-09-29",
"users": 1640,
"week": "2014-12-29"
},
{
"birth_week": "2014-10-06",
"users": 2988,
"week": "2014-12-29"
},
{
"birth_week": "2014-10-13",
"users": 4747,
"week": "2014-12-29"
},
{
"birth_week": "2014-10-20",
"users": 2443,
"week": "2014-12-29"
},
…

Below is the main function to fetch the Localytics sample data and convert it into a data frame that’s suitable for plotting. Now, admittedly I’m not an R expert so there may well be better ways to slice this JSON response but this is a fairly straight forward approach. Essentially, this fetches the data, converts it from JSON to an R object, extracts the weeks, preallocates a matrix and then iterates over the data filling the matrix to build a data frame.

retentionDF <- function() {
  # Example data from: http://docs.localytics.com/dev/query-api.html#query-api-example-users-by-week-and-birth_week
  localyticsExampleJSON <- getURL('https://gist.githubusercontent.com/strefethen/180efcc1ecda6a02b1351418e95d0a29/raw/1ad93c22488e48b5e62b017dc5428765c5c3ba0f/localyticsexampledata.json')
  cohort <- fromJSON(localyticsExampleJSON)
  weeks <- unique(cohort$results$week)
  numweeks <- length(weeks)
  # Take the JSON response and convert it to a retention matrix (all numeric for easy conversion to a dataframe) like so:
  # Weekly.Cohort Users Week.1
  # 1    2014-12-29  7187   4558
  # 2    2015-01-05  5066     NA
  i <- 1
  # Create a matrix big enough to hold all of the data
  m <- matrix(nrow=numweeks, ncol=numweeks + 1)
  for (week in weeks) {
    # Get data for all weeks of this cohort
    d <- cohort$results[cohort$results$birth_week==week,][,2]
    lencohort <- length(d)
    for (n in 1:lencohort) {
      # Skip the first column using "+ 1" below which will be Weekly.Cohort (date)
      m[i,n + 1] <- d[n]
    }
    i <- i + 1
  }
  # Convert matrix to a dataframe
  df <- as.data.frame(m)
  # Set values of the first column to the cohort dates
  df$V1 <- weeks
  # Set the column names accordingly
  colnames(df) <- c("Weekly.Cohort", "Users", paste0("Week.", rep(1:(numweeks-1))))
  return(df)
}

To make things easy I put together a gist and if you’re using R you can runGist it yourself. It requires several other packages so be sure to check the sources in case you’re missing any.
Fair warning the Localytics API demo has very limited data so the chart, let’s just say simplistic however given many weeks worth of data it will fill out nicely (see example below).

> library(shiny)
> runGist("180efcc1ecda6a02b1351418e95d0a29")
Localytics Retention Plot Example
Retention Chart

Fetching Localytics API Data in R and working with the JSON result

Recently, I’ve been building a dashboard for Find&Save using R, RStudio and Shiny Dashboards and one of our core mobile metrics is 30 day active users for both iOS and Android which I can get from Localytics (our analytics platform) via their API. Googling around I found I’m not reinventing the wheel (via SO) though I did run into a problem where the response failed to convert properly thus the post.
Here’s the original function from SO with some slight tweaks for my needs:
Continue reading Fetching Localytics API Data in R and working with the JSON result