Category Archives: Programming

OAuth 2.0 reference for Facebook development

As I mentioned, I’m working on Facebook support which involves the transition to the new $g(OAuth 2) authentication. What I’m going to do here, for myself if no one else, is keep a list of links I’ve found related to the topic. Without further adieu:

Of course, feel free to leave a comment with anything useful you’ve found.

Improving the Facebook Python SDK GraphAPI.request method

While working on Facebook functionality using Python I ran into a few cases where the GraphAPI.request method caused Facebook to choke on parameters with a value of None. Thus here’s a simple override to allow for None parameters which subsequently get stripped out.

def request(self, path, args=None, post_args=None):
    ''' Improves handling for post_args where any arg with a None value gets removed to avoid FB API errors.
        Allows for methods that accept all possible Facebook parameters and only passes those that are specified.
    '''
    if post_args:
        d = {}
        for a in post_args:
            if post_args[a] != None:
                d.setdefault(a, post_args[a])
        post_args = d
    return GraphAPI.request(self, path, args, post_args)

I’ve created a descendant class I call GraphAPIEx where this above method appears.

Using flot to chart data using a date range

jquery flot chart I’m working on developing Bloglines these days and one of the features I wanted as an admin was the ability to see various pieces of data related to the site over time. Ideally what I wanted were some charts like those on $g(Google Analytics) for things like:

  • users joining/day
  • votes cast/day
  • blogs submitted per day

I’ve previously experimented with the $l(Google Chart Tools) and that’s where my search started but that lead to a bit of a dead end. I found a related post on Tom Fotherby’s blog but as Greg Fitzgerald pointed out there are still a few more issues to be worked out. Since we use jQuery on the site I started searching along that vein which led me to flot and more specifically this example (caution the examples site seems really slow) which fit perfectly.

Looking at the code and markup I quickly found this was something I could have working right away. The data in JavaScript looks like this:

var d = [[1196463600000, 0], [1196550000000, 0], [1196636400000, 0], ...];
 

Where the first value is a $g(JavaScript timestamp) and the second is the actual data to be plotted. There are a couple of notes regarding the timestamp mentioned here:

The timestamps must be specified as Javascript timestamps, as milliseconds since January 1, 1970 00:00. This is like Unix timestamps, but in milliseconds instead of seconds (remember to multiply with 1000!).

As an extra caveat, the timestamps are interpreted according to UTC to avoid having the graph shift with each visitor’s local time zone. So you might have to add your local time zone offset to the timestamps or simply pretend that the data was produced in UTC instead of your local time zone.

We’re using $g(Postgres) and the tables I need to query all have a date_created field of timestamp without timezone. Here’s the SQL to fetch the data:

SELECT extract(epoch from date_trunc('day', date_created)) * 1000, count(*) from blog group by extract(epoch from date_trunc('day', date_created)) * 1000 order by extract(epoch from date_trunc('day', date_created)) * 1000 DESC

From the results I use the following python method to create the above data structure which is ready to feed into the flot chart:

    @staticmethod
    def statsByDay(query):
        data = PubBase.sqlQuery(query)
        dl = []
        for d in data:
            if d[0] != None and d[1] != None:
                dl.append([int(d[0]), int(d[1])])
        return dl         

I’ve been working with a lot of JavaScript frameworks/libraries lately and this is on that integrated into the project pretty flawlessly so kudos to the flot folks!

Using the Dealmap API from Python

Update Aug 1, 2011:With Google’s acquisition of Dealmap one would have to assume this will become a Google API. The deal raises a number of interesting questions related to how deals would be served to specific clients. Should be interesting to watch.

Update Jun 11, 2011: I made one more tweak I neglected to mention to the Dealmap python API which I highly recommend if you intend to use it in a production environment which is to modify the call to urllib2.urlopen and add the timeout parameter. In fact, in my revision I’ve modified all of the calls into the API to accept a timeout parameter which is passed along to the urllib2.urlopen call.

Continue reading

Getting the Tornado Facebook demo working

Recently, I downloaded Tornado:

Tornado is an open source version of the scalable, non-blocking web server and tools that power FriendFeed.

Included is a facebook demo that displays your feed though it didn’t work “out-of-the-box” raising the error:

File "modules/post.html", line 26, in _execute
KeyError: 'message'

modules/post.html is a snippet of html used to render each feed item. The snippet looks like this:

<div class="post">
  <div class="picture">
    {% set author_url="http://www.facebook.com/profile.php?id=" + escape(post["from"]["id"]) %}
    <a href="{{ author_url }}"><img src="//graph.facebook.com/{{ escape(post["from"]["id"]) }}/picture?type=square"/></a>
  </div>
  <div class="body">
    <a href="{{ author_url }}" class="actor">{{ escape(post["from"]["name"]) }}&lt;/a>
    {% if post["message"] %}
      <span class="message">{{ escape(post["message"]) }}</span>
    {% end %}
    <div class="meta">
      <a href="{{ escape(post["actions"][0]["link"]) }}" class="permalink">{{ locale.format_date(datetime.datetime.strptime(post["created_time"], "%Y-%m-%dT%H:%M:%S+0000")) }}</a>
    </div>
  </div>
</div>

The curly braces are python code snippets and the line reading if post[“message”] is causing the problem. In python referencing a non-existent dictionary key raises a KeyError exception thus changing the code to use:

{% if “message” in post %}

To debug this problem I simply removed the code referencing post[“message”] and replaced it with str(post) which simply dumped the JSON contents onto the page. From there I could see that not all entries had a message key thus the error.

One more note. Looking at the documentation Tornado uses a library call epoll which at first glance sounds familiar to I/O completion ports on Windows which is something I’m familiar with.

FileSystemInfo.LastWriteTime and 12/31/1600

I’ve been working on an application tracking the age of files and if they reaches a certain threshold an error gets trigged.

FileInfo f = new FileInfo(new SystemPath(m.FolderName).Combine(m.FileName).ToString());
double totalminutes = DateTime.Now.Subtract(f.LastWriteTime).TotalMinutes;
if (totalminutes >= ErrorIntervalInMinutes)

I setup an error message to display information about the files when the error occurs and got something like this:

Error: File (d:\outbound\997_42772_06182010_1504_91.txt) File Time: 12/31/1600 4:00:00 PM Current Time: 6/18/2010 3:06:10 PM has failed to upload via FTP for 215356266.168919 minutes.

Whoa, over 400 years! That’s a lot of minutes not to mention 12/31/1600 looks a bit suspicious. A peek at the documentation for FileInfo.LastWriteTime reveals:

If the file described in the FileSystemInfo object does not exist, this property will return 12:00 midnight, January 1, 1601 A.D. (C.E.) Coordinated Universal Time (UTC), adjusted to local time. 

Ah ha! The list of files being processed in this case is static and files are getting uploaded to an FTP server so clearly the file causing the error is no longer on disk thus the error. Adding an if(!f.Exists) continue; should do the trick.

Now, I didn’t investigate UTC once converted to local time resulting in 12/31/1600 vs. 1/1/1601 but I found my answer and after adding a simple check I was on my way.

Google Calendar on a Simile Timeline

At Falafel Software Google calendar is a core tool used by the company. I’ve been working to leverage the data calendar in a number of interesting ways such as in the DIV below rendered using JavaScript based on calendar data from Falafel training calendar displaying upcoming training events over the next 120 days.

February 2

Telerik Advanced RadGrid Data Editing and Validation
FREE! (Click to register)

February 23

Telerik Advanced RadGrid Paging, Sorting, and Filtering
FREE! (Click to register)

March 1

Telerik Sitefinity Online Training
$399/person

I’ve mentioned the above example before from Google’s playground and I found another example which uses $g(MIT’s Simile) project Timeline control to render calendar data. As you can see from that example it’s not actually working so I thought I’d post an update with some changes I made to get it working. Here is my example. I’ve made changes to get the import of the data working though there remains a number of other improvements that could be made. I’ve yet to determine if this timeline view is really all that useful and haven’t settled on a view that I really like. At any rate, here is the code, again take a look at the source for my example above for a complete working page.

var gEventSource;
 
function loadGDataCallback(json) {
  var entries = json.feed.entry;
  var timelinerEntries = [];
  for (var i = 0; i < entries.length; ++i) {
    var entry = entries[i];
    if(entry["gd$when"] == null) continue;
    var when = entry["gd$when"][0];
    var start = convertFromGDataDate(when.startTime);
    var end = convertFromGDataDate(when.endTime);
    var webContent = null;
    var links = entry.link;
    for (var j = 0; j < links.length; ++j) {
      if (links[j].rel == "alternate") {
        webContent = links[j];
        break;
      }
    }
    var title = entry.title.$t;
    var link = entry.link;
    var icon = entry.link[0].href;

    var description = '<img src="' + link + '">';
    timelinerEntries.push(new Timeline.DefaultEventSource.Event(
      start,
      end, // end - when not set, event displayed with icon (looks better)
      null, // latestStart
      null, // latestEnd
      false, // not isDuration
      entry.title.$t,
      entry.content.$t,
      icon, // image
      entry.link[0].href, // link - destination when clicking on title
      entry.link[0].href,
      undefined, // color
      undefined  // textColor
    ));
  }
  gEventSource.addMany(timelinerEntries);
};
 
function zeroPad(n) {
  if (n < 0) throw new Error('n is negative');
  return (n < 10) ? '0' + n : n;
}
 
function convertToGDataDate(/*Date*/ date) {
  return date.getFullYear() + '-' +
         zeroPad(date.getMonth() + 1) + '-' +
         zeroPad(date.getDate());
}
 
function convertFromGDataDate(/*string<YYYY-MM-DD>*/ date) {
  var match = date.match(/(\d{4})-(\d{2})-(\d{2})/);
  return new Date(parseInt(match[1], 10), parseInt(match[2], 10) - 1, parseInt(match[3], 10));
}
 
function onLoad() {
  gEventSource = new Timeline.DefaultEventSource();
 
  var theme = Timeline.ClassicTheme.create();
  theme.event.bubble.width = 320;
  theme.event.bubble.height = 180;
 
  // centering the timeline three months previous makes it look nicer on load
  var threeDaysFromNow =
      new Date(((new Date).getTime()) + 3 * 24 * 60 * 60 * 1000);
  var bandInfos = [
    Timeline.createBandInfo({
        eventSource:    gEventSource,
        date:           threeDaysFromNow,
        width:          "40%", 
        intervalUnit:   Timeline.DateTime.WEEK, 
        intervalPixels: 300,
        theme:          theme
    }),
    Timeline.createBandInfo({
        eventSource:    gEventSource,
        date:           threeDaysFromNow,
        width:          "60%", 
        intervalUnit:   Timeline.DateTime.MONTH, 
        intervalPixels: 550,
        theme:          theme
    })
/*    ,
    Timeline.createBandInfo({
        showEventText:  false,
        trackHeight:    0.5,
        trackGap:       0.2,
        eventSource:    gEventSource,
        date:           threeDaysFromNow,
        width:          "10%", 
        intervalUnit:   Timeline.DateTime.YEAR, 
        intervalPixels: 200
    })
*/    
  ];
  bandInfos[1].syncWith = 0;
  bandInfos[1].highlight = true;
/*  
  bandInfos[2].syncWith = 0;
  bandInfos[2].highlight = true;
*/  
  tl = Timeline.create(document.getElementById("my-timeline"), bandInfos);
 
  // Atom feed from a Google calendar
  var feedUrl = "http://www.google.com/calendar/feeds/4u13qtkl3bcpao0ofgod94s960@group.calendar.google.com/public/full";
 
  var startDate = new Date((new Date).getDate());
  var endDate = new Date(((new Date).getTime()) + 3 * 30 * 24 * 60 * 60 * 1000);

 
  var getParams = '?start-min=' + convertToGDataDate(startDate) +
                  '&start-max=' + convertToGDataDate(endDate) +
                  '&alt=json-in-script' +
                  '&callback=loadGDataCallback' +
                  '&max-results=5000'; // choose 5000 as an arbitrarily large number
  feedUrl += getParams;
  var scriptTag = document.createElement('script');
  scriptTag.src = feedUrl;
  document.body.appendChild(scriptTag);
}
 
var resizeTimerID = null;
function onResize() {
    if (resizeTimerID == null) {
        resizeTimerID = window.setTimeout(function() {
            resizeTimerID = null;
            tl.layout();
        }, 500);
    }
}

I’m interested in any other ways people are using Google’s calendar data so let me know if you done/seen anything cool!

CruiseControl.NET VS.NET Starter Kit for plugin development

At the Silicon Valley Code Camp 2009 I gave a talk called Extending CruiseControl.NET through the use of plugins. I discussed the necessary steps and illustrated with an example ISourceControl provider using the LinqToTwitter OS project on Codeplex. The provider polls a configurable Twitter account looking for Tweets that that start with “CI:” allowing the Tweeter to trigger a build of the project simply be tweeting something like: “CI:Start the build”.

It’s a simple example but illustrates how easy it is to create CC.NET plugins an extend the platform to uses beyond classic Continuous Integration.

Here is a link to the Starter Kit. Btw, you will need to update the ThoughtWorks assembly references to match your build of CCNET.

dasBlog provider for use with BlogEngine.NET

I’ve used dasBlog since I first began hosting my own blog and previously considered a move to BlogEngine.NET (BE) but the URL’s aren’t compatible so there remains some investigation to figure out the best mechanism prior to switching. Unfortunately, dasblog seems to be on it’s final legs so spending time there at this point doesn’t seem wise.

To that end, I’ve been investigating various issues involved in a switch and in an attempt to make a transition easier I’ve created a BE provider which reads/writes blog posts and comments using dasBlog’s IBlogDataService rather than BE’s own XML provider. Basically, I’ve replace one XML provider with another, actually that’s not entirely true as this new provide is a descendant of XmlBlogProvider included with BE. Using this approach I avoid having to constantly re-import my blog data as I look into solving the URL issues.

I’ve made the code available via SVN on Google Code here so feel free to grab it and give it a go. I chose Google Code over Codeplex because the latter doesn’t want abandoned projects and I don’t see this as an ongoing project as it’s scope is quite narrow. What I’m interested in is any feedback regarding issues using the code and taking this sort of approach. In my initial playing around with BE it seems to function properly and allows me to use the “slug” feature of BE to “fix” at least part of the URL problem by simply removing spaces from Titles rather than replacing then with dashes.

Things I’ve tested (manually)

  • Adding/Editing/Deleting posts
  • Adding/Editing/Deleting comments

I haven’t played with images or attachments though my blog data seems to render fairly well. I can see some limitations such as no support for threaded comments though I don’t view this as a long term solution but I think it’s a step in the right direction.

Things that need testing/implementation:

  • DateTime conversions from dasBlog to BE
  • Image uploads
  • Windows Live support

Btw, I’m looking for help with the problem of retaining my existing URLs so if you’re a dasblog user and potentially interested in switching ping me and perhaps we can collaborate.

ASP.NET Facebook Starter Kit for VS.NET 2008 updated to v0.8

I’ve updated my ASP.NET Facebook Starter Kit with the following improvements/changes:

  • Added support for dynamically resizable iframe which avoids a scrollbar though may have some side effects/caveats. Code is based on JavaScript sample from Facebook.
  • Added an example of XFBML which requires xd_receiver.htm (included) allowing for a Cross Domain Communication Channel
  • Added example of using FQL, refer to FQL.aspx
  • Updated Facebook Dev Toolkit assemblies
  • Added Facebook stylesheet from Bill Konrad

The download is here and full directions for installation and use are here. To see this application running on Facebook click here. As always, please direct questions on the Facebook API to the Developer’s Forum, or on the Facebook Developer’s Toolkit to the discussion list.

[Updated: Jan 15, 2010] This post is now outdated so please look for my updated versions. The latest is always available on my wiki.