Category Archives: Open Source

React Native bundle loading failing on a physical device

I’m building a React Native application and recently updated to v0.31.0 and at first things were working well debugging on the device benefiting from a feature of the react-native-xcode.sh script. The script copies your Dev machine’s IP address to a text file called ip.txt which is used for establishing the connection to your machine from your device since localhost points to the wrong place. Here’s the line of code in ./node_modules/react-native/packager/react-native-xcode.sh:

echo "$IP.xip.io" > "$DEST/ip.txt"

Continue reading

Publishing Python unit test results in Jenkins

When I switched to developing on an OS stack one of the first things I look for was a $g(Continuous Integration server) and settled on Hudson which, after some tumult with surrounding Oracle’s acquisition of Sun, was forked into Jenkins. Getting jenkins setup couldn’t be easier and the web UI is comprehensive and full of options.

My day-to-day development is in Python and I’ve written a bunch of tests based on the core unittest module though it doesn’t natively produce results that can be consumed by Jenkins. To that end, I searched around and found the necessary pieces which I wanted to capture.

First, you need to install the unittest-xml-reporting package which is described as:

PyUnit-based test runner with JUnit like XML reporting.

sudo easy_install unittest-xml-reporting

Once installed you need to add the following to your unittests so they will produce the necessary XML result output:

import xmlrunner

...

if __name__ == '__main__':
    unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test-reports'))

Next, in Jenkins click the configure link for your project and check the Publish JUnit test result report and set the path to the output location for the unit tests. In my case the full path to the XML output is /.hudson/jobs/publishing/workspace/trunk/test/test-reports. In Jenkins the path to use for publishing is **/trunk/test/test-reports/*.xml

Jenkins JUnit publisher settings

This will also add a chart to the project page in Jenkins:

image

Getting the Tornado Facebook demo working

Recently, I downloaded Tornado:

Tornado is an open source version of the scalable, non-blocking web server and tools that power FriendFeed.

Included is a facebook demo that displays your feed though it didn’t work “out-of-the-box” raising the error:

File "modules/post.html", line 26, in _execute
KeyError: 'message'

modules/post.html is a snippet of html used to render each feed item. The snippet looks like this:

<div class="post">
  <div class="picture">
    {% set author_url="http://www.facebook.com/profile.php?id=" + escape(post["from"]["id"]) %}
    <a href="{{ author_url }}"><img src="//graph.facebook.com/{{ escape(post["from"]["id"]) }}/picture?type=square"/></a>
  </div>
  <div class="body">
    <a href="{{ author_url }}" class="actor">{{ escape(post["from"]["name"]) }}&lt;/a>
    {% if post["message"] %}
      <span class="message">{{ escape(post["message"]) }}</span>
    {% end %}
    <div class="meta">
      <a href="{{ escape(post["actions"][0]["link"]) }}" class="permalink">{{ locale.format_date(datetime.datetime.strptime(post["created_time"], "%Y-%m-%dT%H:%M:%S+0000")) }}</a>
    </div>
  </div>
</div>

The curly braces are python code snippets and the line reading if post[“message”] is causing the problem. In python referencing a non-existent dictionary key raises a KeyError exception thus changing the code to use:

{% if “message” in post %}

To debug this problem I simply removed the code referencing post[“message”] and replaced it with str(post) which simply dumped the JSON contents onto the page. From there I could see that not all entries had a message key thus the error.

One more note. Looking at the documentation Tornado uses a library call epoll which at first glance sounds familiar to I/O completion ports on Windows which is something I’m familiar with.

Installing Google’s python client API and running samples on 2.4.6

At work I’m using Python 2.4.6 and wanted to play around with some of the Google API’s like Buzz and Moderator. Things didn’t quite work out-of-the-box so I thought I’d document my steps here for future reference and in the event anyone else might find it useful.

First, I read over the Python Google API page and found that in order to grab the code I’d first need to install Mercurial.

SteveT:cc strefethen$ sudo port install mercurial
--->  Computing dependencies for mercurial
--->  Dependencies to be installed: curl-ca-bundle python26 db46 gdbm sqlite3
--->  Fetching curl-ca-bundle
--->  Attempting to fetch curl-7.21.2.tar.bz2 from http://distfiles.macports.org/curl
--->  Attempting to fetch certdata-1.70.txt from http://distfiles.macports.org/curl
--->  Verifying checksum(s) for curl-ca-bundle
--->  Extracting curl-ca-bundle
--->  Applying patches to curl-ca-bundle
--->  Configuring curl-ca-bundle
--->  Building curl-ca-bundle
--->  Staging curl-ca-bundle into destroot
--->  Installing curl-ca-bundle @7.21.2_4
--->  Activating curl-ca-bundle @7.21.2_4
--->  Cleaning curl-ca-bundle
--->  Fetching db46
--->  Attempting to fetch patch.4.6.21.1 from http://distfiles.macports.org/db4/4.6.21_6
--->  Attempting to fetch patch.4.6.21.2 from http://distfiles.macports.org/db4/4.6.21_6
--->  Attempting to fetch patch.4.6.21.3 from http://distfiles.macports.org/db4/4.6.21_6
--->  Attempting to fetch patch.4.6.21.4 from http://distfiles.macports.org/db4/4.6.21_6
--->  Attempting to fetch db-4.6.21.tar.gz from http://distfiles.macports.org/db4/4.6.21_6
--->  Verifying checksum(s) for db46
--->  Extracting db46
--->  Applying patches to db46
--->  Configuring db46
--->  Building db46
--->  Staging db46 into destroot
--->  Installing db46 @4.6.21_6
--->  Activating db46 @4.6.21_6
--->  Cleaning db46
--->  Fetching gdbm
--->  Attempting to fetch gdbm-1.8.3.tar.gz from http://mirrors.kernel.org/gnu/gdbm
--->  Verifying checksum(s) for gdbm
--->  Extracting gdbm
--->  Configuring gdbm
--->  Building gdbm
--->  Staging gdbm into destroot
--->  Installing gdbm @1.8.3_3
--->  Activating gdbm @1.8.3_3
--->  Cleaning gdbm
--->  Fetching sqlite3
--->  Attempting to fetch sqlite-autoconf-3070400.tar.gz from http://distfiles.macports.org/sqlite3
--->  Attempting to fetch sqlite-autoconf-3070400.tar.gz from http://www.sqlite.org/
--->  Verifying checksum(s) for sqlite3
--->  Extracting sqlite3
--->  Applying patches to sqlite3
--->  Configuring sqlite3
--->  Building sqlite3
--->  Staging sqlite3 into destroot
--->  Installing sqlite3 @3.7.4_0
--->  Activating sqlite3 @3.7.4_0
--->  Cleaning sqlite3
--->  Fetching python26
--->  Attempting to fetch Python-2.6.6.tar.bz2 from http://distfiles.macports.org/python26
--->  Verifying checksum(s) for python26
--->  Extracting python26
--->  Applying patches to python26
--->  Configuring python26
--->  Building python26
--->  Staging python26 into destroot
--->  Installing python26 @2.6.6_1
--->  Activating python26 @2.6.6_1

To fully complete your installation and make python 2.6 the default,  please run:
 	sudo port install python_select
 	sudo python_select python26

--->  Cleaning python26
--->  Fetching mercurial
--->  Attempting to fetch mercurial-1.7.3.tar.gz from http://distfiles.macports.org/python
--->  Attempting to fetch mercurial-1.7.3.tar.gz from http://mercurial.selenic.com/release/
--->  Verifying checksum(s) for mercurial
--->  Extracting mercurial
--->  Configuring mercurial
--->  Building mercurial
--->  Staging mercurial into destroot
--->  Installing mercurial @1.7.3_0
--->  Activating mercurial @1.7.3_0
--->  Cleaning mercurial

As you can see after a fairly lengthy install I was ready to grab the API sources:

SteveT:work strefethen$ hg clone https://google-api-python-client.googlecode.com/hg/ google-api-python-client
requesting all changes
adding changesets
adding manifests
adding file changes
added 153 changesets with 468 changes to 204 files
updating to branch default
172 files updated, 0 files merged, 0 files removed, 0 files unresolved</pre>
<p>Next, was to setup the Google API python client</p>
<pre class="brush: bash;">SteveT:work strefethen$ cd google-api-python-client/
SteveT:google-api-python-client strefethen$ sudo python setup.py install
Password:
Installing the following packages: 
'apiclient', 'apiclient.ext', 'uritemplate'
Loaded setuptools
running install
Checking .pth file support in /Users/strefethen/Library/Python2.4/site-packages/
/opt/local/bin/python -E -c pass
TEST PASSED: /Users/strefethen/Library/Python2.4/site-packages/ appears to support .pth files
running bdist_egg
running egg_info
creating google_api_python_client.egg-info
writing google_api_python_client.egg-info/PKG-INFO
writing top-level names to google_api_python_client.egg-info/top_level.txt
writing dependency_links to google_api_python_client.egg-info/dependency_links.txt
writing manifest file 'google_api_python_client.egg-info/SOURCES.txt'
reading manifest file 'google_api_python_client.egg-info/SOURCES.txt'
writing manifest file 'google_api_python_client.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.6-i386/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/apiclient
copying apiclient/__init__.py -&gt; build/lib/apiclient
copying apiclient/anyjson.py -&gt; build/lib/apiclient
copying apiclient/discovery.py -&gt; build/lib/apiclient
copying apiclient/errors.py -&gt; build/lib/apiclient
copying apiclient/http.py -&gt; build/lib/apiclient
copying apiclient/model.py -&gt; build/lib/apiclient
copying apiclient/oauth.py -&gt; build/lib/apiclient
creating build/lib/apiclient/ext
copying apiclient/ext/__init__.py -&gt; build/lib/apiclient/ext
copying apiclient/ext/appengine.py -&gt; build/lib/apiclient/ext
copying apiclient/ext/authtools.py -&gt; build/lib/apiclient/ext
copying apiclient/ext/django_orm.py -&gt; build/lib/apiclient/ext
creating build/lib/uritemplate
copying uritemplate/__init__.py -&gt; build/lib/uritemplate
creating build/lib/apiclient/contrib
creating build/lib/apiclient/contrib/buzz
copying apiclient/contrib/buzz/future.json -&gt; build/lib/apiclient/contrib/buzz
creating build/lib/apiclient/contrib/latitude
copying apiclient/contrib/latitude/future.json -&gt; build/lib/apiclient/contrib/latitude
creating build/lib/apiclient/contrib/moderator
copying apiclient/contrib/moderator/future.json -&gt; build/lib/apiclient/contrib/moderator
creating build/bdist.macosx-10.6-i386
creating build/bdist.macosx-10.6-i386/egg
creating build/bdist.macosx-10.6-i386/egg/apiclient
copying build/lib/apiclient/__init__.py -&gt; build/bdist.macosx-10.6-i386/egg/apiclient
copying build/lib/apiclient/anyjson.py -&gt; build/bdist.macosx-10.6-i386/egg/apiclient
creating build/bdist.macosx-10.6-i386/egg/apiclient/contrib
creating build/bdist.macosx-10.6-i386/egg/apiclient/contrib/buzz
copying build/lib/apiclient/contrib/buzz/future.json -&gt; build/bdist.macosx-10.6-i386/egg/apiclient/contrib/buzz
creating build/bdist.macosx-10.6-i386/egg/apiclient/contrib/latitude
copying build/lib/apiclient/contrib/latitude/future.json -&gt; build/bdist.macosx-10.6-i386/egg/apiclient/contrib/latitude
creating build/bdist.macosx-10.6-i386/egg/apiclient/contrib/moderator
copying build/lib/apiclient/contrib/moderator/future.json -&gt; build/bdist.macosx-10.6-i386/egg/apiclient/contrib/moderator
copying build/lib/apiclient/discovery.py -&gt; build/bdist.macosx-10.6-i386/egg/apiclient
copying build/lib/apiclient/errors.py -&gt; build/bdist.macosx-10.6-i386/egg/apiclient
creating build/bdist.macosx-10.6-i386/egg/apiclient/ext
copying build/lib/apiclient/ext/__init__.py -&gt; build/bdist.macosx-10.6-i386/egg/apiclient/ext
copying build/lib/apiclient/ext/appengine.py -&gt; build/bdist.macosx-10.6-i386/egg/apiclient/ext
copying build/lib/apiclient/ext/authtools.py -&gt; build/bdist.macosx-10.6-i386/egg/apiclient/ext
copying build/lib/apiclient/ext/django_orm.py -&gt; build/bdist.macosx-10.6-i386/egg/apiclient/ext
copying build/lib/apiclient/http.py -&gt; build/bdist.macosx-10.6-i386/egg/apiclient
copying build/lib/apiclient/model.py -&gt; build/bdist.macosx-10.6-i386/egg/apiclient
copying build/lib/apiclient/oauth.py -&gt; build/bdist.macosx-10.6-i386/egg/apiclient
creating build/bdist.macosx-10.6-i386/egg/uritemplate
copying build/lib/uritemplate/__init__.py -&gt; build/bdist.macosx-10.6-i386/egg/uritemplate
byte-compiling build/bdist.macosx-10.6-i386/egg/apiclient/__init__.py to __init__.pyc
byte-compiling build/bdist.macosx-10.6-i386/egg/apiclient/anyjson.py to anyjson.pyc
byte-compiling build/bdist.macosx-10.6-i386/egg/apiclient/discovery.py to discovery.pyc
byte-compiling build/bdist.macosx-10.6-i386/egg/apiclient/errors.py to errors.pyc
byte-compiling build/bdist.macosx-10.6-i386/egg/apiclient/ext/__init__.py to __init__.pyc
byte-compiling build/bdist.macosx-10.6-i386/egg/apiclient/ext/appengine.py to appengine.pyc
byte-compiling build/bdist.macosx-10.6-i386/egg/apiclient/ext/authtools.py to authtools.pyc
byte-compiling build/bdist.macosx-10.6-i386/egg/apiclient/ext/django_orm.py to django_orm.pyc
byte-compiling build/bdist.macosx-10.6-i386/egg/apiclient/http.py to http.pyc
byte-compiling build/bdist.macosx-10.6-i386/egg/apiclient/model.py to model.pyc
byte-compiling build/bdist.macosx-10.6-i386/egg/apiclient/oauth.py to oauth.pyc
byte-compiling build/bdist.macosx-10.6-i386/egg/uritemplate/__init__.py to __init__.pyc
creating build/bdist.macosx-10.6-i386/egg/EGG-INFO
copying google_api_python_client.egg-info/PKG-INFO -&gt; build/bdist.macosx-10.6-i386/egg/EGG-INFO
copying google_api_python_client.egg-info/SOURCES.txt -&gt; build/bdist.macosx-10.6-i386/egg/EGG-INFO
copying google_api_python_client.egg-info/dependency_links.txt -&gt; build/bdist.macosx-10.6-i386/egg/EGG-INFO
copying google_api_python_client.egg-info/top_level.txt -&gt; build/bdist.macosx-10.6-i386/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
apiclient.discovery: module references __file__
creating dist
creating 'dist/google_api_python_client-0.1-py2.4.egg' and adding 'build/bdist.macosx-10.6-i386/egg' to it
removing 'build/bdist.macosx-10.6-i386/egg' (and everything under it)
Processing google_api_python_client-0.1-py2.4.egg
creating /Users/strefethen/Library/Python2.4/site-packages/google_api_python_client-0.1-py2.4.egg
Extracting google_api_python_client-0.1-py2.4.egg to /Users/strefethen/Library/Python2.4/site-packages
Adding google-api-python-client 0.1 to easy-install.pth file

Installed /Users/strefethen/Library/Python2.4/site-packages/google_api_python_client-0.1-py2.4.egg
Processing dependencies for google-api-python-client==0.1
Finished processing dependencies for google-api-python-client==0.1
Setup complete!

Finally, attempt to run the Buzz sample which caused the following error

SteveT:buzz strefethen$ python three_legged_dance.py 
Go to the following link in your browser:
https://www.google.com/buzz/api/auth/OAuthAuthorizeToken?scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fbuzz&domain=anonymous&oauth_token=4%2FetZ5lubFdHfPmrqf0pg9IRR_Nury

    

^CTraceback (most recent call last):
  File "three_legged_dance.py", line 40, in ?
    run(flow, 'buzz.dat')
  File "/Users/strefethen/Library/Python2.4/site-packages/google_api_python_client-0.1-py2.4.egg/apiclient/ext/authtools.py", line 121, in run
    httpd.handle_request()
  File "/opt/local/lib/python2.4/SocketServer.py", line 217, in handle_request
    request, client_address = self.get_request()
  File "/opt/local/lib/python2.4/SocketServer.py", line 373, in get_request
    return self.socket.accept()
  File "/opt/local/lib/python2.4/socket.py", line 161, in accept
    sock, addr = self._sock.accept():

I hadn’t reviewed the code (yeah, not necessarily the brightest idea but hey, it’s Google right?). The above error occurred because I had an application running on port 8080 and didn’t realize the demo was going to require that. After shutting down Hudson I was able to authenticate and move forward.

SteveT:buzz strefethen$ python three_legged_dance.py 
Go to the following link in your browser:
https://www.google.com/buzz/api/auth/OAuthAuthorizeToken?scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fbuzz&domain=anonymous&oauth_token=4%2FyDtHZJcU8xD_XP023fUiaKC93wyE

You have successfully authenticated.

However, I ran into another problem which is where the significance of python 2.4 comes in. Attempting to run the buzz.py example I got the following error:

SteveT:buzz strefethen$ python buzz.py 
Traceback (most recent call last):
  File "buzz.py", line 75, in ?
    main()
  File "buzz.py", line 37, in main
    activitylist = activities.list(
  File "/Users/strefethen/Library/Python2.4/site-packages/google_api_python_client-0.1-py2.4.egg/apiclient/discovery.py", line 243, in method
    new_base_url = url_result.scheme + '://' + url_result.netloc
AttributeError: 'tuple' object has no attribute 'scheme'

After a few minutes of tangling with Google I found this page which gave me a clue. Specifically, it was dag’s post from 2008/12/9 relating to urlparse and a difference between python 2.4 and 2.5. I opened the above discovery.py file and searched for ‘scheme’ in order to change the reference to something that would work in 2.4. I had to repeat this for ‘netloc’ and ‘path’ with url_result[0], [1], [2] respectively. After saving the file the sample worked as expected.

Granted I’ve probably included a lot more information with respect to the console output above but there are are few things I’m interested in keeping so this is a great place to do just that.

Now, time to go off and have some fun with these Google API’s.

I’m particularly interested in the Moderator API and I’m curious if anyone has written apps to completely replace the Google Moderator UI with a custom design?

dasBlog provider for use with BlogEngine.NET

I’ve used dasBlog since I first began hosting my own blog and previously considered a move to BlogEngine.NET (BE) but the URL’s aren’t compatible so there remains some investigation to figure out the best mechanism prior to switching. Unfortunately, dasblog seems to be on it’s final legs so spending time there at this point doesn’t seem wise.

To that end, I’ve been investigating various issues involved in a switch and in an attempt to make a transition easier I’ve created a BE provider which reads/writes blog posts and comments using dasBlog’s IBlogDataService rather than BE’s own XML provider. Basically, I’ve replace one XML provider with another, actually that’s not entirely true as this new provide is a descendant of XmlBlogProvider included with BE. Using this approach I avoid having to constantly re-import my blog data as I look into solving the URL issues.

I’ve made the code available via SVN on Google Code here so feel free to grab it and give it a go. I chose Google Code over Codeplex because the latter doesn’t want abandoned projects and I don’t see this as an ongoing project as it’s scope is quite narrow. What I’m interested in is any feedback regarding issues using the code and taking this sort of approach. In my initial playing around with BE it seems to function properly and allows me to use the “slug” feature of BE to “fix” at least part of the URL problem by simply removing spaces from Titles rather than replacing then with dashes.

Things I’ve tested (manually)

  • Adding/Editing/Deleting posts
  • Adding/Editing/Deleting comments

I haven’t played with images or attachments though my blog data seems to render fairly well. I can see some limitations such as no support for threaded comments though I don’t view this as a long term solution but I think it’s a step in the right direction.

Things that need testing/implementation:

  • DateTime conversions from dasBlog to BE
  • Image uploads
  • Windows Live support

Btw, I’m looking for help with the problem of retaining my existing URLs so if you’re a dasblog user and potentially interested in switching ping me and perhaps we can collaborate.

Copyright and the Free Pascal project

Updated July 31, 2011 Another interesting battle brewing with regard to Open Source and copying of source code between Google and Oracle over Java.

If you’ve been following my blog recently perhaps you’re aware of activity related to a post I wrote back in September asking for a free version of the Delphi command line compiler. In a response to a comment asking if I’d ever tried the Free Pascal compiler I stated:

I haven’t tried the Free Pascal compiler for a number of reasons including the fact that it’s RTL/FCL violate Borland/CodeGear’s copyright. I’ve looked at the code closely and IMO copyright issues abound.

At that time, there were a few replies but over the last few days members of the Free Pascal core team as well as the FP community attacked me personally over this statement. Upon request I published one example illustrating my point using Classes.ExtractStrings. Unfortunately, this was met by further attacks until a comment from Marco van de Voort and subsequent email apology from Daniël Mantione, both members of the core Free Pascal team. The following republished with Daniël’s permission:

from: daniel.mantione@freepascal.org
to:strefethen@<deleted>
date:13 Nov 2007 15:25:14 -0800
subject:Weblog Mail from ‘Dani&#235;l Mantione’ on ‘Steve Trefethen’s Weblog’

Hello Steve,

I would like to offer apologies on behalf of the Free Pascal development team for the aggressive comments on your weblog. The situation puzzles us; that code you showed is more similar than one would expect with an independend implementation, yet it doesn’t look like being taken from Delphi source. Michael, who comitted it, cannot remember having written it so it might have been contributed by someone, but searches through e-mail archives haven’t revealed anything yet.

Anyway, it’s good that this is on the agenda now. Copyrights are something we have to be carefull about.

Best regards,

Daniël Mantione

In further support my statements I added five additional examples all from classes.pp which covers roughly 16% of that file, which is not exactly a “fringe file”. I’ll add that this more recent review took but a few minutes and is by no mean extensive nor exhaustive unlike a review I did earlier this year. Daniël has since provided me plausible explanations as to how these issues might have arisen and indicated that the core team is reviewing sysutils.pp and classes.pp.

Unfortunately, this is neither a new issue nor limited to sysutils.pp and classes.pp. I’m disappointed that seemingly no one from the Free Pascal team or community discovered nor, to my knowledge, raised these issues years ago.

And, fwiw, I haven’t been employed by Borland since June 28, 2007 nor, as was implied in a comment, do I have any financial interest in the company whatsoever.

Of course, this raises the issue of how do you go about validating the originality of Open Source source code in general? And no, I don’t have a good answer.

Thunderbird rocks

Until fairly recently I was an Outlook Express user and had been for many
years.  I’ve now switched to using Thunderbird which is a
great mail client.  I never liked the regular Outlook client since it doesn’t
have builtin support for newsgroups which is a requirement for me and
Thunderbird fits the bill nicely.  Another bonus of using Thunderbird is that
it’s being updated regularly and in fact I’ve now installed several updates with
new features that I can immediately appreciate.  It also has some very nice
plugins like Quote Colors which
makes reading newsgroups really nice.