After working on this problem the other day I started Googling looking for posts written about using LINQ for text file processing. I found the post Parsing textfiles with LINQ (or LINQ-to-TextReader) by Arjan Einbu.

LINQ shows us alternate ways to write code, introducing a more declarative coding paradigm. To use LINQ over the lines of a file, we can read all the lines in the file into a collection, and use LINQ over that collection. There’s some overhead to this; the need to read the entire file upfront and to fit the entire file in memory at once.

The solution was to create an extension method on TextReader for IEnumerable<string>. That post was followed up by another post, rather unfortunately titled, improving upon the solution using  TextFieldParser class in the Microsoft.VisualBasic.FileIO namespace, something I wasn’t aware existed and now find it odd this class is stuck well off in left field.

One of the reasons this subject interests me is I’ve been working with EDI files for awhile now and querying data directly from this file format would be really nice. For example, given a PO with line item segments like this:

PO1*1*36*CA*11.15*PE***VP*RRSKRC85*PI*0001111091127~
PID*F****PRSL ROMNE BBY TRAY ORGNC~
PO1*2*84*CA*11.15*PE***VP*RSMKRC85*PI*0001111091131~
PID*F****PRSL SPRG BBY TRAY ORGNC~
PO1*3*84*CA*11.15*PE***VP*RBSKRC85*PI*0001111091128~
PID*F****PRSL SPNCH BBY TRAY ORGNC~
PO1*4*72*CA*11.15*PE***VP*RHEKRC85*PI*0001111091126~
PID*F****PRSL SPRG W/HRB CLM ORGNC~

You can calculate the total quantity, highlighted in yellow, of all line items using LINQ like this:

using (var reader = new StreamReader("c:\\edi\\inbound\\850_09022009_1311_89.txt"))
{
    var query = (from line in reader.GetSplittedLines("*")
                            where line[0].Equals("PO1") && line[2].Length > 0
                            select Convert.ToInt32(line[2])).Sum();
    ...
}

Using Arjan’s implementation of GetSpittedLines, that’s his name not mine for the extension method he wrote, you can apply logic to any of the columns from the file which is pretty cool.

Of course, there are a myriad of ways of doing the same thing but it’s interesting to have access to the columns allowing for calculations and querying. For my EDI work I’m using FileHelpers which works well though I really like this LINQ option. That said, I haven’t done any benchmarking so I’m not sure about the performance but most of the PO’s I’m working with are less than 4KB and the volume isn’t so great that this would be a major factor. At any rate, I hope you find useful for you too.

Btw, if you’re looking for custom EDI implementations feel free to contact me.

In my SocialMine Facebook application, which uses MIT’s Simile Exhibit, I needed to fetch Facebook FQL query results from the Exhibit client-side JavaScript. While it may not sound that complicated it’s not something directly supported by the Facebook Developer Toolkit (FDT). I created an HttpHandler to respond to the Exhibit requests for the JSON data which populates the Exhibit. The HTTP request includes a Referrer header that contains the necessary data, fb_sig_user and fb_sig_session_key to construct an instance of facebook.Components.FacebookService. For completeness here is a list of the queryparams contained in the Referrer URL with the important lines highlighted:

auth_token=1379094a2a42e1c0037a33048912a875
fb_sig_in_iframe=1
fb_sig_locale=en_US
fb_sig_in_new_facebook=1
fb_sig_time=1245301045.1412
fb_sig_added=1
fb_sig_profile_update_time=1241328269
fb_sig_expires=0
fb_sig_user=719571200
fb_sig_session_key=c93185e145713ce98d72dfc2-719571700
fb_sig_ss=57abbef0284354d029fe8a19fa00dfce
fb_sig_ext_perms=offline_access,email,auto_publish_recent_activity
fb_sig_api_key=33c52d0a0c9867f677a1f05154c28478
fb_sig_app_id=5012360767
fb_sig=9ad664af23910ca18aeb203203c366cd

My HttpHandler is an abstract class with a single abstract method called FacebookRequest that descendents override to handle the request. As you can see below the handler itself is quite simple, parsing the queryparams into a Dictionary, creating an instance of the FacebookService and calling FacebookRequest passing the service instance and the HttpContext passed to the handler.

public abstract class FacebookHandler : IHttpHandler
{
    private const string REQUEST_SESSION_KEY = "fb_sig_session_key";
    private const string REQUEST_USER_ID = "fb_sig_user";

    #region IHttpHandler Members

    public bool IsReusable
    {
        get { return false; }
    }

    void IHttpHandler.ProcessRequest(HttpContext ctxt)
    {
        facebook.Components.FacebookService f = new facebook.Components.FacebookService();
        f.ApplicationKey = WebConfigurationManager.AppSettings["APIKey"];
        f.Secret = WebConfigurationManager.AppSettings["Secret"];

        Dictionary<string, string> dic = new Dictionary<string, string>();

        string[] split = ctxt.Request.UrlReferrer.Query.Split(new char[] { '?', '&', '=' });
        // Skip the first item since it’s the "?"
        for (int i = 1; i < split.Length - 1; i += 2)
        {
            if(i < split.Length)
                dic.Add(split[i], split[i + 1]);
        }

        f.SessionKey = dic[REQUEST_SESSION_KEY];
        f.uid = Convert.ToInt64(dic[REQUEST_USER_ID]);
        FacebookRequest(f, ctxt);
    }
    #endregion

    protected abstract void FacebookRequest(facebook.Components.FacebookService fbservice, HttpContext ctxt);
}

The JavaScript of Simile Exhibit requests the JSON data using the following LINK tag:

<link href="friendsjson.aspx" type="application/json" rel="exhibit/data" />    

The reference to friendsjson.aspx isn’t actually an ASPX as it’s handled by a descendent of this HttpHandler which is registered in the httpHandlers section of the web.config as follows:

<add verb="*" path="friendsjson.aspx" type="FBFQL.JSONHandler" />

In my application the FacebookRequest method queries using FQL for profile data and returns a JSON result that populates the Exhibit.

Btw, if you're looking for help on Facebook development be sure to read my wiki article.

A little Friday fun… while reviewing some code awhile ago I ran into this little gem:

// Warning: Suckage++

The remainder of the comment is in reference to a situation beyond the developers control where they had to special case logic to deal with some missing data. My comments typically aren’t quite this creative and I usually denote code that has issues using NOTE: or "!!!".

How do you mark code that otherwise denotes some nastiness?

I’ve updated my ASP.NET Facebook Starter Kit with the following improvements/changes:

• Added support for dynamically resizable iframe which avoids a scrollbar though may have some side effects/caveats. Code is based on JavaScript sample from Facebook.
• Added an example of XFBML which requires xd_receiver.htm (included) allowing for a Cross Domain Communication Channel
• Added example of using FQL, refer to FQL.aspx
• Updated Facebook Dev Toolkit assemblies
• Added Facebook stylesheet from Bill Konrad

The download is here and full directions for installation and use are here. To see this application running on Facebook click here. As always, please direct questions on the Facebook API to the Developer’s Forum, or on the Facebook Developer’s Toolkit to the discussion list.

[Updated: Jan 15, 2010] This post is now outdated so please look for my updated versions. The latest is always available on my wiki.

One of the features I really miss from Delphi is the option to close files implicitly opened while debugging. You know, those files that are opened by the debugger as your stepping through code. In Delphi, those files are closed automatically at the end of the debug session thus helping keep you focused on the code you’ve been working on. I was just wading through all of the open files in my current solution and trying to close files I no longer wanted open the vast majority of which were opened during a debug session. After closing 6-8 files manually I clicked on the tab dropdown and there was still a huge list of open files so my usual approach is to close all my tabs and start opening files individually again. Here is the option as it appears in Delphi, note the default is checked. I’m sure lots of Delphi developers now take this option for granted.

I wonder if CodeGear will have this in Prism? Anyone know how/where to make a VS.NET feature requests?

[Update: Oct. 30, 2008] Thanks to a comment I’ve logged a suggestion for this feature and if this is something you would like to see as well please vote for this request. Warning, that link may require login.

Congratulations CodeGear!

Want to offer my congratulations to the Team at CodeGear/Embarcadero for shipping Delphi 2009 & C++Builder 2009. For sure, it looks like some of the most interesting updates are in the Win32 compiler as I follow Barry Kelly’s blog where he’s been talking about Anonymous Methods and Generics. Of course, Nick Hodges has a rundown of all sorts of additional features. I know lately the gang there has been putting in lots of hours as it’s been hard to catch up with people like Mark Edington. Who, btw, mentioned to me that he has an excellent blog post he said he would write about using AutomatedQA’s AQTime of which he’s a big fan. (Maybe this will put a little pressure on him to get that post written!)

Again, congratulations guys!

[Update: Sept 9th, 2008] Fix the spelling of Barry's name doh!

If you’ve downloaded either the Facebook Developer Toolkit or Facebook.NET Starter Kits I recommend voting in Facebook’s poll regarding which client library you use. It seems to me there’s no good reason Facebook shouldn’t provide an ASP.NET support on par with PHP though I’m biased. Here is the blog post about the poll.

Heck, even if you haven’t downloaded my starter kits but you do ASP.NET development be sure to vote!