Copyright and the Free Pascal project

Updated July 31, 2011 Another interesting battle brewing with regard to Open Source and copying of source code between Google and Oracle over Java.

If you’ve been following my blog recently perhaps you’re aware of activity related to a post I wrote back in September asking for a free version of the Delphi command line compiler. In a response to a comment asking if I’d ever tried the Free Pascal compiler I stated:

I haven’t tried the Free Pascal compiler for a number of reasons including the fact that it’s RTL/FCL violate Borland/CodeGear’s copyright. I’ve looked at the code closely and IMO copyright issues abound.

At that time, there were a few replies but over the last few days members of the Free Pascal core team as well as the FP community attacked me personally over this statement. Upon request I published one example illustrating my point using Classes.ExtractStrings. Unfortunately, this was met by further attacks until a comment from Marco van de Voort and subsequent email apology from Daniël Mantione, both members of the core Free Pascal team. The following republished with Daniël’s permission:

from: daniel.mantione@freepascal.org
to:strefethen@<deleted>
date:13 Nov 2007 15:25:14 -0800
subject:Weblog Mail from ‘Dani&#235;l Mantione’ on ‘Steve Trefethen’s Weblog’

Hello Steve,

I would like to offer apologies on behalf of the Free Pascal development team for the aggressive comments on your weblog. The situation puzzles us; that code you showed is more similar than one would expect with an independend implementation, yet it doesn’t look like being taken from Delphi source. Michael, who comitted it, cannot remember having written it so it might have been contributed by someone, but searches through e-mail archives haven’t revealed anything yet.

Anyway, it’s good that this is on the agenda now. Copyrights are something we have to be carefull about.

Best regards,

Daniël Mantione

In further support my statements I added five additional examples all from classes.pp which covers roughly 16% of that file, which is not exactly a “fringe file”. I’ll add that this more recent review took but a few minutes and is by no mean extensive nor exhaustive unlike a review I did earlier this year. Daniël has since provided me plausible explanations as to how these issues might have arisen and indicated that the core team is reviewing sysutils.pp and classes.pp.

Unfortunately, this is neither a new issue nor limited to sysutils.pp and classes.pp. I’m disappointed that seemingly no one from the Free Pascal team or community discovered nor, to my knowledge, raised these issues years ago.

And, fwiw, I haven’t been employed by Borland since June 28, 2007 nor, as was implied in a comment, do I have any financial interest in the company whatsoever.

Of course, this raises the issue of how do you go about validating the originality of Open Source source code in general? And no, I don’t have a good answer.

29 thoughts on “Copyright and the Free Pascal project

  1. Steve,
    Given that the FP folks attempt to undertake compatibility with Delphi, then clearly, compatible library functions will have identical names, and signatures that may vary only in the parameter names. Worse, when the functions are small, and make extensive use of WinAPI calls, they will be further boxed in. Then, assuming they have a good grip on performance issues, they are likely to follow essentially the same logic, and use essentially the same functions for internal processing.
    I’m not looking to defend FP, or its authors, but am simply making an observation. It might be interesting to examine some longer functions, where there may be fewer points of compatibility constraining what they do, and see whether the similarity reduces at all.
    I recall the lengths to which Phoenix went to produce a clean-room version of the IBM PC BIOS. I think it would be exceedingly difficult to follow the same path in attempting to produce a Delphi-compatible Pascal.
    It would be interesting to see what might be done with some sort of analysis tool. Not that I’m volunteering… my skills are not up to that sort of lexical and semantic analysis.

  2. @Steve: Why do you limit your question to Open Source code? Also closed source code could be a copy and past result. The difference with Open Source is that this can be checked.
    @Bill: There exists some anti copy and paste tools, but I found no sufficient one yet. I would use such a tool for refactoring but you could use them also for finding such suspicious code.

  3. Bill,
    I’ll agree that I think it would be difficult to make a Delphi RTL/VCL clone with out getting very close to the real thing. Not impossible, but clearly very difficult. The Delphi RTL/VCL is fairly large and replicating it in an alternative form would take a very long time. Having stared at the Delphi RTL/VCL source as long as I have it took me no time at all to quickly pin point code that looked familiar so perhaps I’m in a more unique position than many in that regard.
    Uwe,
    Point taken. That’s one of the things that makes this case that much more curious. There are clear cut issues and the source is readily available yet even people on the core team assumed there were none.

  4. Uwe, in some cases it would be legal under the Delphi license to use the RTL source code in closed source works, so you don’t redistribute the source itself (since you’re allowed to redistribute the RTL in compiled form, with certain minor restrictions). If you redistributed the source code itself in a non-open-source product that would not be allowed under the license, but no harder to check for plagiarism / illegal redistribution than with an open source product.
    So: The only cases where you can’t check for plagiarism, regardless of license type, are probably legal anyway. Odd, but true as far as I can see….

  5. Good guys from FreePascal. They fixes a ancient bug in TThread.
    Delphi code:
    Thread.DoTerminate;
    Thread.FFinished: = True;
    FreePascal code:
    Thread.FFinished: = True;
    Thread.DoTerminate;
    Last correctly.

  6. Craig,
    Interesting reply, thanks for the comment.
    Mikhael,
    So what you’re saying is that Delphi’s TThread code copied into Free Pascal has been fixed? That’s a good thing?
    That’s sort of like the ReadComponentResFile example I pointed out where part of the original Delphi code is commented out which to me clearly indicates the code is a direct copy.

  7. Very bad.
    I think all implementation sections in FCL units have to be rewritten to exclude a proprietary code.
    Though I think if CodeGear start to attack FP project it’ll be sensible impact for the whole Delphi community, because the Free pascal is an only cross platform pascal compiler for now.

  8. Steve,
    I said that this is good or bad? I said that they fixes a bug in TThread.
    Consequently, in this place there is no copyright code.

  9. Kryvich: I think it would be very difficult to write an FCL fully compatible with VCL and at the same time, not copy any code. I also think that the likely reason that the FP project has not been attacked formally by CodeGear is that they consider it might damage the Pascal community.
    Donald: So, because the FP folks may have copied code, CodeGear must give up its rights to its own IP? And by what exercise of logic did you reach that rather strange conclusion?

  10. Kryvich,
    I don’t see this as an issue of CodeGear attacking the Free Pascal community but rather it being the responsibility of the FP project to ensure they aren’t violating CG copyright which IMO, clearly isn’t happening.
    Mikhael,
    To me, it’s the fact that you’re pointing to Delphi code in the FCL that was "corrected". A truly independent implementation would have had no such problem. Delphi’s own implementation of TThread has changed numerous times over the years illustrating that there are multiple ways to write that class.
    Donald,
    I’m not sure I see the value to CodeGear of releasing the VCL under MPL or some other OS style license. The Delphi RTL/VCL is a proprietary framework which has intricate ties to the Delphi compiler and designer/IDE. I can clearly see the value to the FP project but not to CodeGear.

  11. Steve, the only value to Codegear will be cross plataform. how many resources must expense Codegar to go linux or mac? A lot. If Codegear release VCL like OS and make it work on linux/macx with a little help to lazarus comunnity, Codegear will continue selling the IDE for windows and the compiler for windows (a lot better than FPC IMHO) and give that guys do the mac/linux stuff. The actual business of Coddegear is win32, so they are just open the horizons in that way, not losting any markets. Delphi users will continue using Win32 (and future Win64) codegear for be BETTER compilers and having a company support. Linux an mac user will use FPC compiler and then the delphi developers will continue using delphi in place of go to c# (Visual Studio) for cross development. There is my logic, for Bill Meyer
    . You can agree or not, but is a idea.
    best regards.

  12. Donald,
    I do understand though while CG might not be losing any market(s) they’re not gaining either so from a business perspective I think it makes little sense. An important note, I think it’s valuable for CG to control the quality of the experience developers have with their framework though providing the framework to a community that has poor tooling doesn’t seem to have a lot of benefit particularly these days when IDE’s are more or less commodities.

  13. As Steve said, I can see the benefit to the FP folks, but not to CodeGear. A great deal of time and money have been invested in the development of the VCL over the last 12+ years, and for FP to gain that benefit at no cost makes no sense.
    FP has a problem, if they wish to maintain Delphi compatibility. They need to do so without making unlicensed use of CodeGear source code. Others have approached similar problems in a clean-room fashion, and reached their goals. Steve’s investigations suggest that in this case, the room was not so clean as it should have been. Not proof, but cause for concern. And from what I have seen, the similarities in code are so striking that the only defense I can imagine would be by the FP folks claiming that the routines are so simple, or so constrained on the one hand by VCL compatibility concerns, and on the other by Windows APIs, that they could hardly have written anything else. Pretty weak, and not a very attractive approach.

  14. Codegear will gain markets, Mac and Linux markets. They can produce a cross compile tool (using fpc) to build from the IDE for win(32/64) mac and linux. Wich IDE do you think will use the companies or even the professional developers? Lazarus?
    OTOH, they can release only the subset of VCL for OS and keep others key parts owned.
    Again, is just a idea, but the main point is IMHO Delphi need a cross compiler and dont have the time/resources/whatever to make it happens before 2009/2010. Is a lot of time and ‘the others’ markets change from Kylix days, just keep an eye on IBM/Netware/Macintosh/MS agreement with Netware, i mind, theres not a native solution for all that targets and .NET dont fit the native game.
    Again, just imo. Best regards.

  15. Thanks for you comment, i already expose my ideas to the one on charge of Delphi (imagine who) but he tell me theres not the enough support in the board of directors (sadly).
    Best regards.

  16. Hi Steve,
    I would be interested how users of FPC who own Delphi would be effected by this (if at all). Lets assume that there is suspect code, does owning Delphi not exempt the users from any legal implications as they are entitled under the VCL license to use the said code?
    Thanks
    Dean

  17. Dean,
    For a legal opinion you should consult a lawyer which I am not.
    My personal opinion is no, absolutely not. Owning Delphi in no way whatsoever changes nor diminishes the fact you’re using code which violates Borland copyright (my words). Additionally, the FP code changes the copyright to members of the FP core team (refer to this file as an example) and is redistributing it under the GNU Library GPL.
    Unfortunately, I think the road chosen by the Free Pascal project puts it’s existing RTL in a very awkward position from which it may be difficult to recover. A definite credibility problem.
    I believe if you’re selling commercial applications developed with Free Pascal you’ve likely got a problem on your hands.
    Good luck.

  18. Hi Steve,
    Thanks for the advice. That is a real pity. We have committed a substantial amount of time, money and effort towards a rewrite of an existing system in Delphi. We use Delphi for both the client and server components. Unfortunately, a multi platform solution for the server portion is necessary for us. We did some tests in recompiling in FPC and were happy with the results which is why we progressed with Delphi. This sort of possible legal situation would make it neccessary for us to drop Delphi for the server portion all together. FPC to us was always a back stop in case Codegear did not further support multiple platforms. While I don’t see CG starting a case against their customers any time soon, this sort of issue is not something anyone wants hanging over their heads.
    Hopefully some of your old friends at CG read these comments and understand the implications. It would be great if they could sit down with the FPC community and indicate the problem areas so that the FPC team could address them. While Lazarus may be seen as a competitor to Delphi, FPC is most certainly not. There are a number of developers (and third party tool vendors) who are keeping an eye on FPC, not as a replacement for Delphi, but rather as an addtiional deployment avenue. If FPC is no longer part of the equation, that only takes away from Delphi.
    Cheers
    Dean

  19. Let me first state that I’m not employed by Borland, I do not have any financial interest in the company nor am I working with or on behalf of Borland in relation to these issues. I’ve responded in kind due to the strong allegations made against me in the comments to my original post. You could say it’s the FP team and community’s fault opening this can of worms as it was their comments which forced me to provide what I consider to be proof with which the FP core team largely agreed.
    I’d be careful about confusing FP customers with Borland customers as there is no requirement one way or the other. IMO, this is strictly an FP created problem not a Borland problem. Borland has a fiduciary responsibility to protect it’s IP thus protecting it’s revenue stream. When it comes to copyright it’s not Borland’s responsibility to ensure compliance, that lies with the FP project. It’s hard for me to imagine Borland yielding rights to Delphi source without any financial gain.
    Since part of FP’s goal is to be an OS alternative to Delphi’s compiler, RTL and VCL (for non-visual components) it’s more than fair for Borland to view it as competition. If FP were to become wildly successful particularly on Win32 I’d expect it would almost certainly have a negative impact on Borland’s bottom line.
    Keep in mind, as long as FP is in violation of Borland’s copyright it should never have been considered "as an additional deployment avenue" by anyone. From the way I read your comments it sounds like you’re trying to place the responsibility on Borland. If anything, I’d say FP users should be demanding of the FP core team and community to know how they’re going to clean things up, a hard task for sure.
    Would you ever be completely comfortable with the code for commercial applications?

  20. @Steven
    As I mentioned in your other blog which started all this. I was simply interested in an example, which you supplied and I thanked you for that. Since then I found a tool (SIM 2.21) which was used to compare code between Linux and Minix. SIM also supports other languages and two of them being Pascal and Modula/2. I ran a few initial tests and did find more suspect code but did no history comparisons. I brought this to the FPC teams attention. The FPC team has since extended SIM to support the Object Pascal language features and compared all FPC’s code to Delphi and Kylix.
    The suspect code has been flagged and will be rewritten by a independent/not compromised developers, that has no knowledge of Delphi or Kylix code. Only the function interfaces and parameters may be used. The FPC team has been having further discussions with CodeGear about past and present releases of FPC.
    As I said in your previous blog – the FPC team takes this seariously and is fully committed to resolve this issue. Now that we have a automated tool to help use, things should go much better in the future.
    Now lets hope CodeGear follows suite and does a comparison of their own. After all, FPC and Lazarus code is a lot more public than Delphi/Kylix code. Copy & Pasting of code can go both ways. I’ve seen features that have been in Lazarus for years and now they appear in Delphi 2007. I’m not accusing anybody just making a statement that it could go both ways.

  21. << Let me first state that I’m not employed by Borland, >>
    Understand 100%. My hope is that some of the CG staff read your blog.
    << IMO, this is strictly an FP created problem not a Borland problem. Borland has a fiduciary responsibility to protect it’s IP thus protecting it’s revenue stream. When it comes to copyright it’s not Borland’s responsibility to ensure compliance, that lies with the FP project. It’s hard for me to imagine Borland yielding rights to Delphi source without any financial gain. >>
    100% on all points.
    << Since part of FP’s goal is to be an OS alternative to Delphi’s compiler, RTL and VCL (for non-visual components) it’s more than fair for Borland to view it as competition. If FP were to become wildly successful particularly on Win32 I’d expect it would almost certainly have a negative impact on Borland’s bottom line. >>
    This is where I would disagree. Delphi is more then just the compiler. On the Win32 side, the Turbo Delphi Express versions are, IMO, light years ahead of Lazarus. What I get with Delphi (IDE, Experts and Third Party Tools etc) make it an extremely productive environment. Lazarus does not even come in a distant second to all of this functionality. The real competition is VS 2005 and not FPC. I know of a company that is about to switch their whole team across from Delphi to VS. FPC gives Delphi some more "marketing bullets".
    << Keep in mind, as long as FP is in violation of Borland’s copyright it should never have been considered "as an additional deployment avenue" by anyone. From the way I read your comments it sounds like you’re trying to place the responsibility on Borland. If anything, I’d say FP users should be demanding of the FP core team and community to know how they’re going to clean things up, a hard task for sure. >>
    The problem is most certainly not Borlands. I just believe that FPC offers more to Delphi then it takes away. Borland have no obligations to do anything to resolve the issues. It’s just, IMO, in their interests to do so.
    << Would you ever be completely comfortable with the code for commercial applications? >>
    The problem is that there are very few sound reasons to build a Windows only server side of ones solution in Delphi other then "I like Delphi". We have to have a multiplatform server solution otherwise the MS licensing costs would murder us. That leaves us with C, C++, Java and FPC. Mono potentially has copyright issues of it’s own.
    Appologies in advance for the long post.

  22. Of course, this raises the issue of how do you go about validating the originality of Open Source source code in general? And no, I don’t have a good answer.

    I’d say the issue was more general: validating the originality of any source code. Consider a closed source software project. Even if the company’s regular employees are copyright-respecting saints, what about code contributed by consultants, outsourcing, or interns? Are all of them more reliable than a gaggle of OS volunteers? It’s worth remembering that DEC once sued Microsoft for VMS code that allegedly made its way into the NT kernel and MS settled for $150 million. This stuff’s hard.
    With open source projects at least people can can check easily, as you have. That means you’ll find more copyright violations in open source software, but not necessarily that there are more violations than in closed source software.

  23. "Of course, this raises the issue of how do you go about validating the originality of Open Source source code in general? And no, I don’t have a good answer."
    Of course, this raises the issue of how do you go about validating the originality of Closed Source code in general? And no, I don’t have a good answer.

  24. Dean,
    Good comment. No apology necessary. There might be some merit in the thought that FPC could offer Delphi some needed breadth though I’m not sure CG is in a position to capitalize on it which relates at least partially to my decision to leave.

  25. An update:
    The new release 2.2.2 has the disputed code removed, and the cut was made fairly wide (using a tool to identify candidates), and this has been merged to all currently live branches. This is also why it took so long.
    We have retired the old releases from our site. (rather than relicense, which could have been since all the disputed code was available under GPL via FreeCLX. This was deemed to confusing)
    Since nearly all public releases were affected by these disputed routines (most disputed routines arrived in one batch in 1998-1999, which was pre 1.0), this is particularly sad for some of the more odd ball platforms (like Atari) that are not supported anymore.
    Maybe in the future some attempt will be made to manually rebuild these with the replacement code, depending on hardware availability and manpower.

Comments are closed.