Functional Programming, the opiate of some people (or... Tripping on CAL)

If you don't like opinion pieces on the joys of particular programming styles/languages - don't read on ;-)

It has been months since I wrote any functional code in earnest. Most recently I have been busy coding in Objective-C, and I've a long history with Java, C++ and C, with the latter language being used for many years in conjunction with a very elegant home-brew object-like framework.

As something of a student of language technology: principles and practice, I usually follow all the news and buzz to be found spread around the infowebs, in such places as Lambda the Ultimate, various blogs and language mailing lists (such as Ruby-core).

Just today, I had reason to fire up the CAL tools again in order to knock up some prototypes. The overall experience was surprising to me, in terms of the effect that producing a program in a (lazy) functional language actually had on my psyche. Having given functional programming a rest for the several months, I was almost "accosted" by the flavours of functional programming afresh. Being one with a reasonable grasp of the concepts, I naturally did not have the learning curve that one would experience from scratch (i.e. this is not an ab initio experience), yet the hiatus was sufficient to being the differences between functional and procedural/OO programming into sharper relief, and evidently to tickle my brain anew in areas that I had forgotten were differentiated for the kind of cognition that you do when crafting a functional solution.

I was using the combination of CAL's textual and graphical development tools (the Eclipse plug-in and the Gem Cutter to be precise). I have erstwhile found that the combination of these two can be very potent, especially in the exploratory stages of development - when you are surveying the problem domain and starting to form hypotheses about the structure of a solution.

Once I had completed the tasks at hand, I sat back, and was aware of the 'buzz' that I was feeling. This is the buzz you get from general problem solving success, presumably a literal reaction to endorphins released (I assure you there were no other overt substances involved, unless you could my normal coffee intake that morning!). Thinking about why I was feeling so chipper, I surmised that it was for two reasons:
1. A strong sense of puzzle solving.
2. A sense of the elegance of the emergent solution.

On the first point, it occurred to me that the whole exercise of assembling a functional solution is, for the most part (that is, ignoring some of the operational issues), one of dealing directly with fitting shapes together. This is of course an analogy, but what is going on in my head feels like the assemblage of shapes - fitted together perfectly to create the required abstracts. Of course, this is nothing more than a whiff of mathematics in general - the perception of structures and order, and formal assemblies of such things. I think the whole perception of 'shape' is made even more tangible by the ingredient of having the visual development environment alongside the syntactic one - two kinds of symbols to aid the perception of abstracts. In CAL, the visual environment is also much more interactive with a lot of information-rich feedback (rollover type inferencing on any argument/output, popup lists of proposed functional compositions etc.). I suppose that this sense of "assemblage" is at least partly responsible for the strong sense of puzzle solving. One experiences similar sentiments having designed a nice OO pattern/architecture, but not in the same way.

To the second point, concerning elegance, this is something that is strongly related to the way symbols have such a direct relationship to the semantics of functional programs. Any part of a CAL program that actually compiles, makes a formal and precise (though not necessarily narrow) contract with the rest of the world as to the quantum of processing it contributes. Part of the elegance comes from the richness of the type system and the sets of types of values that can be described as forming this contract. Another part, however, comes from the fact that the function (as a package of processing) is a highly flexible unit. The contract that functions make in a lazy functional language concern the relationships between the input and output values, but these relationships can be composed into larger structures in any way that satisfies the strictures of the type system. Elegance, though is merely a perceived quality, what is more important is how the manifestation of elegance is related to practical effects on the economics of software development.

At a low level, this manifests as a beautiful ability to build abstracts through binding arguments in any order as larger logical structures are created. In other words, the way in which you abstract is fluid, and the little semantic package of the single function can be utilised in so many different ways, compared to the strict notion of a 'call' in procedural languages.

At a high level, this behaviour results in the functional packages being able to be combined (and importantly, reused) very powerful ways, with a direct bearing on the way the intended semantics can be conjured, but always under the watchful eye of the type system - that is able to act in both a confirming and informational way. The latter can feel like epiphany. Many times have I been focussed on a specific problem, and composing functions for that purpose, only to have the compiler's inferred type tell me that I've produced something much more general and powerful than I had thought (sometimes it even tells me that I've been stupid in overlooking that function already in the library that does exactly what I'm trying to do again!).

Today's fun with CAL had all these facets. The qualia of functional programming is quite different to that of OO programming and in many ways you are much more constrained than the latter. Good OO design is certainly critical to creating correct, efficient and maintainable software, and while there is therefore a real spectrum of 'better' and 'worse' designs/implementations for a given problem, much of the structure of an OO program itself is informal to the implementing language and lives in the minds of the developers who created it (and perhaps in their technical documentation). The reasons why certain encapsulations were chosen over others, and why certain code paths/sequences are established are undoubtedly built on reason, but they become major artifacts of the particular solution. In the functional world, things are both more sophisticated and more simple at the same time (naturally, 'complexity' has to go somewhere). Functions are not the atomically executed entities of the procedural world, and their algebraic composition is a very powerful method of abstraction, as described earlier. The type system is much more pervasive and complete, which is a double-edged sword: it forces you to think about the realities of a problem/solution much earlier (which feels like constraint), but it also enables the aforementioned much more meaningful 'conversation' with the compiler. The up-front requirement to conform to formal boundaries in expressing a solution costs a little more at the beginning, but pays back handsomely downstream - both in terms of the earlier/deeper accrued understanding of the problem domain, but also the much higher likelihood that better abstractions have been selected. As any first year Comp Sci undergraduate knows, the costs of correcting bad assumptions/design later in the software lifecycle are much higher than earlier. There are still choices about encapsulation in functional languages (which modules to create, how to deliver functionality in appropriately sized quanta to allow requisite reuse and generality of components), but the packets of semantic themselves, and the manner of their abstraction is far more fluid. The denotational quality of the syntax also has value for reasoning too, but that's another kettle o' fish.

At the end of the day, any developer will get a buzz out of using a tool that allows rapid manifestation of a good solution to the problem at hand (by some definition of "good"). The qualities of the functional approach however imbue a certain concision and confidence to the construction, and with the type system, really appear to have a pseudo-physical quality of "inserting the round peg into the round hole". So it is (I think) that when you stand back from the 'model' you have just assembled, there is a much more tangible quality to the functional construction - that it has been assembled from shapes, and that those shapes in turn had a robust definition. The whole model has been 'validated' by the type system (as an instance of a theorem prover), and you are now the proud owner of some new 'shape' with its tractable, testable, semantic, and its ability to be further glued into an even larger (and impressive?) construct, with some degrees of freedom about which vertices and edges 'stick out' to be perceived on the surface of the larger creation.

Whatever, dear reader, you may adjudge as the real reasons for my trippy experiences, I'm guessing that most developers who take the time to really understand what functional languages offer are likely to come away from the experience (and hopefully some real practice) appreciating some aspects of the foregoing. I'm not personally one of those who would use a functional language for every problem (at least the current batch, with the current libraries available, to say nothing of available developer skill sets), but I'm beyond persuaded as to the very real advantages they offer on certain problems, as a component of applications. Perhaps it is a growing appreciation of this sentiment that is driving the apparent uptake of what I'll loosely call 'functional concepts' within mainstream languages, or extensions thereof. Lambdas/blocks, closures etc. have appeared in Python, Ruby, Scala, C#/LINQ and (maybe) Java. It will be fascinating to see how these fare and evolve as embedded features of languages that are centred around long standing procedural concepts. Certainly these features allow, and even encourage, a more functional style for use where appropriate. However, the basic tenets of a language are a critical factor, and so far these languages are a far cry from the algebraically typed world, combinatorial construction and semantics of a modern lazy functional language.

Right. Back to the OO world now then...

[glow fades away]