F# Option orElse getOrElse functions

Consider the following function that takes a Sequence of (pairs) int * int.

let f s =
  let o = Seq.tryFind (fun (_, x) -> x < 0) s
  if Option.isSome o then
    Option.get o |> fst 
  else
    let p = Seq.tryFind (fun (_, x) -> x = 0) s
    if Option.isSome p then Option.get p |> fst
    else 10

Essentially I am looking for the first pair in the sequence where the second element in the pair satisfies a particular condition and then returning the corresponding first element or a default (i.e. 10) if no satisfactory pair was found.

That code is pretty ordinary. First of all, I would like to improve on the two calls to Seq.tryFind.

let f s =
  let tf g = Seq.tryFind (fun (_,x) -> g x) s
  let o = tf (fun x -> x < 0)
  if Option.isSome o then
    Option.get o |> fst 
  else
    let p = tf ((=) 0)
    if Option.isSome p then Option.get p |> fst
    else 10

Now if the Scala Option orElse and getOrElse functions were available as well as the Haskell backquotes infix syntax then we could really make a difference.

orElse : 'T option -> 'T option -> 'T option
let orElse o p = if Option.isSome o then o else p

getOrElse : 'T option -> 'T -> 'T
let getOrElse o d = match o with | Some x -> x | _ -> d

// N.B. this is not valid F#

let f s =
  let tf g = Seq.tryFind (fun (_,x) -> g x) s
  tf (fun x -> x < 0) `orElse` tf ((=) 0) |> Option.map fst `getOrElse` 10

Neither orElse or getOrElse exist in the F# Option module. However the Core function defaultArg is essentially getOrElse.

Unfortunately F# doesn't have a way to use a (non-operator) function in infix form, like the backquotes in Haskell. However, we can define operators, which are essentially just infix functions.

let (|?) = orElse

let (|?|) = defaultArg

Now we can write a valid F# version.

let f s =
  let tf g = Seq.tryFind (fun (_,x) -> g x) s
  tf (fun x -> x < 0) |? tf ((=) 0) |> Option.map fst |?| 10

Laziness and Composability
Notice that in the original example, the second tryFind is only executed if the first one is unsuccessful because the then expression of an if statement is lazy, i.e. it is only evaluated if the condition is false.

However functions in F# are strict by default i.e. their arguments are evaluated prior to application. Consequently both tryFinds are evaluated, irrespective of their values, as they are arguments to orElse.

So here is an example where laziness is required to achieve reasonable composability. Implementing this gives the final code.

let orElse o (p:'a option Lazy) = if Option.isSome o then o else p.Force()

let (|?) = orElse

let (|?|) = defaultArg

let f s =
  let tf g = Seq.tryFind (fun (_,x) -> g x) s
  tf (fun x -> x < 0) |? lazy (tf ((=) 0)) |> Option.map fst |?| 10

University lectures

I enjoy learning, especially in the context of improving whatever it is I am doing. I do not enjoy the university lecturing system, even when the content is relevant and the lecturer is good. In contrast to interactive learning in the context of relevant activities, it is very inefficient.

In the industrial model of student mass production, the teacher is the broadcaster. A broadcast is by definition the transmission of information from transmitter to receiver in a one-way, linear fashion. The teacher is the transmitter and student is a receptor in the learning process. The formula goes like this: "I'm a professor and I have knowledge. You're a student you're an empty vassal and you don't. Get ready, here it comes. Your goal is to take this data into your short-term memory and through practice and repetition build deeper cognitive structures so you can recall it to me when I test you."

The definition of a lecture has become the process in which the notes of the teacher go to the notes of the student without going through the brains of either.

Provocative but unfortunately sometimes true. In The Impending Demise Of The University Don Tapscott goes on to write about the generational and technological forces that are impacting the traditional university model of learning.

Carl Wieman, recipient of the Nobel Prize in Physics in 2001 writes in Why Not Try A Scientific Approach To Science Education? about his experiences in educating.

I have conducted an extensive research program in atomic physics over many years that has involved many graduate students, on whose professional development I have spent a lot of time and thought. And over the years I became aware of a consistent pattern: New graduate students would come to work in my laboratory after 17 years of extraordinary success in classes, but when they were given research projects to work on, they were clueless about how to proceed. Or worse — often it seemed that they didn’t even really understand what physics was.

But then an amazing thing happened: After just a few years of working in my research lab, interacting with me and the other students, they were transformed. I’d suddenly realize they were now expert physicists, genuine colleagues. If this had happened only once or twice it would have just seemed an oddity, but I realized it was a consistent pattern. So I decided to figure it out.

He goes on to talk about measuring student learning, research on learning, the role of technology and various approaches tried. Towards the end of his multi-part series he offers the following summary.

Our society faces both a demand for improved science education and exciting opportunities for meeting those demands. Taking a more scholarly approach to education—that is, utilizing research on how the brain learns, carrying out careful research on what students are learning, and adjusting our instructional practices accordingly—has great promise.

Research clearly shows the failures of traditional methods and the superiority of some new approaches for most students. However, it remains a challenge to insert into every college and university classroom these pedagogical approaches and a mindset that teaching should be pursued with the same rigorous standards of scholarship as scientific research.

Confused twitter

Suddenly I have no followers on twitter. I wonder what the probability of all my followers stopping in the last hour is?



Also I am only following about 16 people at the moment, but you'll notice about 36 following icons. This is the third time in the last three days that twitter has magically added about 20 followers, which I have deleted again and again. Is this a feature?

Why did the Government give Gerry my money?

“I’ve been in business since 1961 and I’ve never seen this sort of sales decline,” Harvey said. “I don’t know what we would do if we didn’t have Australia growing to make up for it.”

From Harvey Norman to ‘Persist’ in Ireland Amid Mounting Losses

"It is worth noting that the sector least affected (i.e. with little change in hours worked) is retail, no doubt buoyed by the Government cash packages," the report notes.

From 'Armageddon' averted but recession still likely: NAB

If I recall correctly, the first Government 'stimulus' payment was made just before Christmas, in time for the peak retail period and before the effects of the global financial situation were really being felt by the general population.

Firefox 3.5 password manager changes

I use the Secure Login Firefox add-on and store the password files in Git. Upgrading to Firefox 3.5 changes those files.

• signons3.txt is replaced by signons.sqlite


• key3.db is still required as before

Unfortunately signons.sqlite is a binary file and so doesn't play as nicely with Git.

Flickr scare

Just signed in to flickr and realised my pro account had expired and the free usage limits applied, so some of my old photos were missing!

I quickly paid for the pro account and they were back again.

I did not receive an email notifying me of pending expiry at my primary email account (which I have registered with flickr) and I don't check the yahoo mail associated with my flickr account.

Was quite worried there for a few minutes. Flickr has been pretty good, but a bit disappointed about this.

The Chinese Equity Bubble: Ready to Burst

Amid the current financial crisis, there has been one equity index beating all others: the Shanghai Composite. Our analysis of this main Chinese equity index shows clear signatures of a bubble build up and we go on to predict its most likely crash date: July 17-27, 2009 (20%/80% quantile confidence interval).

From this paper.

Only a few days to go before that time period is up! :-)

Ken Robinson says schools kill creativity

A humorous, but poignant view on how our education system devalues creativity in the interests of industrialism to the detriment of people in general.



I disagree with the implication that mathematics is not creative. Paul Lockhart in A Mathematician’s Lament writes about the education system destroying the curiosity and creativity in maths.

I agree wholeheartedly that the education system values certain qualities in people over others and that can deeply (negatively) impact the self worth of young children on the wrong side of that.

Workingmouse, a short history

I was one of the founders of Workingmouse and started as the CTO. Over time the other founders moved on, one to property valuation, another to .net development/project management and the last became the CIO of one Australia's largest retail companies. I took over as managing director, but stayed involved in development and project management until the end.

Workingmouse' history can be considered as consisting of three broad periods, that don't necessarily have distinct boundaries, but serve to characterise some important aspects of the organisation at the time.

1. Central Control
Workingmouse was founded by four software professionals in January 2000 and incorporated the following month. The company grew rapidly to about 19 people, servicing one major client. The Tech wreck of 2000/01 greatly impacted us and so we went through the unpleasant task of letting most of the staff go and then living off our own personal savings/credit to be able to pay the few remaining people.

In those early days we operated in a fairly traditional manner for the time - waterfall style process, multi-tiered Java applications with specialists in each particular technical role, e.g. Java programmer, DB programmer, HTML/JavaScript programmer, system administrator. I was probably the only person technically across all roles.

We grappled with the usual issues of such a situation. How do we do fixed price, fixed scope work and

1. make a profit


2. have a happy customer


3. have happy staff


4. actually do a good job technically

I eventually concluded that it was very difficult (if not impossible in our context) to achieve all four.

2. People/Agile
Initially one of the other founders was highly focused on quality customer service, while I was driven to balance factors such as technical quality and work/life balance (as I had small children with a medical condition at the time). This dynamic, anchored by a developing mutual respect for one another drove radical changes in Workingmouse. Many things we did bucked the local trends of the time and I discovered just how dysfunctional the international software industry is.

The key changes that started brewing in 2001 were a shift from primarily procedural coding to OO domain modeling, waterfall to agile and an emphasis on hiring and empowering talented, passionate, innovative developers. While I describe these individually, they were very much intertwined chronologically.

N.B. this is intended to convey a historical perspective and I don't necessarily hold these opinions or discuss issues in these terms now.

I was driven to optimise our development speed by trying to work smarter rather than longer. When I discovered the idea of modeling the client's problem domain in an OO style, it appealed to me for several reasons.

• The prevailing multi-tiered architecture ideas of the time involved transforming essentially the same data structure between the implementations in each tier. This seemed like a gross waste of time and energy when a lightweight object model could act as a common data structure (and behaviour), utilised across the entire application.


• The object model could predominantly contain and enforce the various 'business rules' and 'validation rules', making them easier to locate and reducing duplication across the application.


• Objects could provide a more natural mapping between the code and the client's description of their world, thus making it easier for the developer to do the mental translations between them. The benefits being efficiency and less complexity and therefore hopefully less errors.


• Application errors could be reduced by composing small robust objects that ensure their interface doesn't provide a way to manipulate them into an invalid state.

Early on we did all our work with simple servlets and straight JDBC in the Resin servlet engine. I thought we had better catch up with everyone else and I investigated EJB around version 2.0. It didn't take too long to figure out that O/R mapping was what we needed, but CMP/BMP was terrible. Tried out Castor for a while, but it had issues and moved on to Hibernate.

The only Application Server that was remotely usable in development was JBoss. At the time I just couldn't believe how vendors could sell such rubbish and customers would actually spend tens of thousands of dollars buying it. Not only that, but they would then incur massive programmer costs trying to work with it. Running WebLogic in development was akin to applying a giant handbrake. We stuck with Resin until Tomcat eventually stablised enough to use it in production.

For some time both clients and Workingmouse had been facing the high cost of dealing with content changes in web sites/applications. We decided to solve this and build a Content Management System. This was our first (and only) product and we sold it as a service. While it was good, I believe it suffered from two business problems. Firstly the market wasn't ready and secondly we targeted the wrong segment (SME when we should have chosen enterprise). Consequently a decision was eventually made to retire it and focus our energies on services, which was profitable.

At some point I investigated the Spring framework and we originally adopted it to discard our own proprietary configuration libraries. It was only later on, when we moved to TDD, that we valued it as a Dependency Injection container. Back when we started with it though, heavyweight EJB and an App Server was "Best Practice". As far as I know we were the first doing Hibernate/Spring/POJO development in Brisbane and possibly in Australia.

Extreme Programming (XP) is the first agile methodology I explored deeply. It looked attractive as it attempted to address the issues we were experiencing.

• Clients don't really know what they want and programmers are terrible at estimating. Short cycle, iterative development with feedback attempts to address this.


• Clients change their mind regularly. XP attempts to positively support that rather than fight it.


• The genuine opportunity to do a good job. Traditionally, everyone pays lip service to quality, but when push comes to shove, other drivers usually take precedence.


• Emphasis on sustainability, of which maintaining a healthy work/life balance is important.

Like many groups, we struggled with getting stakeholder involvement and enterprise structures. Some experiences are documented in this presentation. We explored TDD and pair programming and questioned the costs and benefits in relation to the capabilities of our people.

The name 'Agile' is a misnomer though. I think of sustainable business agility, but most developers think of freedom. Freedom from being held to estimates they couldn't realistically make in the first place or freedom to do a quality job (from their perspective) or freedom to work in a more social/group manner. Technical people try to spin business benefits on top of all this, often with a long-term outlook. However, because of their essentially self motivated position, they don't credibly understand the short vs long term business issues. Consequently hiring existing Agile professionals was sometimes challenging and naturally I don't fit into the orthodox Agile community particularly well.

From around 2003 when we started hiring again, we focused critically and selectively on finding the right people. We looked for technical talent, passion for improvement and honesty with the hope that we could provide an environment for such people to thrive. Sometimes we made mistakes and commercial realities did intrude, but other times I like to think we really got things right. In the Mythical Man Month, Fred Brooks writes of the massive difference in abilities between programmers. This is consistent with my observations of many people employed as programmers, both staff and clients. The differences in productivity and quality are surprisingly large.

When Ruby started emerging as a fashion, we like many Java developers that used TDD, wondered whether Ruby + TDD would be better. It was easy to dismiss the claim of 10x productivity gains, but while skeptical we undertook a few Rails projects. One of those projects was quite long running and exposed issues with reasoning about larger Ruby applications. Our interest in functional programming was emerging throughout this time and it easily eclipsed our interest in Ruby.

3. Functional
In 2006 I employed Tony Morris, who had previously worked for IBM on Tivoli and their JDK. Tony is one of the most intelligent people I know and he seriously challenged our ideas about software development. We thought we were critical and open minded but he took that to a whole new level and many passionate debates ensued. After some investigation into functional languages Tony and I began learning Haskell.

For some people this was a difficult time within the organisation. I was busy unlearning much of what I had built up, while those involved in ongoing projects didn't have the time to be involved in such a journey. I have always tried to be honest with staff and clients and so because of my fundamentally changing ideas I could no longer participate as easily in the general technical discussions. This situation brought about uncertainty as to the ongoing direction of the company.

In 2007 I decided that for all Haskell's elegance it was not going to be a viable option for enterprise software development services in Australia. Therefore we conceded on Scala as a compromise. It is far more expressive than Java, but still executes on the JVM and interoperates with existing Java code, making it far more commercially palatable. We completed our first Scala project in late 2007.

2008 saw Workingmouse in a position to confidently offer general Scala development and training services. Several more of our (highly capable) people had undertook learning Haskell and Scala. It was interesting to observe how difficult the transition can really be. It is relatively easy to learn the syntax of a new language, but much more difficult to fundamentally change the core abstractions one thinks and expresses ideas with.

After breaking the shackles of Java it is painful to go back to it. Workingmouse needed to find some Scala projects, but the global economic situation was deteriorating and so the clock was ticking. In general, technical management is very conservative here in Australia (that may deserve a blog post of its own) and so only one more Scala project was forthcoming. This was not sustainable and with financial markets crashing it was time for this services business to call it a day.

QOTD

I can honestly say if someone had shown me the Programming in Scala book by by Martin Odersky, Lex Spoon & Bill Venners back in 2003 I'd probably have never created Groovy.

- James Strachan

From Scala as the long term replacement for java/javac?

QOTD

The value of a college education is not the learning of many facts but the training of the mind to think.

- Albert Einstein

From Einstein, by Walter Isaacson, p299.

It is a shame we haven't got this sorted out yet.

Workingmouse R.I.P.

After being part of Workingmouse for the last nine years, it is time for a change. The general economic situation, the emotional cost of managing staff and clients, company administration and the ongoing technical pursuit of better software development can make life a little busy and intense. So apart from one or two administrative things, Workingmouse is now completely dormant and only time will tell if it starts up again.

I would like to thank all the people that made it such an enjoyable and interesting place to work. If you find somewhere else like it, please let me know. :-)

Brad Clow
Managing Director