Speed vs Quality

It seems that there were some issues with voting machines in the US recently. Paul Johnson proposed a reason why.

Too often projects are run predominantly with a short-term business focus. Lip service is paid to quality, but when things get tight, it is not actually that important. However, all that changes once the project is 'finished'. Suddenly the longer-term becomes more important, but by then it is too late.

In a software project, there is generally a relationship between quality ¹ (from the developer perspective) and speed of development (from the business perspective). You can sacrifice quality for speed, but as the project becomes bigger or more complex, it becomes harder to change and so development speed decreases. The short-term gain (probably measured in days/weeks) becomes a long-term loss.

At the other end of the spectrum, you can spend most of your time focussed on quality and not worry about speed of development. As previously posted, this is not commercially sustainable either.

IMHO, one of the most interesting things Agile Methodologies bring to a project, is the opportunity to make development cost/speed and quality more transparent throughout the life of the project. In the end though, someone has to make decisions about quality and speed on a day-to-day basis. It seems to me that a business savvy developer will be more likely to succeed than a tech savvy business person. It is a tough line to walk though.

¹ Quality means different things to different people. In this case I am referring to both how well code is written and whether it exhibits correct behaviour.

Lockharts Lament

Lockharts Lament is a very interesting and worthwhile read. It is about mathematics, art, society and education. Especially education. A few interesting quotes and ones that made me laugh:

"The only people who understand what is going on are the ones most often blamed and least often heard: the students. They say, “math class is stupid and boring,” and they are right."

"High School Geometry: Instrument of the Devil"

"We’re killing people’s interest in circles for god’s sake!"

"The first thing to understand is that mathematics is an art."

It is a about 25 pages long, at least read the conclusion (the last two pages).

Via Enfranchised Mind.

Tim Sweeney - FP, Type Systems in game development

Tim Sweeney of Epic Games spoke at POPL 2006 about development of the successful Unreal 3 game engine. In his slides, he looks at challenges as well as potential solutions such as functional programming, stronger, static type systems and STM. It seems he has now started research on Unreal 4. It will be interesting to see what comes out of his work.

There is also some comments from Tim and his staff on his slides at Lambda The Ultimate.

University - context free zone

About 15 years ago, I was finishing my undergraduate degree in Computing Science. I remember feeling like it was largely a gross waste of time, but I had come this far, so I should finish to get the piece of paper. Furthermore, upon finishing my degree I vowed not to do any more programming! Somewhat ironic since I now run a software company.

Fast forward to the present day and I have just started the first subject of a Masters. This subject is an undergrad course on Discrete Mathematics, which I am really enjoying. So, why the difference between these two university experiences?

I am postulating it is because I see the relevance of the topic to my own context. I am able to apply what I am learning in some meaningful way. There is a reason for learning this stuff!

Note the university has helped very little here. They no longer have a program containing significant theoretical computer science. So I am investigating courses offered in the school of Mathematics and selecting them best I can.

Back to my undergraduate degree. I chose Computing Science because I had an aptitude for computers. I had already learnt simple imperative programming and relational databases at high school and in my own time. So listening to a lecturer read the database textbook was pretty boring and why was I doing a compulsory accounting subject when I had enrolled in computing, not business?

As far as I can remember, the University did little to give me a reason as to why I was studying the subjects I was, or to draw me in and engage me on the topics at hand (apart from a crypto subject in final year, run by the Dean at the time). I went straight from high school to uni and had never worked on a significant software project, so I certainly didn't have the context to find relevance myself.

Now I look around at the first year students, many straight out of high school, doing the maths subject with me. I wonder how many of them will probably grow to dislike this subject, as they struggle with the formality of the content for the first time. How many will study for the exam and then promptly forget most of what they learned. What a waste of time, effort and money!

If my conjecture about context and relevance is correct, would it be better to study programming as an apprenticeship - where there are periods of practical work in industry, separated by periods of classroom time? Or should Universities simply do a much better job?

Microsoft Research pays off

With the similarity between C#/Java over the past years, it is interesting to see Microsoft starting to make some serious moves to forge ahead. Although the syntax isn't anywhere as nice as Haskell. :-)

Then there is F#, which is getting Visual Studio support. A friend of mine also mentioned that VB is getting some serious functional influence as well - we shall see.

Now over in the java world there is still talk and debate regarding closures and type inference. That's pretty poor competition. Fortunately for the Java community, there is Scala.

Two influential people in all this - Eric Meijer and Don Syme from Microsoft Research - will both be speaking at the Brisbane JAOO conference in May.

Mac the Scoodi cat

Just chillin'. Watching the world go by.

Mac belongs to a friend of mine, who is involved with Scoodi.

Converting a Subversion repository to Git

Previously, I had tried converting from a local Subversion repository to Mercurial. I thought I would try converting the same projects into Git.

Firstly, to install Git on the mac with Subversion support:
$sudo port install git-core +svn

Secondly, setup an authors file to map Subversion committer names to appropriate Git format.

Now my local svn repository has multiple projects in it. The layout is like:

bar/artifacts
    branches
    tags
    trunk
foo/artifacts
    branches
    tags
    trunk

git-svnimport seems to expect the standard svn directories (branches, tags, trunk) to be at the root. So for each import, the paths to these directories need to be specified explicitly.

Foo is a fairly new project with no branches, 2 tags and about 27 commits.
$git-svnimport -C foo -A authors.txt -T foo/trunk -t foo/tags file:///path/to/repos

Completed with no errors.

Bar is an older project with 1 branch, 13 tags and about 464 commits.
$git-svnimport -C bar -A authors.txt -T bar/trunk -t bar/tags -b bar/branches file:///path/to/repos
Initialized empty Git repository in /Users/cbrad/tmp/bar/.git/
...
fatal: Needed a single revision
64: cannot find commit 'origin'!
...
Use of uninitialized value in system at /opt/local/bin/git-svnimport line 877.
fatal: Failed to resolve '' as a valid ref.
Cannot create tag 10:

There were many of the 'fatal: Needed a single revision' errors.

This article mentions the potential for issues if the svn repository layout had changed over time. The bar project was one of the first things in this repository, so it has been moved and changed several times, as the overall repository layout evolved. I tried doing imports around the movement points in the history, with the view to merge (like in this post), but that also failed.

I guess I can start a fresh Git repository, add the current project head and keep the svn repository lying around for reference, but I was hoping not to have to do that.

John Goerzen and Git

John Goerzen has been writing about switching from Mercurial to Git recently on his blog.

Not Sustainable Software Development

Locally there is a group that has put together a number of practices (named Sustainable Software Development or SSD for short) for use in software development projects. I know a number of the people in this group. They are talented software developers who are passionate about what they do and are genuinely searching for how they can do it better (qualities I respect). I do have concerns about the practices though, as I don't see them as sustainable.

Premises:
1. Commercially, at some point, a consumer will stop paying for something if they don't perceive enough value from it.

2. SSD does not deliver value that the consumer (person/organisation paying for the development) can perceive in a timely manner.

Conclusion:
SSD is not commercially sustainable.

The basis for premise 2 is that the daily, primary focus of developers in SSD is to 'refactor' the codebase for improvement, rather than delivering customer visible features. Generally stakeholders use visible features as a significant aspect in their valuation of a project.

The biggest risk IMHO is that I see SSD enabling and encouraging a very developer centric situation. Essentially as a developer you can do a lot of things that you want to do, without having to have an equally balancing consideration of how that will help the business meet its objectives. This is the opposite of the more common situation where the business drives an unachievable workload without consideration of the developers and the quality of their work.

Instead of either of these scenarios, I think we should be aiming for projects where the business values and empowers the sustainability of developers and the developers deliver to both the short term and long term objectives of the stakeholders. A win-win for everyone.

To achieve this, there needs to be a cooperative approach between all parties involved. However, a cooperative approach will only work effectively once trust has developed. To develop trust requires everyone doing the hard yards of demonstrating their concern in meeting the needs of others.

Finally, one of the interesting things I have observed from some practitioners of SSD is their tendency to break their code down into small chunks, which they will attribute to their interpretation of TDD. However I suspect it is more an unconscious movement towards better composability as a means to deal with complexity. The ability to implement a new requirement, that looks similar to an existing one by composing the building blocks of the existing implementation with whatever is actually different about the new requirement is a significant step in managing complexity. Unfortunately most current software does not decompose well, due to the languages we use and the way we use them. Brian Beckman talks about composition and complexity in this video.

Converting a Subversion repository to Mercurial

I was unsuccessful in my last attempts at converting from a Subversion repository to Mercurial.

Yesterday I thought I would give it another go. I setup a new local Subversion repository and imported a dump from one of my existing repositories. In that repository I have a fairly new project (let's call it foo) which has the standard Subversion layout (trunk, tags, branches), plus an extra directory named artifacts. There are no branches, 2 tags and about 27 commits.

$hg convert file:///path/to/repos/foo

Completed with no errors.

That looked promising, so it was time to try the biggest project in the same repository (let's call it bar). It has 1 branch, 13 tags and about 464 commits.

$hg convert file:///path/to/repos/bar
assuming destination bar-hg
initializing destination bar-hg repository
scanning source...
** unknown exception encountered, details follow
** report bug details to http://www.selenic.com/mercurial/bts
** or mercurial@selenic.com
** Mercurial Distributed SCM (version 0.9.5)
Traceback (most recent call last):
File "/opt/local/bin/hg", line 14, in ?
mercurial.dispatch.run()
File "/opt/local/lib/python2.4/site-packages/mercurial/dispatch.py", line 20, in run
sys.exit(dispatch(sys.argv[1:]))
File "/opt/local/lib/python2.4/site-packages/mercurial/dispatch.py", line 29, in dispatch
return _runcatch(u, args)
File "/opt/local/lib/python2.4/site-packages/mercurial/dispatch.py", line 45, in _runcatch
return _dispatch(ui, args)
File "/opt/local/lib/python2.4/site-packages/mercurial/dispatch.py", line 348, in _dispatch
ret = _runcommand(ui, options, cmd, d)
File "/opt/local/lib/python2.4/site-packages/mercurial/dispatch.py", line 401, in _runcommand
return checkargs()
File "/opt/local/lib/python2.4/site-packages/mercurial/dispatch.py", line 357, in checkargs
return cmdfunc()
File "/opt/local/lib/python2.4/site-packages/mercurial/dispatch.py", line 342, in
d = lambda: func(ui, *args, **cmdoptions)
File "/opt/local/lib/python2.4/site-packages/hgext/convert/__init__.py", line 380, in convert
c.convert()
File "/opt/local/lib/python2.4/site-packages/hgext/convert/__init__.py", line 257, in convert
parents = self.walktree(heads)
File "/opt/local/lib/python2.4/site-packages/hgext/convert/__init__.py", line 97, in walktree
commit = self.cachecommit(n)
File "/opt/local/lib/python2.4/site-packages/hgext/convert/__init__.py", line 204, in cachecommit
commit = self.source.getcommit(rev)
File "/opt/local/lib/python2.4/site-packages/hgext/convert/subversion.py", line 245, in getcommit
self._fetch_revisions(from_revnum=revnum, to_revnum=stop)
File "/opt/local/lib/python2.4/site-packages/hgext/convert/subversion.py", line 620, in _fetch_revisions
for entry in self.get_log([self.module], from_revnum, to_revnum):
File "/opt/local/lib/python2.4/site-packages/hgext/convert/subversion.py", line 275, in get_log
for p in parent(stdout):
File "/opt/local/lib/python2.4/site-packages/hgext/convert/subversion.py", line 262, in parent
raise SubversionException("child raised exception", entry)
libsvn._core.SubversionException: ('child raised exception', 160013)

Not so good. However, if I just try converting the trunk instead of the whole project:

$hg convert file:///path/to/repos/bar/trunk

The conversion completes with no errors.

In summary, trying to convert projects in Subversion to Mercurial has not gone smoothly. Doing a quick google search reveals a recent Mercurial trac issue that looks similar. Time to put this aside for now and switch to evaluating git.