nerdRIDER logo

29 April, 2010

23 April, 2010

3 February, 2010

Does Technical Debt Matter?

Filed under: Computing — Tags: , — Andrew @ 8:59 pm

I have some strong views on code quality.  One of my professional goals is to always attempt to improve my coding with the aim of producing better code.  In this day and age, making software “less broken” is about the most I can hope for.  I cannot foresee a time when written software becomes “perfect” / “bug free”.  Maybe it will – I have learnt: never say never…

Anyway, this is an article akin to playing devil’s advocate.  I am not particularly comfortable with what I suggest below. I have written it purely to get people thinking about the time and effort expended writing software.  As always, I encourage your comments – positive or negative.

One of the odd things about the software industry, is that code “rots”.  This is somewhat strange.  Source code, written in text files does not “degrade”.  Unlike organic reproduction, copying a file leads to a perfect reproduction.  If you kept a copy of code written say twenty years ago, it would still be the same today as it was then.  Things change rapidly in the computing industry.  As a result, it is extremely unlikely that you could use that twenty-year old code on a modern computer.  A different form of “rotting code” exists precisely because the code does change.  Over time, countless little hacks or quirks can be added to an active code base that leads to obfuscation and “unhealthy” code.

The common technique to reducing code-rot is refactoring.  By having comprehensive unit tests, refactoring exercises help keep a code base current and ensure that changes made do not lead to regression bugs.  Working on a well-maintained code-base is a more pleasant experience for a developer.  Well-maintained code is easier to extend and developers have less fear of making mistakes.
“Technical debt” is a term coined by Ward Cunningham and refers to the “price paid” for releasing code for the first time.  He argued that a small debt was useful to incur, as it helped speed up the development process.  It was okay to incur some debt as long as it was “paid back quickly” by refactoring the code.

“The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation…”

Productivity graphs show how progress drops away on poorly maintained projects.  The longer the project runs, the harder it becomes to add features or fix bugs without causing other regressions.  The solution to the problem is to avoid as much technical debt as possible. Following best practices will help achieve this goal.  But is this done at too high a cost?  Following best practices adds extra work and thus does slow down development early in the life-cycle of the project.

Not repaying technical debt will grind a software project to a halt.  If you use an analogy of credtit card debt equating to technical debt, having the software project grind to a halt is the equivalent of declaring bankruptcy.   Obviously, this is not a great outcome, but it is not the end of the world either.

What if your software has run the term of its natural life?  Your software will have been written to meet a specific need.  Maybe that need is no longer there.  Maybe that need is now perfectly met, or the market has been saturated and sales have evaporated.  Maybe every feature that could be added, has been added (including reading mail).    If the project gets “sun-setted” does it really matter how much technical debt is left in the code base?

Not “doing things the best I can” is something I struggle with.  “Doing the best I can” and “doing things well” do not necessarily mean the same thing.  Obviously, the process of software development happens on a linear scale.  Software tends not to be written “the best way” or “the worst way” but rather somewhere in the middle.  If the process your team uses is close enough to “the best way” end of the scale to not be crippled by technical debt, then maybe that is good enough.

24 January, 2010

Say “No” to Band-aids!

Filed under: Computing — Tags: , , — Andrew @ 4:15 pm

Sooner or later, there will be a need to fix bugs in whatever software you work on.  How long it takes to fix a bug tends to be non-deterministic.  Some bugs will be easy to fix, and others not so.  Unfortunately, bug fixes on commercial software are often done the wrong way – under the guise of being done quickly.  The “band-aid fix” is the wrong way of fixing a problem.  The metaphor of the “band-aid fix” extends beyond the software industry, but I.T. has turned it into a real art-form.

At the heart of a lot of band-aid fixes is the notion that you can fix a problem without really knowing what the problem is.  Commercial reality may well prevent a code base from being perfect, but the more band-aids that are applied to the code, the worse the software becomes to work on.

There may be a genuine need to apply a band-aid fix to code.  When there is a real financial loss or damage to customers’ data, expediting a fix is understandable.  Removing this kludged fix should be given a high priority.  It is important to recognise that the band-aid won’t continue to hold a large wound together!  If you do not remove the band-aid and perform proper surgery, the wound will rot.  Once you allow the code to start “rotting”, it becomes difficult to arrest this negative momentum.  It damages the maintainability of the code and encourages other programmers to act irresponsibly too.  It is difficult to put enough emphasis on this point.

Depending on the culture in the workplace, it can be easy to dismiss fixing “less than ideal” code.  Studies have shown how counter-productive poorly maintained code is on development productivity.  I have yet to work with someone in the software industry that would disagree with that thought. Yet barriers are still erected that prevent acting upon it.  There is a vicious circle alive and well in parts of the software industry:

  • Code is recognised to be poorly maintained.
  • Poorly maintained code is recognised to hinder productivity
  • People are too busy to fix old code.

I cannot believe people do not see the irony with this!  Allowing software to get in to this vicious circle is the first mistake.  Programmers need to promote the culture that refactoring is not a luxury, but a necessity.  Allowing some refactoring time on all development work can avoid the problem in the first place.  Digging your software out of the hole created by the vicious circle is altogether a more expensive proposition.  Not refactoring the code at all is even worse!
The idea that refactoring time needs to be allocated with development time appears to imply that you will not be able to push out new features as quickly.  At a superficial level, this is true enough.  Over the lifetime of the code base, this argument would not hold up.  The neater and more maintainable the code base, the quicker it is to perform work on.  In other words, good code is easier to add features to.

The biggest problem I see with a “band-aid” fix is simply that it is not a fix at all!  It cures a symptom in rather the same way that pain-killers can stop broken legs from hurting.  It masks the issue – but it does not mean you’re right to walk home! Masking problems in software, just makes them harder to track down.  Software can be complex enough for one bug to be at the root of several problems. If you only mask the problem, you never know where else the bug will show up

19 December, 2009

Office Politics

Filed under: Computing — Tags: , — Andrew @ 6:05 pm

When you work with other people, “office politics” will always be a factor.  I have heard people say that they did not like office politics as if it were something that they could avoid.  I am not talking about the sort of “Office politics” resulting in the metaphorical stabbing of fellow co-workers in the back. It is true that office psychopaths definitely attempt to manipulate co-workers for their own purposes, but a lot of daily interactions can also be seen as a form of “office politics”.

Internal restructuring has seen my role change recently.  I was working on a framework team, providing code (and various other infrastructure) to various teams in my company.  My “customers” were the teams that wrote the applications that sold to the real customers…  That is, the ones that paid money!
Since the restructure, I have been moved onto one of these teams as a senior developer.  Former “customers” are now team-mates.  When I was working on the framework, I had a certain perception of how our code was being used to create the end-product.  Now that I have become exposed to their code base, I have discovered the truth behind how they use the framework code!  The fact that there are differences indicates some degree of a breakdown in communications.  There is nothing catastrophic about what they have done, but it shows a disparity between the directions the framework and end-products were heading.  This difference was due largely to the difference in motivations between the two teams.

The framework was responsible for the core of twelve different applications.  As such, consistency and flexibility in the architecture were highly valuable commodities.  I would not be presumptuous enough to claim that we succeeded in providing the perfect architecture every time, but those were primary goals of the code we wrote.

The end-products have a far more tangible goal: To make money.  I am not in product management, but having developed commercial software for a long time now, “making money” tends to be about writing software that adds features that customers want and alleviates the worst of the bugs that have been reported.  In terms of priority, “adding wanted features” are more important.  Trade shows never focus on showing customers how the software no longer crashes when you do steps X,Y and Z!

In terms of how this affects code in the long-run, there is a natural tendency to leave “good enough” code alone.  Short term deadlines enforce short-term thinking.  Commercial reality allows code to deteriorate in rather the same way that an untended garden becomes an overgrown jungle.  Active pruning and weeding would avoid the problem, if only it were seen as an important goal.
Given that the software still sells and still provides real value to the customers, it can be seen as an unimportant goal.  The fact that new features become difficult to shoe-horn in to the existing code base is seldom given consideration when writing code.  Which brings me back to my original point on office politics.

Now that I am on a new team, I see the “weeds in the garden”.  No individual issue is worthy of much attention, and so the existing team members simply ignore such issues for more important work.  I would highly doubt there is a piece of commercial software being sold that did not have some degree of this occurring in their own code base.  I have worked alongside my new team members for many years.  They know how important code quality is to me and I know that they will expect me to try and improve their code’s overall quality.  Here is where the “office politics” lie.  I could just blunder in and make changes which I believe are for the better.  I have known programmers who would.  Different cultures in different parts of the world would probably react differently to such an “intrusion”.  In Australian culture, this would not go down well and so it will not be the approach I will take!  I’m also someone who is only too painfully aware of their own short-comings as a programmer. So, tact and and a measured approach, remembering one’s own shortcomings will definitely be the order of the day.   See, even in the most ordinary of jobs, “office politics” will play a role!

5 November, 2009

Usability

Filed under: Computing — Tags: , , , , — Andrew @ 5:02 pm

It takes a decent amount of time and effort to design a good user-interface.  One of the problems faced when making a user-interface is that it can take an enormous increase in effort to make an ordinary interface into an extraordinary one.  You may have come across a user interface (be it for a web-site, or an application) and been absolutely flummoxed by its operation.  Unfortunately, that does not mean that a great deal of time and effort were not spent trying to simplify it.  (Of course it may mean that no time and effort were spent trying to get it right!)

There is an extra pressure on designers of external web-sites.  Get it too far wrong and your customers go off to your competitor’s web-site.  In my experience, application developers can get away with worse user-interfaces.  If the program has the features people want, people will make the effort to learn how to use the application.  This should not be seen as an excuse not to care about the user-interface.  There is a saying that if your customers are not aware a feature exists, then it doesn’t.  Unfortunately, most user interfaces end up obscuring some functionality.  In a feature-rich application it becomes increasingly difficult not to do so.

Every time I hear a person talk about “learning” software, I feel that somehow the software has failed.  I would like software to be so intuitive that using it is “natural” – rather than a learned action.  It is probably an unrealistic expectation that all software will be like this, but that does not stop it being a worthy goal to work towards.

When I talk to non-technical people about using software, the thing that becomes apparent is that they all expect to have to learn how to use it.  No-one expects to sit down in front of a new word-processor and just use it to do their job.  One disheartening example came with the release of Microsoft Office 2007.  For me, the ribbon was a huge step in usability enhancements over the traditional tool-bar and menus approach.  The one resounding criticism I hear with Office 2007 was from existing Office 2003 (and prior) users:

“I used to know where everything was and then they went and changed it all.  Now I have to re-learn where things are”

Microsoft puts a great deal of time and effort into usability.  Hopefully, this means the learning curve for Office 2007 was not as severe as with previous versions.  The ribbon was designed to be “a better way”:  Task oriented user-interface is meant to be superior to functional oriented user-interface.  People have been “brought up” thinking along the lines of functional software rather than thinking the computer will aid them in completing their task.  This mind-set will change over time and wide spread adoption of task-oriented user interfaces.

If you ever have to write a user-interface remember this:

  • You either spend your time getting it right, or ask the users to spend their time figuring it out.
  • The world does not need more software that is difficult to use.

30 September, 2009

Where are the good programmers?

Filed under: Computing — Tags: , — Andrew @ 12:48 pm

There seems to be a trend amongst programmers who blog.  They all tend to say that they write rubbish code.  (Some put it more poetically than others…)  I think it is a good idea to steer clear of the Rock-star programmers, but are all who blog, bad at coding?  Or are they merely filled with a sense of modesty endued by self-preservation? (Needed because the Internet is a big scary place and you can’t hide from the knockers forever.)

From my own perspective: because quality takes time, there is always the sense that with more time I would have done a better job.  That is probably true to a certain extent – but there is definitely a point of diminishing returns.  That, plus the fact that I definitely have a finite amount of intelligence means that the quality of my code will probably never exceed a certain level.  Someone smarter than me could possibly turn out better code than I could ever hope to.   Extra intelligence however does not always guarantee better results.  “Care” is an attribute that counts for a lot when writing code.  “Careless programmers” write rubbish code and I find that particularly offensive if I know that they are better problem solvers / generally more intelligent than I am.

Reflecting on my own code at a later date often reveals a painful truth.  Yes, I too write some awful code.  Even code that I was once quiet proud of, I no longer see through rose coloured glasses.  I probably notice this due to looking at the code from a different perspective.  This is impossible to do at the time as you tend to be so engrossed in the code that it seems to be simple.  (To me, simple code that works is a close approximation of good code)

Different perspectives for code arise with different usage of the code.  Code that sticks to some simple rules lends itself to re-use.  Code re-use is somewhat a holy grail of programming, but for a business, it is not as important as having the code you write make money.  Joel Spolsky places a strong emphasis on finding good, talented programmers and judges them as the people who are smart and get things done.

Placed solely on this scale, I have known quite a few programmers who “pass”.  But for some, there is a high price to pay, in the form of code maintainability.  I willingly concede that for the sake of getting a “version 1.0” code base out the door and selling, making code “good” is a luxury.  But carrying on with a relentless drive to push new versions out is counterproductive.  Extending and maintaining a bad code base takes more resources and there have been documented cases where lack of progress due to the bad code base is the eventual undoing of a project.

Maybe this indicates that there are different sorts of “good programmers”.  The ones who ensure there is a product to sell and the ones that ensure that sins of the past are dealt with in a timely fashion.   I suspect software projects need both these types of programmers to succeed. I also suspect that these two groups of programmers annoy each other due to their different outlooks.  But that’s a story for another time.

5 August, 2009

Retrofitting test cases

Filed under: Computing — Tags: , , — Andrew @ 8:41 pm

The main project I work on, has an automated unit test application which is built and run as part of the build process.  For our office, the team was an early adopter of the concept, but unfortunately this does not translate into extensive and well maintained set of unit tests.  Put politely: it would be good to improve this coverage.  This then raises a fairly obvious question: Where do you begin?
First of all, it is worth analysing the statement: “It would be good to improve this coverage”.  There are a couple of benefits to having unit tests and these benefits help explain the statement.
Avoidance of regression bugs.  This is reasonably self-evident.  If you have sufficient coverage of your classes, you cannot introduce faulty logic without a unit-test failing. Over the years many quality assurance staff have told me “the earlier you catch a bug, the less expensive it is to fix”. So many, in fact, that I now believe them.  (In truth I don’t think I ever doubted the “earlier = cheaper” argument) Anyway, if you are running unit tests as part of the build process and building regularly it stands to reason that any bugs introduced and caught, will be fixed cheaply.
Code re-use.  A less obvious benefit is that the code you are testing now has two uses.  One in the application and one in the test-case.  While this may seem a contrived second use, it should help with code abstraction.  The theory goes that the more usages a class has, the less chance it has of being tightly coupled with other classes.  Tightly coupled classes increase code complexity.  The more tightly coupled they are, the more likely a change in one class will introduce a regression bug in another class.
Now that we have defined a couple of benefits that we hope to achieve through the use of unit tests, it helps define where we should begin.  We want the unit tests to reduce instances of regression bugs and improve code abstraction – which is enforced by re-use.  History logs of the revision control system can be studied to show the frequency of changes in a unit.  If a mature project has a hot-spot of continual changes to a given class, then that may well be an indicator of frequent regressions and “hopeful” changes rather than “thoughtful” changes.

There is a very good chance that such a class violates rules of the single responsibility principal and in turn makes writing unit tests for it an unviable proposition.  Now that we have identified what is wrong, we have a good place to start:

  • Look through the class and identify the different responsibilities the class has.
  • Extract the responsibility that is least entangled throughout the class.
  • Write unit tests for this new class.
  • Rinse, repeat.

Once you have extracted a new class and written the unit tests, your changes for it are not necessarily complete.  As you extract more classes, there is a chance that your new class can be further decoupled from the original.  In my experience, the important thing to remember is that just because you remove a responsibility from a class does not immediately decouple the two classes.  Proper decoupling can take several iterations.As I stated, start with the easy abstractions first.  As classes become less entangled, the harder areas will become easier to deal with.  At least, that’s the theory!

24 July, 2009

Second place

Filed under: Computing — Tags: , — Andrew @ 9:02 am

I have a nasty habit of picking the “second placed” technology.  Fortunately, I was too young to have invested in BetaMax and do not have a collection of HD-DVDs lying around, but that is the sort of thing I am talking about.  I suspect I have since thrown it out, but at one stage, I did own a copy of “OS/2 Warp” on 3.5 inch floppy disks.  (I did purchase this prior to the release of Windows 95, I hasten to add!)

When 3G phones were introduced in Australia, I was an early adopter and bought a Motorola A920.  The thought of application development on a phone was an interesting prospect, but any enthusiasm quickly disappeared with the fact it was a “locked platform” that required certification or a certain level of hacking to put applications on it.  The A920 was something of a flawed gem.  Many of the hardware features of the phone were not supported in the initial release of the firmware.  Bluetooth, the IR receiver and GPS functions were all “locked out”.  If I recall correctly, application development required a commercial C++ compiler as well.  As such, apart from the addition of a File Manager / Explorer, my A920 stayed remarkably “standard”.

These days, phone/PDA hardware is significantly more mature.  The vast bulkiness of the hardware has been lost, replaced by sleek stylish devices.  Of course, there are multiple players on the market at the moment, but two of the biggest contenders at this point in time are the iPhone and the HTC Magic with its Android O/S.  It would be remiss of me not to mention the Windows Mobile or Symbian O/S – so, now I have. :-)   To be fair, there are a large number of devices on these two platforms, but they do not capture the public’s imagination, the way the iPhone does.

I would love to write some small applications for a phone.  Nothing serious – nothing that is going to launch me on a stellar career path to be CEO of the next exciting start-up.  For a PC owning hobbyist, this makes Android an obvious choice.  Applications are written in Java and there are Eclipse plug-ins complete with hardware emulators.  This makes the cost of entry free.

Compare this with the iPhone.  Applications are written in Objective C.  Whilst I believe the development tools are free, the cost of entry is buying a Mac.  Now I realise that if you start with a Mac platform, that makes the entry point approximately the same as for Android development, but unfortunately for me, that is not the case. Limited introductory reading has also led me to believe that the API provided by Android is superior to the iPhone API and that getting applications approved by Apple can be problematic, if your vision doesn’t align with Apple’s.
Overall, Android looks to be the obvious choice.  There is just one problem: The iPhone is killing Android in the market place.  Public awareness is heavily tilted in favour of the iPhone, thanks to aggressive advertising campaigns.  I have heard plenty of people saying they wanted to get an iPhone.  I haven’t heard one person say they wanted an HTC Magic.
So which phone would I be buying?  For the time being, neither.  Wanting to write applications and actually getting around to doing so are two different things.  When I had my A920, I discovered it was quite good at doing everything, except being a phone.  Now I own an unremarkable 3G phone that was purchased solely because it was the smallest on the market at the time.  – This was a reaction to the size of the A920, which was so bulky, it was inconvenient to carry. The A920 was not the only consumer electronic device I have owned that was software extensible. If past experience is anything to go by, customisation of such devices is little more than a pipe dream for me.

Realistically, I should attack this problem the other way around.  That is:  have an Android development environment and write and test applications on it.  If I reach the stage where I have written sufficient applications to justify buying an actual phone, then I shall – as long as I can convince myself I’m happy to buy the second-placed technology.

5 July, 2009

Fun with Delphi 2009!

Filed under: Computing — Tags: , , — Andrew @ 9:12 pm

All work done on our project is subject to peer review.  Any code submitted to the version control system, must have an accompanying “change request” which has a unique number.   The reviews are done “incrementally”.  That is, “diffs” are compared to ensure the changes are correct.  (Or at least, that’s the theory!)

To help facilitate this, a Delphi client application was written to access the information necessary.  The diffs are stored as HTML files (generated by a server side application) which an embedded Web browser control displays.  An external “diff tool” can be used for more powerful operations than the web browser allows.  Although in theory, a normal web-browser could be used to perform the review, the HTML diff files are limited in their user-friendliness and non-trivial changes end up being examined by the external diff tool.

The problem I have, is that I work in a remote office to where the “server” is.  Network latency and the low specification of the “server” takes the review process to a new level of tedium.  However, as the review tool was written in house, I had the power to do something about it!  Although I have been using Delphi 2009 since its release, this was the first opportunity I had to put together several of its new language features.

I wrote an simplistic “cache” for the program, that copied the files it needed to reference to a temporary directory on my own machine.  To do this in a unobtrusive manner, the files are copied using a background thread.   The cache keeps a request list, and a list keeping tabs of what files are currently held in the cache.  I utilised closures and anonymous methods to access these lists in a thread safe manner and the generic storage classes found in the Delphi libraries for the lists themselves.  As these classes support iterators, I was even able to use these too. (Yes, I realise iterators aren’t “new” to Delphi)

I know none of this is a “new trick” to the managed languages such as C# under .Net 2.0 and onward, or later versions of Java.   I was never a C++ developer, but I suspect some of these “new tricks” were always possible with it.  Delphi’s TThread class still seems to me a riskier way of writing multi-threaded code than C#, but it is so cool that an “old favourite” can now play along with some of the newer languages and do so “natively” rather than requiring a virtual machine to do so.

Older Posts »

Powered by WordPress