No, You are not Dumb! Programmers do spend a lot of time Understanding Code…
Wednesday, May 5th, 2010 - By Vineet Sinha
While working on a codebase, developers spend a lot of time understanding code. We might be unwilling to admit it for fear of sounding dumb but the huge amount of time spent on making sense of code is staggering. In fact a conference is held every year just to tackle this problem. The good news is that with the emergence of online communities more and more people are discussing the challenges of working with large codebases openly.
The Back Story
About 10 years ago I was working on the Office Codebase for Microsoft. Even while implementing some really small features, it would take me a lot of time just trying to figure out how different parts of the code related to one another. As it was one of my first really large projects in the role of developer, I just quietly ended up spending the time understanding the code and hiding it from the rest of the team. I was not very experienced then and felt that it was just a personal limitation.
The cause of the problems was not the team or the bad code. It was just the simple fact that having 100+ developers writing code day-in day-out on a project means that there is a lot of code to read – even if you are only focusing on one part of the codebase. After a month or so of working on the codebase, I started to wish that I had better tools that could help me. Realizing that Microsoft would spend the money to buy a license of a good tool if I found something I was happy with, I looked for something and was quite upset to not find anything helpful.
In fact, I wanted something so much that I even wrote a tool in my spare time (more on that in another post). As I dove more into building something useful, the project grew into a Master’s thesis and eventually a PhD thesis before I felt like I had gotten to a useful point. But as I researched the topic I was really surprised to find the significant amount of studies done on the topic.
Results from Studies
Update: We have updated the content below and included it in a table on our website. For an easier to read page of this data see here.
An IBM research by Corbi in 1989 found that more than half of the effort in accomplishing a task for the programmer is towards understanding the system. Later on at Bell Labs in 1992, Davison’s team found that new project members spend 60%-80% of their time understanding code, with the number dropping at most down to a low of 20% as the developers gain experience in the code that they are working with. Another study in 1997 at the National Research Council in Canada lead by Singer et al. found developers spending over 25% of their time either searching for or looking at code.
More recently, in 2006 a study conducted at Microsoft with 157 developers found that roughly equal amounts of time is spent understanding code as other tasks such as designing, unit testing, and writing. In 2007, a survey of over 780 developers at Microsoft conducted by a team led by Cherubini found that 95% agreed that understanding existing code is a significant part of their job. Further, over 65% indicated spending time understanding existing code at least once a day (with over 25% indicating doing it multiple times a day).
Even Peter Hellam in his blog mentions that more than 70% of a developers time is spend understanding code as it is a preliminary requisite when testing code, modifying existing code or even writing new code.
Takeaway
Although many tools which help developers understand code are evolving but their effectiveness in day to day work is something yet to be determined. Till then, as developers it is our responsibility to write clear and understandable code to make life easier for anyone who will work on our codebase in future.
Perhaps John Woods says it best:
“Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.” http://bit.ly/d5BXYA
Architexa
Architexa
IMO, these facts argue for the underrated importance of (good) code documentation, as well as effective modularity. Of course, better tooling would still help, too.
Hi Reedo
Along with well written code, (Good) documentation is another important aspect which does help in understanding code. But, as developers we know that well written/documented code is not something we encounter frequently. As such good tools which help developers quickly understand and document their code become really important.
We need tools and frameworks that are centered around understandability instead of productivity.
I meant languages and frameworks.
I agree that more verbose/understandable languages and frameworks will be a boon to the developer community. But in order for that to happen we will need more intelligent tools and IDEs to help write verbose code faster and help developers understand and document denser code.
Please point out tools that help you understand code.
Himanshu, good point. We are and will be putting links and talking to helpful tools here on the blog. And with tools, I mean both tools that run on your code and also architectural patterns that you might find helpful to include in your code.
Making your code open source is also a great way to make sure your code is easy to understand.
This comment was originally posted on Hacker News
Please expand?
This comment was originally posted on Hacker News
Having anyone review your code is a good way to make sure it is easy to understand.
This comment was originally posted on Hacker News
Part of the problem is how much "infrastructure" you need to learn too. What I like about Python is that a lot of the infrastructure is common and so all Python coders know it. In C and C++ every team seems to use a different set of tools, libraries and debug macros which makes it harder to dive into a project quickly.On another note: is there any decent open source code study tool? I only know of Source Navigator but it always felt bloated to me and the UI was always quite tedious.
This comment was originally posted on Hacker News
Most good programmers I know have some measure of Larry Wall’s three great virtues: laziness, impatience, and hubris.[1]Hubris is defined by Larry as:
"Excessive pride, the sort of thing Zeus zaps you for. Also the quality that makes you write (and maintain) programs that other people won’t want to say bad things about. Hence, the third great virtue of a programmer."
When your code is open source, anyone can see it. A prideful programmer isn’t going to want to hear others badmouthing his/her code. Thus, the hubris this open source programmer feels will compel him/her to write code that is easy to understand and easy to maintain.
At least, hopefully
[1] http://en.wikipedia.org/wiki/Larry_Wall#Virtues_of_a_program…
This comment was originally posted on Hacker News
Very true. I didn’t mean open source is the only way.Code reviews, extreme/paired programming, personal discipline, and lots of experience on development teams are also great ways. I’m sure there are others too.
This comment was originally posted on Hacker News
I suppose the same effect would happen in a organization, perhaps even more because the ones reviewing have power over you…
This comment was originally posted on Hacker News
These all are very good points but well written code is not easy to find and tools which help one to quickly understand code become really important in such cases.
This comment was originally posted on Hacker News
In my experience, I’ve found that processes that influence the desired behavior (through the use of incentives) have been more effective than tools. But I’m certainly no expert and am always looking for better ways to do things.Have you used any tools that you’ve found to encourage well written code? I’d really love to give them a try.
This comment was originally posted on Hacker News
In an ideal world the tools programmers use would enable them to write easily understandable/maintainable code; no matter what their virtues are. The best tools should take these traits into account.
This comment was originally posted on Hacker News
I agree to your point that that we need to write more understandable code(whatever the influences may be). By tools I meant some application which would help me understand already available code (well written or otherwise). Something which would quickly let me jump into the code and start working even when I do not have any background information about the code base.
This comment was originally posted on Hacker News
I agree that the current code study tools are hard to use and we need something more intuitive. Many tools show too much information and are hard to learn. Is this what you meant when you said that the UI was tedious and bloated?
This comment was originally posted on Hacker News
> In an ideal world the tools programmers use would enable them to write easily understandable/maintainable code;Yes. And you chose the right word "enable". Your tools can make it easier to write good code, but you still have to put in the effort.
This comment was originally posted on Hacker News
It seemed to take too many clicks and concentration to do the common things. I’d like to be able to browse through source without the navigator getting in the way, and it felt like I was fighting the UI all the time. The use of a strange GUI toolkit (Tk?) didn’t help. I don’t mind learning a few keystrokes if it means that I can do the basic things like navigating the call graph without taking my eyes off the code.
This comment was originally posted on Hacker News
Some open source code is shockingly bad and difficult to understand. Some is rather good and a joy to work in (eg: Samba). I’m not sure the open source -ness makes a difference. Perhaps it is just that the successful open source projects that you’ve heard about or seen have easy to understand code, and that is the reason for their success (sampling bias)?
This comment was originally posted on Hacker News
What would you want in a code study tool?
This comment was originally posted on Hacker News
Does something like JavaDoc, PHPDoc, JSDoc help?That would be an awesome tool though. Sounds like a great startup idea if someone can figure out how to build something like that
This comment was originally posted on Hacker News
Yeah, developers on Open Source projects often don’t spend as much time on Design or UI Polish.
This comment was originally posted on Hacker News
There is some academic work on the topic. Watch our blog – I am going to tease out some of the highlights.
This comment was originally posted on Hacker News
Yes, a great startup idea. What do you think of our product, i.e the video on our blog (the Architexa blog)?
This comment was originally posted on Hacker News
Looks interesting, I like how you can drill down into a sequence diagram, something I have tried to find in a tool in the past.What languages/platforms does it support? .Net?
This comment was originally posted on Hacker News
Currently we are starting with Java, but do intend to get to other languages. Feel free to vote for your language on the blog or on the HN post: http://news.ycombinator.com/item?id=1338310
This comment was originally posted on Hacker News
Nice. Reminds me of some of the plugins for .Net Reflector, without being interactive.
This comment was originally posted on Hacker News
Nice. Reminds me of some of the plugins for .Net Reflector, except more interactive.
This comment was originally posted on Hacker News
Although there are open source projects which are quite bad, I still think being open source does motivate one to write better code. Regardless, having tools at your disposal to quickly understand/ document code for yourself and maybe relay your ideas to others can be really helpful.
This comment was originally posted on Hacker News
That’s a good point. Bad code can definitely appear anywhere. I like to think that open source motivates good practices, like abhirakshit said, but I have no statistics to back up that belief.This would be a very interesting research topic though. If I had more time, I’d attempt it. Anyone else know of such research, or want to try it?
This comment was originally posted on Hacker News
It’s a very interesting take on UML tooling. Your product has a lot of potential. I’m curious to play around with it once it’s released.
This comment was originally posted on Hacker News
We are letting people in slowly for the trial version of our tool to provide good support. I would suggest you to sign up here (http://www.architexa.com/) and we will get back to you. Thanks for showing interest in our product.
This comment was originally posted on Hacker News
[...] else’s experience on reading code and, during my research, I bumped into very nice articles here, here, here, and here. As I have had to read a lot of code myself I found the readings instructive [...]
[...] This link has some references to studies: http://blog.architexa.com/2010/05/no-you-are-not-dumb-programmers-do-spend-a-lot-of-time-understand... [...]
Please continue discussion on the forum: link