Poor unit testing makes hiring and keeping good QA staff very difficult because it’s the rare tester who can feel any sense of job satisfaction testing and re-testing the same code several times, without any realistic expectation that all the defects reported will be corrected the first time around or the second.  Testers are further frustrated by finding and reporting bugs that appear to be quite obvious and should have been found during the most basic unit testing.  Poor unit testing is not usually due to poor programming skills, or even a lack of conscientiousness.  The problem is due to the programmer’s perspective.

When I was a programmer, I was often guilty of the same crimes that I just mentioned.  To make it worse, I often incriminated myself.  After completing a critical module, I was often eager to show someone how the module worked so I would call them over and begin to demo it.  As soon as I began demoing my work, defects, both visual and functional became glaringly obvious to me.  I was often dumbfounded how I didn’t see those defects before I went out of my way to embarrass myself.  

As a project manager I occasionally ask programmers to demo their completed work to me before submitting it to QA when their assignment is critical to a project.  About 9 months ago, I asked a programmer to demo his completed work to me before going to QA because I wanted to make sure the visual concept was implemented properly.  He began demoing his work and immediately found a bug.  He apologized and continued the demo, only to run into another bug a few seconds later.  Then he noticed a couple of visual issues, one related to alignment and a misspelled word.  He apologized again and asked if he could stop to fix the issues and start the demo again later in the day.  About an hour later we began again.  He fixed the earlier issues, but stumbled into a few more issue later in the demo.  I never said a word because my commentary wasn’t needed.  The programmer found all these issues without any prompting from me.  Something began to nag at me about how easily the programmer found his own bugs while demoing the software to me, but was unable to see the same bugs earlier while unit testing, knowing that he was going to demo the code to me later.  Then it occurred to me that something was qualitative different with the programmer while demoing his own code.  The difference was that his perspective had changed from the myopic view of a programmer to the broader view of a presenter.

Programming is a myopic job.  To write software, a programmer must dive deep into the logic that controls the program.  Each minute piece of software requires his/her undivided attention, keeping in mind the many section of supporting code needed to program a function to work properly.  Each screen that a user interacts with is made up of hundreds of these little bits of logic, each requiring all the programmers focus for a period of time.  When programmers test their own work, they don’t realize how myopic their view is, so they miss obvious defects visible outside their current view.  Something changes in every programmers’ perspective they demonstrate their work to others.  The act of demoing lifts their focus so they see more broadly.  All of a sudden, the bugs that were hidden from view become glaringly obvious.

Earlier I explained the need to design processes that support the most common outcomes.  Once it dawned the reason for poor unit testing dawned on me I implemented a new step in our unit testing process.  Programmers are now required to video capture each completed unit test case using video capture software such as Jing.  Programmers simulate a demonstration by verbally explaining the purpose of each unit test, then describing each step as it’s performed.  If they run into a bug or notice a visual defect while videoing the test case, they must stop the video and correct the issue.  Once correct, they must start the video over again from the beginning.  They cannot consider the code unit tested and ready for QA until they are able to flawlessly video the unit test case from beginning to end without interruption and without finding any bugs or visual defects.  Once the video is complete, the video is submitted to QA.  If QA finds any defects in the video, an issue is created in our issue tracking system and assigned to the programmer.  The turnaround from QA back to the programmer is many times faster reviewing a video then executing test cases.  Once the programmer fixes all reported issue(s) he/she is required to create a new demo video with the same requirements as before.  They must flawlessly video the entire unit test scenario again from beginning to end, not just the fix, before resubmitting it to QA.  Only when QA can validate that the video received is without any defects will they begin to execute all the test cases themselves.  This change in our unit testing procedures and QA feedback process has dramatically improved the quality of the code delivered to QA, and reduced the number of bug report/fix cycles between QA and programmers.  Programmers also benefit because they are more confident that the code they deliver to QA works correctly.  

We named this new unit test procedure, “Video QA”.  In the past programmers and QA suffered with what seemed to be an inevitable cycle of bugs reports and fixes.  With the implementation of the Video QA process, no one seems to be suffering any longer.

Most project management tools are designed around the concept of managing large complex projects with critical resource, organizational and dependency issues. Microsoft Project is an excellent example. Tools like this are great for building bridges and sky scrapers which must follow an extremely prescribed process from beginning to end. However, for small to medium size software development projects, they are not appropriate. Software projects require flexibility at many levels that make these more sophisticated project management systems unwieldy. Also needed is the ability to make clear relationships from requirements to features and project tasks, with the ability to easily change requirements or change their relationship to both features and project tasks. Another missing piece with existing project management systems is the ability to track and manage bugs that crop up during the testing process. Having to use a separate bug tracking system just further disintegrates management’s ability to easily control and track the progress of any project.

Another category of tools that has been touted for project management are like whiteboard, allowing users to click freeform all kinds of project documentation, drawings and notes in a collaborative space. Although this whiteboard metaphor does provide a much more flexible communications between team members, it lacks the ability to impose any structure on a project. Products like Basecamp provide the ability to create task lists but they lack the ability to create project hierarchies, or establish dependencies between tasks. Now it may sound like I’m making an argument for rigid products like MS Project. What is really needed is a blending of the two, with much more flexibility then is available with products like MS Project while allowing a user to create some structure when needed. In addition, we need the ability to track requirements cleanly with flexible associations to the project. We also need the ability to track bugs and communicate about them with other team members.

There is still another important piece missing that I haven’t mentioned, but is just as important. As project managers or small to medium size projects, we need the ability to easily prototype screens and navigation in an online collaborative environment. This technology is important for two reasons. 1) Prototyping helps us to communicate our understanding and our implementation ideas to our customers, getting buy in quickly before designers waste any time creating mach ups. 2) Prototyping helps us to explain requirements and implementation concepts to developers before they waste any time building it wrong. The problem is that there are no good prototyping tools. The few I have seen take as much time to prototype as it does for a designer to mach up screens. None that I have seen are collaborative in an online environment. Most companies who have tried to create a good prototyping tool end up making it way too complex, with the idea that if the user creates a sophisticated enough prototype, they can use it to generate code.

I don’t know about you, but I have not desire to generate code from a prototype. I want an online prototyping environment where I can literally draw screens, widgets etc. while my customer or team members are watching and easily couple screens together to illustrate navigation. It should be easy enough to use that it doesn’t feel like extra work that later thrown out. Once a prototype is detailed enough, I also want to tie specific elements of the prototype to both requirements and project features. Then developers will be able to quickly display a visual reference to any requirement or project task. Software development would proceed faster with much more confidence. If a piece of the prototype changes, it would be easy to see which requirements and project tasks would be affected and need to be updated as well. The opposite is also true if a project task were to change. It would be easy to see what requirements and what other tasks might be affected.

Are you frustrated like I am? If not please let me know how you address these issues. The last part of this series will examine what Tekyz is doing to address these shortcomings.

• CRM
• Requirements Analysis & Tracking
• Prototyping
• Systems & DB Analysis
• Project/Feature Tracking
• Project Planning
• Iterative Development
• Bug Tracking

Has anyone ever built a system for a customer that didn’t include all aspects mentioned above. I haven’t. What confuses me about the tools available today to help manage software development projects is that they include very little beyond project task related planning and tracking. And, the majority of them are better suited for building bridges than software. Why has no one ever taken a wholelistic approach to building software and created the tool(s) needed to facilitate the entire lifecycle. The answer in my opinion is that everyone is focused on the details and has lost sight of the bigger picture. Let’s look at this problem from two distincts perspectives.

The first is to look at tools that facilitate the technical aspects of building software. Those tools include IDEs, database modeling and design, code generators, methodology related analysis and design tools such as UML, prototyping, etc. With the exception of prototyping, there are more than enough of these types of tools on the market, and some quite good. However, none of these address the planning and tracking mechanics of a software projects except from a technical perspective. Nor do they help to communicate and coordinate the efforts with the customer, management team and technical staff.