D15c0v3r3d L05T

I've been an observer of discussions recently where the existence of RAD environments/tools are called into question. Discussions revolved around Lotus Notes, Rails, .Net and included some other random references to tools that put development power in the hands of people that really don't deeply understand technology. Talking through languages and frameworks that require more skill and planning revealed some startling facts surrounding RAD technologies and their tendency to maul developers into frustrated hackers.



Typical RAD tools and projects harbor:

• Development that lacks planning


• Lower code / technology understanding


• Less focus on responsible coding & reuse


• Lack of Coding Standards


• Poor user experiences

There are several facets of the questions against RAD tools which I will address in this post. Although my greatest exposure is via Lotus Notes and MS VBA in Office, I see glaring similarities across multiple technologies. I'm going to try to keep this light-hearted... but for some of you the flames will burn quite hot I'm sure.

Planning is over rated, especially when I can jam something together in a few hours that works:

Around our office we have some catch phrases around this mentality, "Dump It", "Ship It" and "80 Percent" most frequently. (With the occasional "Oh, It's just a simple template change")



For years I've been disturbed by the glaring absence of skills and a lack of desire to advance skill level by a number of Lotus Notes development colleagues I have worked for and with, though I pretty much expected that attitude from the VBA developers I knew since Microsoft nearly always threw them under the bus on upgrades and product add ons. Almost universally I have observed an almost adolescent approach to developing applications with little or no regard to sustainability and scalability. As I watch drag and drop design of views and forms with almost no concern to standardization and consistency I am taken back to the days when I worked with erector sets as a child. Erector sets were a great introduction to what today I have grown to call the "kloodge"... slap some pieces together, throw on some wheels and a boom and call it a tank, or a crane, or an airplane... depending on your viewing angle. If the tool will let you build something with little or no planning, ad hoc of sorts, then the door has opened for extremely poor and risky solutions.



Back in the days where waterfall was the way projects ran we found RAD to be a bright and shiny way to shortcut all of that arduous up front work to preload and scope the project. With the introduction of Agile approaches the drag we once felt to how projects evolve is less of an issue. Clear vision, sprints, and frequent refactors as the application grows allow design to become more elegant and ultimately more reliable. You plan what you are going to code, you code it the way you said you would, and you release it to the wolves.



In an hour I can craft and deploy a Rails application running on mySQL without knowing SQL, Ruby, and Web Integration into the DBMS. I then could spend several hours hacking controllers and making viewers look pretty. The application might do exactly what I want it to do at first... but when my model changes a little will I be able to sustain the application? Probably not. That's what we fight in RAD, speed to market vs. sustainability... speed and sustainability... sustainability and scalability... and interoperability... and compatibility... and usability.



Let me encourage any of you that have that project time bomb, or extremely urgent fix looming over you. There is only one person around you that can make sure the application, the user, and the boss is protected... that's you. Hurrying through a feature or fix is NEVER worth the consequence of failure, squelch the desire to come out looking like an Olympian and focus on being the trusted mechanic. Breath, map out your expected behavior and code to match it.



Code reuse is too hard... and Copying code is safer anyway

Indeed RAD tools have led to some fairly unique ways of accomplishing tasks for programmers. You could fill my web server with laughable code hacks you've done around limitations within easy development environments. There is no argument that each programmer has a personal style they can lend to coding in applications, but far too often it is taken too far and what is left behind can be a tangled mess. I often times argue with myself on whether a piece of code I write will be intuitive enough for a developer following me to understand. I believe in establishing reliable patterns of development for teams, I lobby for the re-use of code rather than making "safe" copies. Most often I see whole chunks of code copied and pasted into new libraries because the person doing copy/paste coding doesn't want to break anything, and even worse that same person doesn't usually understand what they are copying. How often have you copied a piece of code and just hacked at it until it worked? Abysmal.



I remember a few years back where I refactored a section of LotusScript that contained roughly 20 blocks of identical code (if statements) in which the only difference was the name of a field that was referenced. As I was tasked in adding a couple new features I took some time and refactored those twenty or so blocks into one loop within which I iterated a list of field names to update. Once finished there were fewer lines of code and a wrap that I felt was something the development staff could support easily. Roughly a week after my refactor I received a requirement to do validation on a few more fields... imagine my shock when the code I wrote was remarked out and replaced with the old code with the note "...removed loop below and replaced with standard if statements...". The kicker is that we had no standards, there was no drive for standardization, I had simply breached the other developer's comfort zone of copying, pasting, and tweaking.



A great friend and developer, Bill Wheaton, once mentored a couple people in close working proximity to me with a statement that will always live with me: "Unless you have completely read, analyzed, and understand a piece of code... don't change it." Since copying and pasting of code is typically performed for speed of delivery and avoidance of familiarity I would take Bill's statement a touch further and charge any developer to resist EVER doing a copy/paste of code. The minute you are copying and doing a partial reuse of some code is the minute you should realize it's time to refactor it. Copying code is not re-use... it's abuse.



Standardization doesn't work... RAD Code Development is an art form

Here's a real eye-roller. I remember hearing project managers resisting code standardization petitions from large organizations that had huge successes with initiatives in COBOL, C, and JAVA projects. Somehow the "artform" mentality stuck. I'd classify most people using that statement as classically misusing the concept. I recently was working in an application's import feature where there were 3 unique approaches to how the messages coming out of the import would be logged. All 3 approaches sent an email message containing events that had transpired during a given import process, but all 3 logs call different libraries, all 3 do counters differently, and all 3 were done by different developers. One feature, 3 log styles... and the system users getting the logs wondered why all the logs looked different. Since we weren't focusing on standardization I wrote a 4th type of logging just to keep things consistently out of consistency. (::grin::) I actually folded in a couple custom class libraries that allowed me to ditch all 3 previous logging styles and replace them with one I had been using in other portions of our application for awhile. The new approach allowed VERY flexible logging approaches that were configurable via templates external to the code. The most important part of that particular code change was team buy in. Even to those who don't fully understand the mechanics of the new code can see that we are better off now because of the black box nature of it's design.



All of that just to point out that a pattern of development that produces components that stand alone, are predictable, and re-usable requires standardization. Holy cow, those three are pretty much standards on their own. Take my word for it, ignore developing towards standards and you are destined for failure and embarrassment.



We trained the users how to use the system, if they do something different and it breaks then it's their fault

For years I developed under the assumption that it was always my responsibility to protect our systems from the people that used the system. My applications were released with lots of safeguards against the user doing something they shouldn't and messing up my workflow or data. I considered my self fairly successful and prided myself in high code reliability. Then one day Jason Rabinowitz refactored my thinking. He observed that what we really need to be focusing on is protecting the users of our applications. I melted on the inside when I thought of what a huge difference in the process of design, development, and testing this was... I melted but was and AM totally in agreement.



Slapping a bunch of UI controls and some logic together is easy, bolt on some security and validation to keep the app running reliably is tougher, but to get in there and make sure the users don't lose work or suffer for our vigilant attempt to keep the system core functioning is down right hard. I'm afraid the general demeanor of RAD development does not encourage protecting our system users from the system itself.



My story requires some set up for anyone who does not program in Lotus Notes. If you are familiar with Notes then you can just go ahead and skip to the next paragraph. Lotus Notes is a client server environment where one or many document centric databases are forged into applications. The application might me a Discussion application or it might be a CRM. In Lotus Notes there are a number of macro type formulas we can use, we call this feature "Formula Language". The formula language contains a number of extremely powerful actions that are ready to use and leverage almost trivial syntax. Three of the many formulas available are:

• @DBLookup - given a database server, path, view, key, and fieldname will return a value or list of values contained within a document's field. This formula is very useful when the design of a database form has some workflow that needs the system to retrieve some data for a record that lives on another document. An example using this would be status:=@DBLookup( "HRServer";"common\healthcarePlans.nsf";"consultants";"Rich Wheadon";"Marital Status") - Which would return "Married"


• @DBColumn - given a database server, path, view, and column will return a value or list of values matching on the column. This formula is very useful when you want to get a list of things, like a list of choices. An example using this would be statusChoices:=@DBColumn("HRServer";"common\healthcarePlans.nsf";"Plan Vendors";1) - Which would return a list like "BCBS","Principal","Travellers"


• @GetDocField - given a document ID and fieldname will return the value of that document's field. This formula is very useful when you know exactly what document in the database you are looking for. An example of using this would be employeePhoneNumber:=@GetDocField("0987RTEW45648I9O:0W0E9R8C76FG54X"; "phoneNumber") - Which would return something like "867-5309"

Now that we're on the same page...



I recently was reviewing a single form in a database because that form takes a minute or so to load in certain circumstances where bandwidth is stressed. As I evaluated what exactly could be slowing the opening of our problem child form I found that the developer(s) had programmed in over 180 instances of the three lookups outlined above. 180 individual lookups EVERY TIME the form was opened or refreshed. This is a classic instance of a place where the program had been written to protect the system... but the user suffered. Unfortunately the fix for this treacherous design is a complete re-write of the form and the customer won't spring for it, so the users will be forced to endure. This is an example of a system user experience that suffers for the desire of a developer to take their own existence into hand and total disregard for those the system is designed to serve.



A second example of the user experience that has not been protected exists in a support system where requests are entered by employees needing anything from office Supplies to application support. The tickets are fairly simple, but it is important to do everything in perfect order. We are trained to:

1. Select the person the ticket is for (defaults to person creating the ticket)


2. Select the department the issue should go to


3. Select the Problem type


4. Enter a description of what you need or are experiencing.


5. Click the submit button.

In reality there are more steps than that.... but I'm trying to stay simple. One day I was filling out a ticket for a system user who was on a client site and having trouble. I created the trouble ticket, selected the department, selected the trouble type, entered a detailed description of the person's system problem and got ready to submit it.... but wait... I hadn't changed the "Entered For" value to that caller's name. Once I had clicked the caller's name and committed the new value I was greeted by a nice, clean request form... all fields had been cleared. I was not protected.



I could go on to UI design stuff, Leveraging the right technology for the job, and making a personal roadmap... but I'm tired and this ended up being a fairly long post. So I'm going to wind down. But about now you could be wondering where this post came from. It is true that each action results in an equal or greater reaction, this post is a result of a couple things that came up recently. One was the general office discussion around RAD... but the real motivator is a mauled Dijkstra quote from Ferdy Cristant - Bashing Domino

I'm ashamed to have to say it, but he might be right. Our development environment is archaic, the client is 10 years behind any given web browser in keyboard response, and the developer base is getting weary of lackluster version upgrades. I love Lotus Notes and Domino... but little things like client JavaScript deficiency, no line numbers in the developer IDE, custom classes not showing up in the IDE browser, and a debugger that causes more tears than it helps to avoid... what do I have to look forward to?



Unless a Domino/Notes developer has intentionally forced themselves into a frequent regimen of new technology research and modern programming/project approaches then they are probably mutilated and destined for the slow and painful death of a career that ten years ago was glorious. The easy route for a Notes/Domino developer seems to be .Net, yet another RAD environment promising career success and popularity.



Thanks to Willard, Phil, Mark, Chad, Wayne, and Randy for lots of engaging conversation on the topic. And thanks to all of the rockin code and process apologists I've been googling and blogging for help, opinions and answers.



Here are some links Reading Material that have formed my general attitude and outlook in the past year or so:

Coding Horror - Attwood : Attwood's blog is good, but his responders are the real reason I go here.

Head First Java - Sierra/Bates : of all the java books I've gone through this one was the most readable.

Peopleware - DeMarco/Lister : you'll cry when you see how little gain most companies have made in 30 years of software development

Why I'm Not a [Insert Language Here] Programmer - Josh Carter : toolsets as opposed to tools

Most Software Stinks! - Chuck Connell : elegant/beautiful software does not have to be complicated. I don't agree with everything he outlines, but great points and a great title.

GoTo Statement Considered Harmful - Edsger W. Dijkstra - *gasp* spaghetti code is bad?

Unjust Deserts - Poppendieck : Just found this one on Attwood's blog... very interesting and quite inline with the Peopleware book above.



And for some comic relief... join Twitter ... it's kinda funny.. mini blogging on Rails. I'm giving it a run (@rwheadon)