6. Deployment

Main Points

• Rewriting the Application Because the first time was just for practice.
• Communicating the Project Effectively It's one thing to build a better mousetrap, then you have to market it.
• Deployment Architecture Architectural considerations that ensure project success.

Once you've completed project construction you are ready for project deployment. What all is involved with that? Doesn't someone just push a button to push the website to the server? Construction is finished, so everything is ready to go now, isn't it? In fact, I've already got a book that covers the deployment options for Visual Studio so that's all I need, right? Certainly I can skip this? Ahhhh... if only it were that easy.

Here's one of the dirty little secrets of project management. If you think you're done, you have put your clients in a position where they will have problems with broken, overbuilt software and 1) they will complain to you and demand additional work or 2) they will complain among themselves, their friends, their wives, their vendors until you have a very bad reputation indeed. The only way to really write a program... is to write it twice.

Rewriting the Application

Steve McConnell describes programming as a wicked problem. "Horst Rittel and Melvin Webber defined a 'wicked' problem as one that could be clearly defined only by solving it, or by solving part of it (1973). This paradox implies, essentially, that you have to "solve" the problem once in order to clearly define it and then solve it again to create a solution that works. This process has been motherhood and apple pie in software development for decades (Peters and Tripp 1976)." (McConnell Code Complete 2, 74).

Construction is a testing phase. There are developers and firms who try to "save time" by grouping these phases together, but assembly line development in the FSDLC is the process which has consistently shown itself to be the fastest way to program. Don't try to create perfect working code for production during construction – you waste far more time and distract yourself from the much more important job that you are supposed to be paying attention to – learning how to write this program. Construction and deployment allow you to worry about one phase of coding at a time. It is this specialization in each step that allows you to bring assembly line efficiencies to bear on the problem and shorten development time.

Communicating the Project Effectively

It is sometimes hard to explain this process to people unfamiliar with software or other production processes. I find that the vocabulary you choose is critical. Never tell a client that you are working on their program during what we call construction. You are working on the "testing application" "test version" "prototype" or something similar. Make sure this vocabulary is used consistently not only in conversations, but on documentation as well. Once the "test version" of their software is prepared (i.e. once construction is complete) you are ready to build the "production version" that will run on the live server (i.e. you can start deployment).

This is exactly how I discuss projects with clients starting from the initial fact-finding, problem definition, and requirements phases. Clients generally do not care how a project progresses as long as their time and budget is respected, their needs are being met and they are confident that they understand what is going on. This vocabulary is far more effective and far more accurate at communicating the difference between construction and deployment.

Also, this indicates an important series of steps in user acceptance testing (UAT). While I like to perform UAT at key checkpoints, this is definitely the most important phase. UAT that happens when you're building the production version of the program is about final acceptance and handover of the application. A disagreement at this point is about a client and a developer who no longer agree about whether the contract has been fulfilled or not. Therefore, it's important to be sensitive to the client's requests, and to realize that it's the client's perception of your sensitivity or lack thereof which counts.

Taming Scope Creep

At the same time, this is where many undisciplined managers allow the final strands of control to slip their fingers; that causes scope creep to become a waterfall and so overburdens the team with last minute changes that the project cannot be completed successfully. This is a worst case scenario, because no matter what happens from this point on the project has already failed. Everyone from upper management to the lowest level programmers working on implementation is now stuck in survival mode just trying to clear UAT and survive the project. That's why scope creep is such a major problem.

It is critical to follow your spec, provide the sensible modifications within that framework to satisfy the client's needs and to let them know you're more than willing to do lots of additional projects to meet those needs for them. That's the very definition of a good working relationship. But you should not be shy about identifying which minor changes fall within the reasonable modifications you can make within the scope of the current project and can deliver by the deadline. Other changes can only be addressed in a new iteration of development (i.e. a new project). That involves additional time, money, and probably a new contract; again there is nothing wrong with this, it is in fact the very definition of a client that you have a good working relationship with. This is the goal.

Therefore, I do not really find it difficult to balance these competing interests; at least its easy enough that I'm always shocked when a manager becomes a client's doormat. You do not impress anyone allowing them to walk all over you. On the contrary, if you permit someone to treat you poorly, they presume that your work is easy and / or of low-quality; otherwise you would take some pride in it. While not at all the case, a client in this situation frequently lacks the true ability to gauge the value of software and bases their opinion of their purchase on the quality of the team and managers. Treating yourself and your team with pride and respect is the most effective way to convince your clients that your projects are valuable.

There is one other important aspect to communicating your project, especially during UAT, and that's the tools necessary to present your project appropriately. This is primarily an architectural consideration; however, it is worth taking some time now to talk about exactly the ways in which deployment needs define important aspects of your project architecture.

Deployment Architecture

While I've spoken about architecture and the many ways that different projects can demand certain configurations, I have a great deal of experience dealing with the special needs of web applications and websites. There are some special architectural needs involved with these systems especially when we look at taking them live. Some of these suggestions can be implemented in many different ways, however, I've had the good fortune to work on a large project with a C# developer who introduced me to Subversion, an excellent version control program.

This is probably the single most important recommendation I can make regarding software deployment, especially if there is more than one person working on it. Some people recommend a check-in / check-out system, but I have had bad experiences with such systems. All this does is restrict two people from working on the same file at the same time. When developers work in such a situation, it is not uncommon for developers to miss critical deadlines because someone else has a file they need. Or worse, someone will continue with development even though they can't test; a fundamental violation of incremental development.

When a development team begins using a check-in / check-out system, most of the team members will abide by good manners; they'll check files out only when they need them, and release them when they're done. However, as deadlines pass, there's a terrible pressure on developers to become more possessive about checking out files which they "might" need in order to keep their work on track. Then there are always one or two developers on the team who are forgetful to check their files back in as soon as they're finished with them. The result is that quickly, key files become battlegrounds for developers to seize whenever they can in order to safeguard their portion of the project. Check-in / check-out systems are a horrible idea because they systematically give members of the development team incentive to compete against each other for possession and control of project files.

A much better choice is a version control repository that handles change management for the system. I've worked with a single Subversion repository which served as the core of multiple development projects, and found the experience so smooth and intelligent that I wish all my projects used this architecture. In fact, unless shared hosting is a requirement of the project, I usually do favor this architecture. A repository is the location that stores all of your code and its entire change history in the same way that a database stores all your data. Usually the repository is set up on a server because communication to and from the repository is so critical. Each developer will connect to the repository and download the latest copy of the repository from time to time so that they have a local copy of the current system on their own computers. It is not at all irrelevant that if multiple servers die, everyone still has a backup copy of the project which you can use to recreate a brand new repository. Thus, catastrophic failure would have to hit not only all your servers And all your backups, but all your developers to wipe out your code base.

For all but the largest projects this architecture is feasible and very freeing. No one has to cope with file check-in or check-out; they simply work on their own files. Subversion (and I would guess many other version control software) can merge files which two developers have worked on without trouble under most circumstances. It is not perfect though. If developers make changes to the same line of code, Subversion will sit down and cry. When someone has made major changes and Subversion can't determine if developers have worked on the same line or not, it can cause the same problem. In that case, Subversion does the most brilliant thing; it doesn't try to automate its way out of failure, it asks you to fix things for it.

It throws you back three files, a copy of the original version of the file which you started modifying (the old file), the current version of the file in the repository (the new file), and the file you tried to commit (the one with your modifications). This is all the information you need to figure things out for yourself. You can compare your modifications to the old file to identify exactly what you've changed and then incorporate those changes into the current version. That is the worst case scenario with Subversion. And most of the time, of course, the version control software can merge your changes transparently without you even knowing that another developer(s) has committed changes since you started working on your file. Most importantly, no one is ever denied access to a file they need to work on for tomorrow's deadline.

This architecture is far more intelligent than a check-in / check-out system. First, Subversion does not pit your team against one another because everyone has their own local copies, and two, it incentivizes best practices. One key that developers quickly learn with this system (and which they sometimes never learn with a check-in / check-out system) is that they need to commit their changes regularly.

If a developer commits their changes regularly, there is little chance that anyone will work on the same line of code and commit changes before you. Sometimes, someone else will submit changes before you, but because the changes are on different lines, Subversion can merge the documents automatically. Only rarely will you submit changes that Subversion can't handle, and then you simply have to sort things out manually for yourself.

On the other hand a developer who does not submit their changes regularly will routinely run into this problem and reap the frustration of their poor development practices. Having a team of developers that are well-trained to keep the repository up to date with their changes is invaluable for the team, the analyst, the project manager, and QA. This means that it's easy to see the current state of the entire system and address deviations from the spec or potential problems, react to insightful ideas, etc. in a very short time frame. All these options can have a major impact on the completion of the project, and it makes a big difference if you can deal with them sooner rather than later. This is another key advantage of a good version control package like Subversion.

It also frees your developers from intrusive monitoring. Managers and supervisors can give developers the freedom to run off by themselves, because the repository puts the results of the team's work under easy scrutiny and does so more regularly than any manager would think of asking for updates. So long as you realize that the repository is the code and do not recognize the argument "I've done work, I just haven't uploaded it yet." then you literally have 24 / 7 / 365 updates in real time. It doesn't get any more immediate than that.

For developers, managers are not in their faces and the need for supervisory project reviews is reduced; developers gain more confidence and more ownership of the project and can do what they think needs to be done. Simultaneously, the repository is safeguarding and supporting supervisors so they can monitor the application's progress, review new code, modifications, and literally everything else to ensure that the project meets spec. This also means developers who are good at coding spend less time flubbing around trying to generate reports on their status, because managers can far more accurately and empirically gauge project progress for themselves.

Oh, and of course, version control software allows you to control the project version. While this should be obvious, this means that if things blow up when you push a new version, it can be rolled back quickly; a major requirement for the live production server especially. It also allows you to pursue branches of the project, split them from the primary line of development and merge desired branches back into the main project. This lets you pursue alternative architectures for the project without completely committing yourself. That's an excellent way to try something risky or to start working on the "next version" of your program while still performing traditional day to day maintenance jobs on the main line of development.

Multiple sites

With a repository set up and developers with their own local copies of the project, you then create a development site. Ideally, you'd like to have the dev site configured on its own server and you'd like it to model the production server as closely as possible. Obviously, if you have a high-volume server farm hosting the web application, that won't be an option, and price is frequently a prohibitive factor regardless of your situation. The goal is to set the dev box up as similarly as possible and don't bother crying over spilled milk; they're never identical. So what usually happens is that the repository is set up on a server that's a reasonable approximation of the production server and the dev site lives there as well. The important thing for the dev site is that it should be close enough to the production site so that you do not have to recode or rewire anything. While developers may be able to run a local server on their local copies of the project, the dev site is where the rubber first hits the road.

When the developers are happy with the development site, the repository is used to push changes to a test site. Ideally, the test site runs on the same server (or server farm) that the production site runs on. This most closely matches the live environment as you can get without actually pushing changes to the live production site. QA can do all the work they want to with the test site without having to bat an eyelash over the needs of the development team; since each group has their own server, there's no overlap and no conflict. This also makes it easy to push the application to the test site regularly; I recommend doing so at each project checkpoint. This helps you involve QA throughout the entire construction process and fix problems early on comparatively small fragments rather than having to test the entire application at once with the final project deadline already overdue.

Changes on the test site can be demoed for final approvals before anything goes live. Once necessary approvals are received, you can push the test site to the production site and they are live. The dev site and the test site typically are distinguished by a subdomain on the parent domain. If the live site is hosted at www.mysite.com, I like to host the dev site at dev.mysite.com and the test site at test.mysite.com. This not only makes it easy to compare individual pages on the different sites, it's a very manageable way for stakeholders and executives to reach the test site so that they can view new functions and issue approvals.

And there you have it. The completion of deployment represents a major fundamental transition in the life of the project. While the great white whale of "Maintenance" lurks beyond the horizon, the conclusion of deployment is typically what people refer to as project completion. Needless to say this is a major accomplishment and one worthy of being celebrated on any level.

The simple fact is that many projects never make it this far; you have a lot to be excited about whether you're a designer seeing your concept come to life, a developer enjoying the fruits of hard programming and testing, or a stakeholder who is finally getting to take advantage of those new features and efficiencies which were, before today, just someone's "nice ideas". So whoever you are, and whatever role you played in the project, this really is a time to step back and appreciate everything that you have contributed to this success.