Testing Procedures 2009/01/25

Main Points

• New Beta Content Page Types Here are the newly formalized page types on our website.
• Step by Step Administration Procedures OK, so we've got three page types, now what do you do with them.

Our informal testing and website maintenance procedures finally bit the dust, basically because it was logical to write down guidelines and we achieved other priorities allowing us to write down a more structured approach. With our time no longer devoted to building the master page template, codifying and streamlining website administration was one of the most critical follow up activities. The site's grown big enough to require something more robust, and we've learned superior testing methodologies by study - and trial and error. These procedures allow us to initiate testing, develop improvements, document the changes, and implement them live. That's right, Earth Chronicle is now under iterative development. Undoubtedly there will be changes to the testing procedures, and that's in and of itself a victory. We finally have a formal process written down, right here, that we can evaluate and improve. So cheers to making ourselves obsolete! ;)

New Beta Content Page Types

Our new testing procedures have defined three classes of pages based on what resources the content page uses. Live / Core pages update as we make changes to the live resources. Frozen testing pages never change and draw from dated resources so you can see exactly when the last change in the resource was completed. Dated files have the date of their creation appended to the end of the file (ie BetaMainMaster20090125.master was completed 2009/01/25; ie 2009, January 25th) Normally a frozen file will have many different dates as the nested master pages update very frequently, whereas the recommended pages may stay the same for years. However, this is the launch state of testing when we first built this testing system for Earth Chronicle. ;)

Active testing pages will mostly depend on the same dated pages, however resources undergoing testing must switch to the Testing version of the resource. (e.g. BetaMainMasterTesting.master) When testing is finished the Testing file is saved to a new dated file and checked out. When the page completes the final checklist, the most recent dated files can be saved over the live files, thus updating the changes to the live site. (The alternative is saving the Testing files over the live files, and if multiple people are testing that would be pretty suicidal.)

Step by Step Administration Procedures

These procedures for the first time allow multiple people to pursue multiple projects at the same time. That's one advantage of this administration system. No one should find their changes wiped out by someone else. Everyone's work will be frozen at the end of testing, before the final checklist is performed. Otherwise, you could run through the checklist and someone else's work could change things. I think the "worst" that can happen is that someone would clean up after their work and reference a frozen dated file in a testing master page. In fact, to simplify this, for now at least, it looks like we're going to perform cycles of testing, that will halt development momentarily while we push changes across the site in "waves". The pages are frozen and then testing can restart. The only downside is that if errors are found in the testing checklist, the page will have to "fail" testing, continue to develop on the new template and go around again when whatever bugs are cleared up.

After working on this for awhile and studying more about project development methodologies, Earth Chronicle makes sense to build using iterative development. It's not a website that will ever be "finished", so changes need to be made to the live site. UI changes should not be made frequently to avoid confusing people; you'll note that even with the magnitude of the changes that have just happened, most of the Beta page layout has not changed. It's been optimized, I've played with the colors to better communicate visually with. The behavior of components has been altered for optimum cross-browser, cross-platform, and cross-display compatibility, but the UI components don't look significantly different.

[chroniclemaster1, 2009/09/20]

The first thing to note is that once a page is dated (e.g. BetaMainMaster20090125.master) it's Done. It shouldn't be modified ever again except to keep workingLinks in line so you can navigate in and out of the page. In this arrangement, it's a little deceptive, but the files which compose the live pages are also effectively dead. The workflow, the lifeblood of development for Earth Chronicle now flows through the Testing files, (e.g. BetaMainMasterTesting.master). The Testing pages are the only ones we make modifications to. Dated pages are frozen; any changes to them would be lost to the live files and it would alter the frozen state of the page. Live files have a limited life expectancy too, as they are lost when the new Testing is saved over them. So here is the "circulation" of the Testing files.

1. Begin Testing

   1. Start a new page (ie copy the testing page template and the holding page template to the appropriate folder in the website where the content page will live)
   2. Rename the holding page file to the filename you want.
   3. Open the holding page and replace "ThisPage" in the @Master Title attribute and the <h1> tag with the page title.
   4. Open the index which will point to the page and add a link to the new page. (plus any accompanying text and explanation)
   5. Make sure the link works in a browser and displays the holding page.
   6. Give the testing page template the same name as the holding page, but with a ~ in front.
   7. Make sure you can manually insert a ~ into the URL in front of the holding page name to access the testing file.

2. Conduct Testing

   1. Don't modify dated files!
   2. Copy the most recent dated file to a Testing file, and change references from dated files to the Testing files. You can then modify the Testing files instead. Testing files are the only ones that should ever be modified.

3. When testing is complete...

   Record

   1. Copy ContentDemoTesting.aspx to a dated file, delete unfinished items. (Delete finished items from ContentDemoTesting.aspx)
   2. Add the new file to OngoingTesting.aspx and update references. Compare the new dated file with the previous version and make sure the ContentDemo page accurately explains all changes.

   Freeze

   3. There are three calling files, files which may call other testing files: the nested master certainly will, and the primary stylesheet & main master page might. Rename any of these testing files to a dated file, e.g. BetaMainMasterTesting.master to BetaMainMaster20090406.master.
   4. Rename all other Testing files referenced in the calling files to new dated files. This freezes development. Be sure to update each file's date in the <head> of the dated nested master, e.g. BetaMainMaster20090406.master.
   5. Change all references in the three calling files to the new dated files. There should be No references to Testing files in the calling files when you're complete, and no Testing files should even exist at this point. (The downside is that testing isn't "really" ongoing, it has a definite end.)
   6. Copy the dated nested master back to a Testing file, e.g. BetaMainMaster20090406.master to BetaMainMasterTesting.master. This ensures that the Testing template doesn't collapse for want of the file, although all the references will now refer to dated files.

   Check

   7. Run the page through the checklist.
   8. If necessary, correct any errors in new dated files and resave them. Rerun the checklist. Repeat until finished.

4. Implement New Changes Live

   1. Compare the Resource Dates in the <head> of the current BetaMainMaster.master to the resources in the new dated master, e.g. BetaMainMaster20090406.master.
   2. All resources that are newer than the versions used by BetaMainMaster.master should be saved over the "live" resources thus taking the changes live. EXCEPT ecBetaPrimary.css! This should only be saved over if files are moved which should be very infrequent.
   3. Open BetaMainMaster and reset the resources from folder/2009/file20090504.blah to folder/file.blah. Otherwise they will still be referencing the frozen test files. (You actually can't tell a difference and it works. We'll have to consider this, but for now, I want to keep all resources in the locations specified on the table above.)

And you're done; that's the flow of a testing cycle. And implementation doesn't get much easier than that. The only wrinkle is for pages that have .cs files. In that case, when you save the pages, you'll need to update the partial class name in the .cs and the parent file to match the new filename. Just one reason why we are very glad that most of our pages reference NoCode files. (since the partial class names will remain "NoCode") Phew!