Guild Blogs

The Consultants Guild is proud to announce that cofounder Wayne Allen is now a Certified Scrum Master.

Just got word today that I am now a Certified Scrum Master!

The experience of a recent Consultants Guild blog reader drives home the key factors that cause projects and engineering organizations to fail. It also is a good reminder of what is necessary in order to be an Excellent Consultant. So indulge me for a moment and let me tell a little story of Excellent Consultants...or Why Technology Kingdoms Rise and Fall.

His experience was being confronted with antagonistic interviewers that didn’t understand a technically superior solution. I had a very similar experience trying to do a Sybase port of the dynamic inline T-SQL aggregation construct that I've used numerous times in SQL Server. I'm first and foremost a SQL Server implementation expert, but very proficient in other database products. We were forced to implement a SQL Server database on Sybase. I used this construct extensively to avoid laborious and expensive CURSORS. Then the unthinkable happened (well not really for Sybase), they released a patch which broke the dynamic inline T-SQL aggregation construct in Sybase. Once we finally tracked down why critical portions of our system were broken, Sybase’s response was first disbelief that this approach even worked. When I proved that it did they responded…“Well, that never should have worked in the first place, so we aren’t going to support it going forward.”

Hmmm. That doesn’t seem too intelligent to me. Am I having a nightmare here? Let me do a reality check. So I asked myself a question…”Mark, I asked…Who is paying Sybase here?”. Yep, the answer is me and the customer that I’m implementing Sybase for.

That lack of disrespect for the basic customer-vendor food chain required about 2 weeks of redesign of some key backend stored procedures and processes. Needless to say, it didn’t serve to improve my perspective of Sybase. The customer spent between $100k-$200k in the original port from Sybase to SQL Server. What value did it add….ZERO. It cost more, took longer and performed poorer. Fast forward...now the customer decided that their strategic direction is to migrate all Sybase applications to Microsoft SQL Server. So, it will cost Sybase millions of dollars from this customer and any customer that I do business with in the future that is willing to listen to my experience.

Nope, it wasn’t a nightmare after all. At least not mine (maybe Sybase’s though). Whew!

Haacked’s experience of interviewers that think they know it all, like my experience, taught me an important lesson. As technicians, we need to continually remind ourselves that fear, pride and ignorance are the greatest barriers to success. There is almost always a more elegant, more simplistic, more cost effective and higher performing way to solve a problem. Furthermore, I have found that fear, pride and ignorance are the hallmarks of project failure. It can have enormous cost implications to us as consultants, to the tool vendors we utilize and ultimately the customers we serve.

For the interviewers that thought they know it all, it cost them a potentially great resource and asset. In my scenario it cost my customer hundreds of thousands of dollars. Ultimately it will cost Sybase millions of dollars.

The lesson I learned is this, technology kingdoms rise and fall primarily due to fear, pride and ignorance. Excellent consultants, at least the ones I am willing to work with, have no room for any of these traits. If you are on a project where fear, pride and ignorance are given free reign, raise the red flag high. Waive the flag with vigor. Get executive sponsorship and root it out. These people are the type that are continually complaining, blaming and failing. An end user cracked me up with describing these type of people like this..."Their cubicles are vortexes where all happy thoughts go to die". Sadly their teams and their organizations are where good projects go to die as well.

What excellent consultants possess, at least the ones that I’m willing to work with, are increasing measures of boldness, confidence and wisdom. Boldness replaces fear, confidence replaces pride and wisdom replaces ignorance. We in the Consultants Guild are passionate about Experience, Integrity and Delivery. They are the mark of Excellent Consultants.

Excellent Consultants are…

Excellent consultants are bold enough to look for new ways to deliver successful solutions better. Whether it’s a technical challenge, a business challenge or a process challenge they always strive for success. They don’t shrink away in fear or let pride or ignorance cause them to fail. Typically they will lean on their wealth of experience and/or the deep pool of experience from other excellent consultants that they’ve been in the trenches with. Boldness is the key to successful delivery when the barriers seem insurmountable.

Excellent consultants are confident in their experience that has led them to success in the past. The proof is in the pudding. A key principle of Agile software development is that “Working software is the primary measure of progress.” Given a reasonable amount of customer trust and freedom, I’m fully confident that I will make them successful. This shouldn’t be confused with pride or arrogance because I’ll readily admit when I need the help of others. Confidence is only developed through experiencing repeated success.

Excellent consultants are wise enough to know what they don’t know. A common misconception about wisdom is that it means knowing everything. It does have to do with knowledge, but its much more than that. Wisdom is when knowledge is applied through the filter of “experience”. I also heard someone once say that “Wisdom is knowing what you don’t know”. Just becaues a ASP.NET application was a perfect solution for one customer, doesn't mean that its the only solution for every customer. We were faced with this challenge on a current project and realized that it was in the best interest of the client to build a .NET Win Forms Smart Client application vs. a traditional .NET Web Form HTML based application. Wisdom grows as we do business with integrity by looking to the mutual best interest of our clients and ourselves.

I had a need (creating CSV BCP files) that required knowing the column names of a MS SQL table in the order they were created. Some inspiration from Mark Clerget and a little fooling around with SQL Query Analyzer resulted in the following.

DECLARE @c varchar(4000), @t varchar(128)
SET @c = ''

SET @t='authors'


SELECT @c = @c + c.name + ', '
    FROM syscolumns c INNER JOIN sysobjects o ON o.id = c.id

    WHERE o.name = @t
    ORDER BY colid

SELECT Substring(@c, 1, Datalength(@c) - 2)

Which gives the following result:

au_id, au_lname, au_fname, phone, address, city, state, zip, contract

Perfect. Note the use of @c = @c + c.name in the select clause to colapse 9 rows into 1 row. I've used this technique many times in the past to generate a single string from multiple rows without resorting to cursors.

Bill Caputo has a wonderfully clear explanation of how to address questions regarding using TDD at system boundaries.

TDD is fundamentally about writing tests (or 'checked examples' as Brian Marick calls them) before you write the code. One of the benefits of this is that the resultant code is decoupled from the rest of the system. This becomes particularly important when the code is near a boundary (e.g. acesses a database, a queue, another system, or even an ordinary class if that class is "outside" the area your trying to work with or are responsible for).

However this benefit doesn't just happen, you have to want to find a way to structure the code so that it can be tested without resorting to resources beyond the test.

[via Bill Caputo]

Thanks to Patrick Logan I now know what Skiplists are.

Skip lists are an ingenious data structure that turns a linked list into a data structure that offers the same running time as the more complex self-balancing tree data structures... In the latter part of the article, we'll be building a skip list class in C#, which can be downloaded.

[via Patrick Logan via MSDN]

Elisabeth Hendrickson has a great post about the practicalities of agile testing where she takes Brian Marick's breakdown of testing vectors and talks about what to actually do differently to achieve agility in testing.

Agile project teams generally reject the notion that they need an independent group to assess their work products or enforce their process. They value the information that testing provides and they value testing activities highly. Indeed, Extreme Programming (XP) teams value testing so much, they practice Test-Driven Development (TDD), writing and executing test code before writing the production code to pass the tests.  However, even though agile teams value testing, they don't always value testers. And they're particularly allergic to the auditing or policing aspects of heavyweight, formal QA.

So how can testers make themselves useful on a team that does not see much use in traditional, formal QA methodologies? Here's what I've been doing.

[via Elisabeth Hendrickson]

1. What Does It Mean To Run A Project Using An Agile Process?

The term agile refers to a set of patterns and practices that builds a highly collaborative partnership between the business and IT, promotes team communication and delivers functional software releases in iterative cycles. By delivering business value early and often, the process allows for a natural feedback loop to adjust and improve the software.

The difference between an agile process and other processes is that an agile process is crafted to integrate seemlessly into its environment. Rather than dictating rigid process practices that may not apply, the team selects the right process fit for the environment and makes adjustments every iteration, tuning it, to result in an increased team output and quality.

Often agile teams are viewed with skepticism as the methodology is different from traditional approaches. Sometimes there is a misconception that there is a lack of process or that it promotes "cowboy coding". However, a close-up look at an agile team in action reveals a highly structured, iterative development life-cycle flow that is enthusiastically followed by people passionate about delivering value to the business and IT.

2. Basic Terminology

The following are a few terms that describe aspects of the agile patterns and practices:

• User Stories and Technical Stories – User stories are high level use cases or functionality that describes a deliverable unit of business value. Business analysts provide most of the user stories and the development team provides most of the technical stories.
• Tasks – Tasks are provided by the development team and are a breakdown of the user story into smaller development chunks. Tasks have an owner and an estimate time assigned. The owner is responsible for tracking actual time against the task to be used later to improve team estimation.
• Team Velocity – The number of delivered user stories for an iteration measured in estimate time. The team velocity is comprised of the sum of individual velocity.
• Unit test – Unit tests are automated regression tests built into the code base that test key business logic.

3. Agile Patterns and Practices To Choose From When Crafting Your Process

Upon starting our project, our team crafted an agile process that enabled us to meet the business and IT needs in the time schedule provided. To do this, we integrated many of the agile patterns and practices described below:

A. Establish Iterations and Release Cycles

Iterations are fixed development cycles at the end of which a "deliverable" product is available. The idea is to manage scope so that you don't have half-working code at the end of an iteration. It is important to note that an iteration does not have to result in a production release. A release cycle is managed by the business analysts and technical lead and varies depending on business and IT needs. For our project, the team decided to work on a 3 week iteration cycle, that leaves approximately the last week for the test cycle.

B. Employ a Planning Game To Identify Stories and Tasks

A planning game is the primary tool for the business to drive the process. This meeting kicks off each new iteration and provides an excellent forum for the business, IT and the development team to identify and prioritize user stories and technical stories. Developers provide task and story level estimated time, which allows the business analysts to prioritize and put together a release cycle. At the end of the planning game the development team have their assigned stories and tasks which will be posted to the planning board. These priorities can be adjusted by the business and IT as needs change during the iteration. Typically this is done during the daily standup meeting.

C. Utilize a Daily Standup Meeting to Reinforce Communication

A daily status meeting that lasts about 15 minutes. People are required to stand for the duration, which naturally enforces brevity. Developers provide feedback on tasks they are working on and identify any roadblocks. Business analysts adjust priorities, monitor progress and provide clarification on any business requirement questions. Interested parties or other stakeholders are invited to attend the daily standup meeting and monitor activity. The daily standup meeting for our project occurs at 8:30am and is open for people to monitor.

D. Use a Planning Board to Communicate Task Activity and Ownership

The planning board is a physical poster board that has the stories and tasks affixed as 3x5 cards. When a task is in progress the developer circles it in blue. Upon completion they circle it in green. It is used in the daily standup meeting to visually communicate activity.

E. Identifying and Procuring the Development Tools to Establish a Refactoring Space

It is critical that the right development tools are in place to enable an agile environment. These tools are identified and the environment is setup early on in the project. A refactor space is one in which the development team has all the tools in place to easily modify the entire code base for the project. For example, if an error was identified in a particular approach, the team could fix it across the entire refactoring space. Key tools in the our refactoring space are: Visual Studio.net 2003, Visual Source Safe 6.0d, nunit, nant, funduc (search and replace) and Araxis Merge. These tools enable the team to manage the implementation process efficiently.

F. Establishing an Automated Build, Test, Source Control, Stage and Deploy Process

Automated builds refer to the automation of the many tedious steps involved in building the application components, managing source control, running unit tests and packaging and deploying the application components to the development, test and production environments. This traditionally can be an entire job description all by itself. By automating this process we eliminate a lot of manual work and potential errors due to human error. Our automated build additionally applies a source control label and creates a new versioned directory to a network location containing all files that will be applied for whatever environment is being deployed to (development, test or production). This staging directory provides for a natural rollback code base in addition to an easy location for future reference or trouble shooting!

G. Use Paired Programming Where It Makes Sense on Difficult Tasks

Paired programming simply refers to working collaboratively with another developer to solve a complex problem. Pairing promotes knowledge dissemination in addition to increasing velocity. The more pairing that occurs in a team, the more aware and cross-trained a team becomes, which will increase the team velocity. If a team member gets sick, others can pick up where they left off. That said, there are many tasks that do not require pairing and are best completed individually and in parallel.

H. Promote Collaboration and Shared Code Ownership

It is important in order for a team to collaborate well together to work as a team. Shared code ownership means that we take responsibility for the code as a team and together share its successes and failures. Pairing on complex tasks will help promote this important component of an agile team. You know you're doing it well when everyone on the team feels a personal responsibility to make the best solution and work together to achieve and support it.

I. Use Automated Regression Testing (Unit Tests) Where It Makes Sense

Unit tests are built into the code base to test key business logic and ensure that future enhancements do not introduce errors or break functionality. It is somewhat of an art form to determine where to apply them most effectively. Unit testing for our project is automatically run when doing an automated build to the development environment.

J. Track Team Velocity by Comparing Estimates Vs Actual Task Time

In order to cost the user stories to determine the amount of features that can be developed in an iteration, the business analysts use estimated time provided by the development team. During the iteration the development team tracks the actual time it takes to complete the work. At the end of the iteration the team velocity is how many stories were fully completed, measured with their estimated time. Team velocity tells the business analysts approximately what they can promise to deliver. Comparing the estimate time vs. actual time helps the development team improve their estimates.

K. Follow a Strict Development, Test, Production Environment Deployment Flow

Professionals never work without a safety net. Maintaining clear lines of demarcation between the development, test, and production environments is critical to help ensure the quality of the code base and keep a stable production environment. Our team reinforces these boundaries by using detailed release guides in addition to automated builds that do most of the tedious work.

L. Use Technical Spikes to Gather Information for Unknowns

No technical team knows everything. A technical spike is a very focused research effort on an issue or technology that the team does not have prior knowledge on. It is undertaken to answer a specific business or IT question and usually has a 2 day maximum time limit.

M. Pay Down Your Projects Technical Debt – Increase Your Velocity!

Technical debt is accumulated naturally as projects get more complex. As issues arise the team can’t solve everything and must resort to workarounds for the lower priority issues. As time continues these workarounds take up measurable time in the iteration, decreasing the team velocity. During each iteration find time to pay down some of this technical debt by fixing what you have time for. This will keep your team producing more value and spending less time on other issues.

N. Design Solutions Using Object Oriented, N-Tier, Service-based Architecture Practices

Object Oriented Programming and Ntier development is alive and well. With the advent of .Net many patterns that were previously reserved for C++ are now available for mass consumption with VB.Net and C#. Good architecture is the foundation for delivering a quality, extensible solution. It allows for unit testing and a clear separation of layers that allows the code to evolve and meet the ever growing needs of the business and IT. Separation of key functions into services allows for a powerful distributed architecture that can be leveraged and reused.

O. Refactor Mercilessly to Keep Your Code Base Cruft-Free

Refactoring is improving your code base with new learnings or better implementation practices, such as moving duplicate code to a common base class. Refactoring should occur often, but only when it makes sense, such as when you need to make a change in the area anyway. Without refactoring your code base continues to gather cruft such as duplicate code, non-performant routines and many other issues that result from building new features within a time constraint. This cruft will choke the maintainability, performance, and quality of your application over time in addition to increasing you bug count.

P. Continue to Improve Your Process – Try Something New Every Iteration!

A key aspect of an agile team is to stay agile! Empower your team to continuously look for ways to do what you do more efficiently with better tools or techniques. Encourage them to try something new, as a team, every iteration and roll the things that work well into your process. For example, our team posts digital pictures of whiteboard drawings to Sharepoint and uses Camtasia movies as test scripts for the QA testers. Both practices save measurable time and were the result of trying something new. Another idea would be to track a new team metric for an iteration to see if there is an opportunity to improve team velocity. For example, track the amount of time your team spends building, source controlling, packaging and deploying your solution every release.

4. Our Teams Development Process Overview

This section outlines an overview of the process followed by our team for each iteration.
Iteration Cycle: every three weeks
Release Cycle: determined by business analysts and technical lead

Planning
Planning Game Meeting: [business analysts, development team, stakeholders, interested parties]

• Business analysts and development team recap previous iteration successes and issues
• Business analysts identify user stories
• Development team identifies tasks
• Development team provides task estimates and those that will require a technical spike
• Business analysts prioritize user stories based on estimates
• Development team signs up for tasks using previous iteration velocity as a guide
• After the meeting the Planning Board is updated to reflect the new iterations story/task owner-ship and activity

Release Planning: [business analysts, technical lead]

• Determine release date and committed user stories based off estimations and spike feedback
• Determine if the release will span more that one iteration

Submit Testing Request: [technical lead]

• Technical lead provides an initiating document that states requirements or what new functionality is being addressed in the iteration
• Technical lead provides the test plan
• Technical lead provides the requested testing release date

Submit Production Request: [technical lead]

• Technical lead provides an initiating document that states requirements or what new functionality is being addressed in the iteration
• Technical lead identifies rollout team
• Technical lead submits the production rollback plan
• Technical lead provides the requested production release date

Implementation

• Daily Standup Meeting: [business analysts, development team, stakeholders, interested parties]

• Developers provide task feedback and identify roadblocks
• Planning Board is updated to reflect activity
• Business analysts provide updated priorities, requirements and business feedback
• Stakeholders and interested parties voice concerns or get information regarding activity

• Developers collaborate on tasks or work independently depending on complexity. Collaboration is encouraged to promote communication and disseminate knowledge.
• Developers look for opportunities to refactor, apply automated unit tests and reduce project technical debt
• Automated processes are tuned to keep running smoothly and integrating new manual processes
• Development team stages build to the development environment often as features are built

Testing Release (Requires IT Approval)

• Development team provides testing scripts or (Camtasia) movies
• Development team stages build to the test environment and submits notification to stakeholders
• Business analysts and Quality Assurance lead coordinate testing effort
• Business analyst maintains, prioritizes and coordinates issues log
• Developers resolve issues and stage subsequent test builds to the test environment
• Technical lead compiles and submits test results to attach to the Production Request
• Business analysts certify the build as ready for production

Production Release (Requires IT Approval)

Note: Not every iteration may result in a production release. Releases can span multiple iterations. Project production releases occur during a published downtime window.

• Technical lead sends upcoming release notification to stakeholders
• Rollout team analyses and reviews the Project Release Guides
• Review DTS Package Checklist
• Review Project Production Ready Reports Checklist
• Review Project Application Updater Release Checklist
• Review Project Rollout Checklist
• Review Production Rollback plan
• Rollout team stages build to the production environment
• Rollout team tests against new release
• Business analyst certifies release
• Rollout team rolls back if release is not certified
• Technical lead sends successful release notification to stakeholders

5. Summary

The development process used by our team was crafted by selecting a mix of agile patterns and practices that make sense for the environment. Although some of the terminology may seem new, many of these practices have been performed by highly productive, successful development teams in the software industry for years. What the process does is formalize the best of breed practices so that the team can follow a simple structured repetitive pattern, allowing them to focus on what really counts, delivering quality solutions for the business and IT.

A popular requirement to display data in a custom .NET Winform application is to structure the data in a hierarchical tree-view format. However, the built-in support for this type of display is limited. Using the ComponentOne FlexGrid control, one can build a set of data tables into a Dataset object, and then relate them together. However, getting this setup, and moving the data into the Flexgrid requires a considerable effort.

Another way to get this result is to use a FlexGroup control from ComponentOne. It implements Outlook-style grouping and filtering using the FlexGrid. However, the control is not included in any of ComponentOne's developer suites. Instead, they provide the source code to the control, and it's up to you to build an instance that can then be used in your project.

Steps
1. Download the FlexGroup sample code under the FlexGrid control at the ComponentOne website: http://www.componentone.com/pages.aspx?pagesid=113

2. Once the download is complete, exract the compressed files and run the FlexGroup.vbproj VB.NET project.

3. View the properties window for the FlexGroupVB project file and change the Output type from a Windows Application to a Class Library.

4. Re-build the solution. When complete you can close this solution.

5. Open your solution. From the Toolbox, click the Customize Toolbox option. In the .NET controls section, browse to the Bin directory of the FlexGroup folder you downloaded. There, you will find the compiled control named FlexGroup.dll. Add this to your Toolbox. Back at your Form, the FlexGroup control is now a selectable item in the Toolbox. Select it and drag or paint it onto a form.

6. Set the control's Grid.Datasource property equal to a Dataset table. Run the project and display the form. You will see a display that looks like the example shown below.

Additional Note
The Dataset you use should be a flat result set with parent/child relationships built in.
For example:
Author1 BookA
Author1 BookB
Author1 BookC
Author2 BookX
Author2 BookY
Author3 BookZ

There is also a property named .ShowGroups. If this is displayed you can drag and drop columns into a Grouping section located just above the grid, allowing the user to customize the groupings displayed. In my example below I have chosen to hide this feature so I can control the look and feel of the user interface.

One "gotcha" I found was dealing with sibling columns. For example, City, State, and Zip are attributes of an address and thus siblings to each other. If you allow each of these columns in the dataset, the display of the data appears less fluid and appealing. I got around this by including sibling columns together as one column (e.g. City + state + zip as Address).

Cliff Atkinson has recently produced several microcasts (using Camtasia)demonstrating several key aspects of his new book "Beyond Bullet Points". I've enjoyed Cliff's eye opening series on how to make stunning PowerPoint presentations.

• Hidden Storyboard Guides - Using hidden slides to structure your presentation.
• Illustrate 1 Blog Post in 2 Minutes - Using PowerPoint graphics in your blog.
• How to Structure a Story: Act I - Describes the 3-act story template from his book.