I Do not have Time to Write Tests!

On more than one occasion during conversations around automating tests, I have heard a programmer say "I don't have time to write tests!"

Why?

1) Any programmer that submits code without compiling the source file, please leave a comment.

2) Okay, of the programmers that at least compile their source, do they build the entire project locally? If not, leave a comment.

It is ok. You aren't part of the team until you break the build for everyone...

3) If you are building for more than one platform, do you compile and link on all platforms before submitting a change? If not, leave a comment.

Screw those guys doing the PS3 port. They are jerks!

4) Of the programmers that compile, link and build for all platforms, how many run their changes under a debugger to prove (at least to themselves) it works? (Leave a comment)

5) Of the programmers that compile, link, build for all platforms and test locally, have found that their features have been wrecked by changes from other developers (artists, designers, programmers)? Comments should be full of this.

To illustrate the point, every project I have worked on has seen violations of all 5 scenarios. I will confess that I am also guilty of some or all of them. It happens and we learn :)

How much time, as the complexity of the project increases each iteration, is spent dealing with feature regression, technical debt and system fragility?

When a live service, earning millions of dollars each iteration, goes tits over an untested feature, leads to a panicked rush to fix it now now now!!! and then leads to a cascade of fail, is it time to consider, as my angry sage and friend Jonathan Tanner said: "slowing down to move faster?"

Most programmers that have been in the business for a while suffer under a completely different regime: they know what to do, but pointy-haired bosses make doing the right thing impossible! 

This majority of engineers should take this little bit of wisdom to heart: producers do not produce code. Business managers have no business to sell without your work. They regard you as freaking wizards. Be honest with them about the costs of doing a bad job.

Do you have time to write tests? Can you make time to help your fellow developers ensure they do not accidentally break what you have created?

For a taste of a solid test framework, check this link: http://logicle-cplusplus.blogspot.sg/2014/03/writing-great-user-stories.html

Writing Great User Stories

A few years ago, I was introduced to Behavior-Driven Development (BDD), RSpec and shortly afterward, Cucumber. Having advocated for Test-Driven Development (TDD), I was curious and cautiously optimistic.

Working with Cucumber (http://cukes.info) has been great. It is feature specification, documentation and testing wrapped up in a single workflow. It starts with a feature file written in plain language that drives the rest of the test framework. It is a close cousin to TDD, in that the tests are written first, then the implementation follows until tests pass. The difference between BDD and TDD: BDD tools like cucumber use behavioral/business specifications as the starting point, not an engineering authored suite of tests.

Feature files usually start with what reads like a typical scrum user story, followed by several scenarios that clearly define the feature's behavior. The scenarios can be used as acceptance criteria in addition to documentation and driving tests. Gherkin (the DSL that drives Cucumber), like any language or technology, has established current best practices. Gherkin doesn't really care how a feature is described, but scrum-like user stories are typical.

One of the guidelines for writing great feature descriptions is to apply an adapted version of the Six Sigma 5-Whys method.

There is a great example of this approach on the cucumber wiki.

[5:08pm] Luis_Byclosure: I'm having problems applying the "5 Why" rule, to the feature 
                         "login" (imagine an application like youtube)
[5:08pm] Luis_Byclosure: how do you explain the business value of the feature "login"?
[5:09pm] Luis_Byclosure: In order to be recognized among other people, I want to login 
                         in the application (?)
[5:09pm] Luis_Byclosure: why do I want to be recognized among other people?
[5:11pm] aslakhellesoy:  Why do people have to log in?
[5:12pm] Luis_Byclosure: I dunno... why? 
[5:12pm] aslakhellesoy:  I'm asking you
[5:13pm] aslakhellesoy:  Why have you decided login is needed?
[5:13pm] Luis_Byclosure: identify users
[5:14pm] aslakhellesoy:  Why do you have to identify users?
[5:14pm] Luis_Byclosure: maybe because people like to know who is 
                         publishing what
[5:15pm] aslakhellesoy:  Why would anyone want to know who's publishing what?
[5:17pm] Luis_Byclosure: because if people feel that that content belongs 
                         to someone, then the content is trustworthy
[5:17pm] aslakhellesoy:  Why does content have to appear trustworthy?
[5:20pm] Luis_Byclosure: Trustworthy makes people interested in the content and 
                         consequently in the website
[5:20pm] Luis_Byclosure: Why do I want to get people interested in the website?
[5:20pm] aslakhellesoy:  :-)
[5:21pm] aslakhellesoy:  Are you selling something there? Or is it just for fun?
[5:21pm] Luis_Byclosure: Because more traffic means more money in ads
[5:21pm] aslakhellesoy:  There you go!
[5:22pm] Luis_Byclosure: Why do I want to get more money in ads? Because I want to increase 
                         de revenues.
[5:22pm] Luis_Byclosure: And this is the end, right?
[5:23pm] aslakhellesoy:  In order to drive more people to the website and earn more admoney, 
                         authors should have to login, 
                         so that the content can be displayed with the author and appear 
                         more trustworthy.
[5:23pm] aslakhellesoy:  Does that make any sense?
[5:25pm] Luis_Byclosure: Yes, I think so
[5:26pm] aslakhellesoy:  It's easier when you have someone clueless (like me) to ask the 
                         stupid why questions
[5:26pm] aslakhellesoy:  Now I know why you want login 
[5:26pm] Luis_Byclosure: but it is difficult to find the reason for everything
[5:26pm] aslakhellesoy:  And if I was the customer I am in better shape to prioritise this 
                         feature among others
[5:29pm] Luis_Byclosure: true!

This is equally valuable for user stories in a typical product backlog. A well groomed backlog should have stories that communicate the value of each item for easy prioritization.

On some scrums, as the product owner, I have been pairing user stories with scenarios as acceptance criteria. Each story is groomed a bit with the team to include scenarios so the criteria for the features are clearly defined, testable, can be automated and communicated with business managers, production teams, artists, designers, testers and other engineers. The output of product backlog grooming (for upcoming items), are cucumber feature files.

As stated previously in this blog, the "business value" for game developers is making fun. It is all too easy to forget that when talking about monetization, operating expenses, development budgets, etc... Yes, we are in business to make money, but we do so by offering a value proposition that is unique -- we sell fun and delight to customers. Everything else we do is pointless if we have nothing of value to offer.

Applying 5-Whys to user stories should focus on whether the feature satisfies Minimum Marketable Features (MMF). One addition to the criteria of problems we need to address with a feature is "will this make the game more fun?"