Thoughts On Unit Testing

Adopting unit testing ...

Much has been said and written about unit testing before. Those previous comments convinced me to try to add unit tests to whatever code I was writing. As I have gone from a non-unit tester to a novice/intermediate one, I have observed a few things.

Unit testing instead of print statements.

Every programmer begins by writing a "Hello World" program. By writing such a program, you learn how to compile, run your task and verify your output. As I built more programs, I still had the old "Hello World" verification process in mind, and I often littered programs with print statements. These print statements would give me confidence that things were working as designed. After I was sure a function or class was working as expected, I would clean up these print statements that I had been using to help me develop.

It dawned on me that print statements that I used during development could be replaced by unit tests. If I wanted to check if a function was being called, I could write an "EXPECT_CALL" test, or if I wanted to validate that a variable was updated in some way, I should invoke an "EXPECT_EQ" test.

Advantages to Unit testing instead of print statements.

There are lots of advantages of using unit tests over print statements. You ask a computer to verify output rather than a human (you) from reading a terminal screen. Verifying mundane things over and over again is ideal work to shift from you to a computer. You keep unit tests in your source control even after you ship your code to production. Which means that your tests can be run again and again for all of time.

The only advantage of using print statements is that it is easy to use. This is true in the absolute bare minimum short run. But for any complex piece of code, it is not true. Are global variables easier to use than classes with encapsulated data? Yes, it is easier to directly modify and access a variable that sits in the global scope. However, it is much harder to understand and debug code that relies on global variables. Over time, the hard-to-use classes that contain private data, setters/getters, and other complexities actually become easier to use. You just have to get used to it. The same goes for unit tests. At first, they seem to add a lot of extraneous code for something that is clearly trivial, but over time, they are worth it.

My way of doing test driven development (TDD)

I have heard that test driven development is the way to go. In practice, I have found it a little bit odd to write a test before doing anything. I mean, how do you write a test for a function that does not exist? Maybe this is clear to others, or maybe I am doing things the wrong way, but here is how I do TDD.

1. Write an interface (either a class or function). The interface should have some dummy implementation so that it compiles and even runs.
2. Write one test that checks one feature of the interface. This test should fail.
3. Update the implemntation such that the one test pases.
4. Repeat steps 2 and 3, until your implementation supports every feature you need.

I find that this process makes me think about a good interface, and it makes me think whether any feature would pass the old YAGNI (You Aint Gonna Need It) smell test. It works for me, for now.

Pros and cons of Unit Tests

I did a code review for a co-worker today.
I suggested that he refactor his code to make unit tests possible and then to add unit tests.

He argued against adding the tests. Here are some of his arguments against unit testing along with my responses.

Against Unit Testing	Responses.
You're making the code more complex! You want me to add factories and abstract classes and so on, just so things are more testable. You're asking me to rewrite a lot. I don't want to re-write code that already works.	This is true. Some things got more complex. But see, writing code to become more testable, has the side effect of making the code more modular. And making code more modular is good.
You have to learn a lot to get going. New frameworks, new tools, new concepts. All of these new things make things more confusing and complex. This might be a fad that we forget about in a month or two.	Learning is part of the job of being a programmer. It's one of my favorite parts. Maybe the framework and tools we use for testing will change over time. But you can't argue that the concepts are a waste to learn about. Even if you don't want to use the concepts, it's cool to understand and learn. No?
Writing test code is a pain in the neck. You are doubling my work load with no increase in production!	I disagree. Writing test code is fun.
I've already tested it. Trust me, it works. Why write a test?	Writing unit tests allow someone else (even a dumb machine) to test your code. Otherwise, if I have to change your code, I'll have to stop by your desk and you'll have to tell me about it. Isn't that a waste of your time?

Android Testing

For the last few days, I have been adding more tests to my Android projects. During this time, I have come to the conclusion that:

Unit tests are impossible in Android.

Maybe I should qualify that statement a little bit. Here's something that is more accurate:

For me, setting up unit tests in Android is so difficult, and running them is so slow, and learning and integrating external tools/code is so arduous, that it doesn't seem worth it to hack together a true set of unit tests.

In the mean time, here are the things I am doing in regards to Android testing.

1. Sticking with the emulator
I am running all of my tests and the app in the emulator or an attached physical device. The downside of this is that things are slow. I briefly tried to use Robotium, a highly regarded test framework that helps run your tests outside of the emulator. Maybe I will try again in the future. For now, I am wary of investing more time to hunt down external tools to get this to work.

2. Using the simplest test case that is needed.
I can't seem to keep track of all the test case classes. Here are the ones I use in order of increasing complexity and dependency on Android APIs:

TestCase > AndroidTestCase > ActivityUnitTestCase > ActivityInstrumentationTestCase.

For any given class that I'd like to test, I try to use the simplest test case that works. It seems like this would be the best way to isolate issues and keep things running as fast as possible.

3. Refactoring my code
In the hope of making each class as simple as possible, I have been refactoring my code. In the least, I have been adding comments to say what dependencies a given class has which tells me which test case class is required.

4. Waiting.
I started developing an Android App just to see how things work. It's a learning adventure, not a business one. This is why I can afford to simply wait for the product to improve.

In the future, I may upgrade my machine, which will speed things up. Google may add more libraries, update Android Studio or modify the testing framework to make things easier.

Always be shipping - onward and upward

It's been said before (by smarter people than me), and it'll be said again. In any case, I feel like blogging about it today, and so it's the subject of this post -- Always Be Shipping.

When I say 'Always Be Shipping', I mean that code should always be moving forward. Code can be in a lot of stages.

At the lowest stage, it's just an idea in my mind.
'I think our app should provide some calculations for the user'.

Then, it might just be a "TODO:" with pseudo code in the real code.  Example,
// TODO:
// loop through item list here
//   for each item hook up to new api 
// return new amazing data

The pseudo code eventually becomes real compiled/interpreted functioning code that you can test on your development machine.
Woohoo - it works on my machine!

If it's working on your machine, you should start rolling the code out. Even in my hobby projects, I have a few levels of roll out. For Android apps, I have:

• I start with the Emulator.
• Then I have a test android device.
• Next, is my actual phone that I use on a day to day basis. 
• Then I can enable for some alpha testers. 
• Then I can slowly roll out to some percentage of users. 
• Then I can roll out to all users. 

If you find yourself stalling at any point of this process, you should figure out why, and then push forward.

If you have lots of ideas and you never get around to coding them up, then you need to get focused and implement one idea.  A functioning piece of code is better than lots of theoretical brainstorming ideas. Every great product had a 0.1 version. Go ahead and write the 0.1 version so you can start on the 0.2 version sooner. Otherwise, you will find yourself at 0 for a while.

If you have a functioning version of your code, but haven't released it to the world, you should ask yourself why. Are you afraid of bad client feedback?  Get a friend to try out the change and get their opinion.

Are you afraid of performance issues? That's good, but don't just sit and worry. Figure out a way to quickly turn off your change if needed. If things go awry, you can get back to a steady state easily. Even if things go bad, at least you have identified a real problem that needs to be fixed. The alternative is that you are optimizing for doomsday scenarios that never occur. In the future, you can use this mechanism (of quickly turning off a change) so you aren't stalled in the future.

For most software development these days, shipping code is something you should do early and often. This is true for other things. For example, when blogging, I try to tell myself -- Always be publishing. It's easy for me to write half a blog post, get tired, and leave it unpublished as a draft. It would be nice if all our code and blog posts were perfect before shipping/publishing, but then we'd never ship anything.

So - stop reading and start shipping.

Testing is Fun

When I started programming, testing was the thing you did after you finished your first draft of a program.  If you've built a website, you may test that it displays the page as you designed it.  If you've built a calculator, you test that adding positive numbers, negative numbers and fractions all work right.  After testing, you'd find some bugs, fix them and say "Oh well, I guess it's time to ship".

Better late than never, I finally hopped aboard the test driven development band wagon.  Armed with Google Test, I'm beginning to make testing part of the early stages of my development process.  There are probably lists that detail all the benefits to test driven development, and you can and should search for them on your own.  But some benefits that I've personally experienced and felt are:

1. Thinking about testing makes me write better code.  If you want to write unit tests, you need to separate your program into distinct modular components, so you can test one part at a time.  Of course, writing small byte-size units of code is generally a good programming practice anyway - so that's one win-win. 
2. Testing gives you confidence.  All programs have some bugs, but testing helps you out in a few ways. 
1. Before you check in code, testing might help you catch a bug.  So, buggy code doesn't enter your repository.  Nobody but you knows that you're a sloppy programmer!
2. After your code has rolled out to the world, your test might find a bug before any user's notice it.  Although not ideal, maybe you can roll back changes or submit a fix before any clients begin complaining.  Only your coworkers know you made a mistake!
3. One of your clients spots a bug and you get called up to fix it.  Testing will help you isolate the problem.  You know what tests have been written and what edge cases haven't thoroughly been tested.  Hopefully your tests help you fix the problem faster.  So, your clients know that you had bugs, but they marvel at your customer service.  

Now that I've experienced some of the benefits of testing, I can't imagine writing code without them.   

Simple C++ program with Google test

I have little to no experience with unit testing.  I want to change that.

In principle, unit testing sounds simple.  Unit tests are used to isolate and validate small parts of a program are working.  In practice, I don't really know how it's done.

As a first attempt, I tried to build a simple program with tests using google test, https://code.google.com/p/googletest/.  After downloading and unzipping the library, I moved it into a folder -- SomeDirectory/libs/gtest-1.6.0.  I then copied these files into a separate directory.

• Makefile -- this is used by make to build both the test program and my program.
• sample1.h, sample1.cc -- These define some functions that are linked into the test program and into my program.
• sample_unittest.cc -- This defines test macros that are run in the test program.

If you put these files into a directory, you are almost ready to run the test.  You will have to update the paths in the Makefile.

• GTEST_DIR = SomeDirectory/libs/gtest-1.6.0
• USER_DIR = ./ 

You can run then run "make" followed by "sample1_unittest" and you should see some successful test output.

I wanted to add a real program with this test suite, so I wrote this:
To build the program I updated the Makefile to build "my_program" like this:
TESTS = sample1_unittest my_program

and then I added this to the end of the Makefile:

my_program : sample1.o main.o 

$(CXX) $(CPPFLAGS) $(CXXFLAGS) -lpthread $^ -o $@

After running make, I had two programs.  "sample1_unittest" was the original test program and "my_program" was the 'real' program that I was testing.