Practicing BDD with integration tests — do I need unit tests too?

At present, my development process flows like this:

  1. I describe the expected behaviour as an integration test using using WebRat
  2. I write the Ruby on Rails code to provide that behaviour, so passing the test
  3. I refactor, ensuring the tests still pass at the end of the process
  4. I write the next integration test

It seems to me that by definition, my integration tests are testing every model, controller and view that I can create. In reality, am I missing anything by not writing unit tests too?

13.10.2009 21:28:27

Do you have any rake tasks? Custom capistrano code? Cron methods? An API? Monkeypatches? How about flex or iPhone app integration? A job runner?

In a typical Rails application, there's lots of code that isn't exercised by the HTML UI. So no, unless your app is incredibly simple, webrat tests won't be sufficient.

13.10.2009 23:03:11
Thanks Sarah. So far, no. It's a simple web app -- deliberately simple. Unless/until I build code that isn't exercised by the HTML UI, I guess WebRat is enough? And if I come to eventually build code that isn't exercised by the HTML UI, I guess some sort of integration test (obviously not WebRat) will be enough? Or is there a genuine need to test each unit separately in addition to testing them through integration tests?
steven_noble 13.10.2009 23:50:43
It depends on your goals. If you are using tests as a safety net for refactoring, or as a way to catch bugs when you add code, then you'll still want unit tests. No set of tests will ever test everything, so yes, you'll have what seem like redundant tests.
Sarah Mei 14.10.2009 02:40:54

Integration tests are useful to verify that different parts of code are well integrated. They may involve all layers and cover all code but... when an integration test fails, will it tell you where the bug is located? I may be wrong but I don't think so. This will just tell you that there is a problem somewhere. On the other hand, when a real unit tests (written in isolation using mocks or stubs) fails, you know exactly in which unit the problem is located (this is actually the purpose of unit testing, verifying that a unit implements the expected behavior). In other words, unit tests and integration tests are both useful but they have different purposes.

24.10.2009 21:04:16
Or, better yet written in isolation with stubs. About 80-90% of the time test expectations are more reliable when they're run against the interface. If two implementations of a method take the same parameters and give the same return values for all possible parameters, then usually you want the same set of tests to pass for both implementations. With a test double that doesn't have expectations, the tests will typically work, but not so with mocks.
Bob Aman 24.10.2009 20:57:01
The other 10-20% of the time are usually either API calls or code with side-effects that you should probably refactor anyways.
Bob Aman 24.10.2009 20:58:10
@Bob that's right and I should have written "written in isolation". I'll update my answer. Mocks or stubs, I don't care that much actually as long as test are written in isolation (and mocks are overused IMHO).
Pascal Thivent 24.10.2009 21:03:51

I'm actually pretty sympathetic to your point of view here. I love Cucumber and I love RSpec -- and I use them both, but not always on the same code. For instance, I rarely write RSpec examples for Rails controllers these days, and I almost never write view specs. Most of my controllers are very similar and don't deviate much from the "stock" controller pattern -- which is already well-tested by Rails's own unit tests. Verifying the same behavior again doesn't gain much for the time it takes and the hassle of mocking all the models. With Cucumber at an integration level I can skip that mocking and get the essential verification I'm looking for. View testing is handled much more transparently in Cucumber most of the time as well. (Then I should see "foo" and so forth.)

But that isn't to say I don't use RSpec extensively in Rails. I use it for the places where my logic lives: models, controller filters, and view helpers. I also have a couple of projects that are almost all business logic, e.g. libraries or API adapters against complex third-party interfaces. For those I usually find myself using RSpec exclusively and skipping Cucumber.

As a heuristic, I'd suggest that you should strongly consider writing a unit test any time any of the following questions can be answered "Yes":

  • Is the code I'm writing more than trivially complicated?
  • Does this code exist primarily to give answers to other code?
  • Is this existing code that I'm refactoring (that doesn't already have a unit test)?
  • Have I found a bug in this code? (If so, write a unit test before fixing it so it never sneaks in again.)
  • Do I have to think for more than ten seconds about the most elegant way to implement this code?
  • Is my Spidey Sense tingling?

If none of the above is true, then maybe you can get away with just doing integration testing. Again, there are a lot of cases where that's reasonable. But if you do run into problems later, be prepared to pay the price -- and that price should include writing unit tests at any moment if they seem called for.

20.10.2009 02:29:19
+1 to avoiding specs on controllers and views. Unit test web service controller calls, models, and stuff in lib, but otherwise leave it to Cucumber.
Bob Aman 24.10.2009 21:01:24