JUnit Lab

Emma reported 100% coverage for my stack project (download). Good. Awesome. Done?

Nope. As the readings pointed out, there could still be a bug. To prove this, we will purposely insert a bug into the Stack project and show this can happen even with 100% coverage.

Program with a bug? What kind of 100% is this!?

Emma cannot read between the lines. It cannot determine what is a good test. It only reveals what lines have been used in the test.

As hinted in class, if the Stack's list were a static size, then an error like IndexOutofBounds (or something similar) should happen if the number of objects pushed on exceeds the maximum length (limit) of the stack. Emma would never be able to report such a case because it doesn't check the boundaries. It only checks the test cases given by the programmer and if the line was ever executed in its respective class.

Unfortunately, I could not figure out how to create a list object with a static size. The list would dynamically extend itself and make room for more objects even when I set an initial size for the object. The only way I know how to resolve this is to change the list to a simple array. However, I avoided that since the Stack's entire project manipulated that very list. To change that would mean to change practically everything.

So instead, I implemented a test that would push on a very large number of objects onto the stack. The point was to exceed whatever the Stack limit is and throw an error. Since I didn't know this limit's exact amount, I started with base 10 and increased it exponentially to determine the limit. My test would throw an OutOfMemoryError after 10,000,000 pushes.

How Do We Know if the Test is Good?

So there are still bugs to discover even after achieving 100% coverage. It's scary because how does one know when they are truly finished and left with a good test? I have no clue. Is testing ever really absolute 100% perfect. I doubt it. It's probably better to let other hackers try to crash your system if you have that luxury. I didn't, but I do have other blogs to check out since we're all working on the same assignment. Some things are just more apparent to others. But the more scenarios and equivalence classes tested, the more robust the test and program will be.