Chapter 4 and 5 activity

After reading chapter 4 and 5 from Introduction to Software Testing, we were asked to answer these questions about TDD and test coverage criteria:

  1. What is “correctness” in agile processes?
    Correctness for testing and software engineers has to do with the quantity of green tests, this means that our code is correct if it passes all the tests.
  2. Do TDD tests do a good job testing the software?
    No, TDD tests are mainly to define the requirements and specifications of our code, but not really to evaluate the right behavior or to cover edge cases.
  3. Can we automate our tests without TDD?
    Yes, automation can be done in test without following the TDD principles. Imagine writing first the code, then just executing the tests to validate it.
  4. Can we use TDD without automating our tests?
    Yes, TDD consists on putting the tests first as a guide for the code to be written, automation can help to write tests and validate faster, but is not mandatory.
  5. What four structures do we use for test criteria?
    1.- Test Requirement: A test requirement is a specific element of a software artifact that a test case must satisfy or cover.
    2.- Coverage Criterion: A coverage criterion is a rule or a collection of rules that impose test requirements on a test set.
    3.- Minimal Test Set: Test set T such that if a test was removed from T, it no longer satisfies all test requirements.
    4.- Minimum Test Set: Smallest possible test set T that satisfies all test requirements.
  6. What usually prevents our tests from achieving 100% coverage?
    What usually prevents our tests from achieving 100% coverage?
    Test requirements that cannot be satisfied are called infeasible. Formally, no test case values exist that meet the test requirements. The detection of infeasible test requirements is formally undecidable for most coverage criteria, and even though researchers try to find partial solutions, they have had only limited success. Thus, 100% coverage is impossible in practice.

  7. Some organizations in industry who adopt TDD report that it succeeds very well, and others report that it fails. Based on your knowledge of TDD and any experience you have, why do you think it succeeds sometimes but not all?
    Companies does not use TDD well, that’s why they don’t succeed using it. It is hard for some engineers to follow TDD principles, some of us are not used to write the tests first before the code.
  8. A few software organizations use test criteria and report great success. However, most organizations do not currently use test criteria. Based on your knowledge and experience, why do you think test criteria are not used more?
    We think that test criteria is very ambiguous, as we don’t have an exact default way of measuring test criteria, everyone does it differently. Maybe having a high test coverage is expensive and most of the times is not worth it, that’s why most companies decide to not focus on having  heavy testing development.

Chapter 4 activity

For this post we are going to show how we solved the exercises for chapter 4 using TDD. We need to add new operations like multiplying and division for the Calc.java file. The test file only has a test for the add operation as it is the only function we have in the class.

testcalc1

First of all, we need to write simple tests that describe what our new operations do. Of course, these tests should fail as we haven’t coded the functions yet.

testcalc2

Now we can start writing the functionality now, we are going to start with the subtraction as it is the first test we should pass to move to the next one. On this exercise the order we decide to write the new functions does not matter, but in more complex projects we should have a specific order to write the new functions because some tests might have multiple dependencies.

testcalc3

The next step is to write the test for the divide function. Here we have a problem, we don’t know how to implement this function because it can return a double or an integer. To follow the TDD correctly first we have to make the test pass, right now the test only ask for an integer so we will write just the code to make that correct.

testcalc4

At last, the multiply function. Here again we ask ourselves if this function should be capable of accepting float values, but it is not specified on our tests so by now we will handle only integers for the input and output.

testcalc5

 

Now lets imagine some guys from testing added new tests, now with more specific cases trying to find faults in our code. They included a test that divides two integers but the result is non integer, a test for a division by zero,  and a test with negative integers. After running the tests this happens:

testcalc6

As we can notice, only the division function is not passing the new tests. On one test it is expecting float numbers, on the other test, it is expecting us to catch an error when the divisor is zero. This means that we need a refactor to return doubles and to detect divisions by zero. This will change some of our previous tests, the first division test is working but it is expecting integers, the new test is expecting  doubles, there is an inconsistency in the tests and we should define this with the team first. The final decision is that a division should return doubles because it gives a more exact result. With that information we can go back to the tests, modify them accordingly and now start working on the refactor.

testcalc7

With this exercise we practiced TDD and refactoring. We learned how important it is to have good tests that defines the requirements for each function and that show how specific behavior should be handled. Also, it showed us how in TDD the tests are the guideline of our code and that it can show design errors or inconsistencies before we start writing the code, a huge advantage over other methodologies.

The code of the exercise is available here:

Calc.java

CalcTest.java

In Class Test

 

  1. Given the 4 @Test methods shown, how many times does the @Before method execute?
    Just one, it initializes global variables. In this case, we are creating objects from the class we want to test.
  2. The contract for equals() states that no exceptions may be thrown. Instead, equals() is supposed to return false if passed a null argument.
    Write a JUnit test that verifies this property for the EH class.

    @Test public void noNPE() {
           assertEquals(false, eh1.equals(null));
    }
  3. Using the given EH objects, write a test that verifies that equals() returns false if the objects are, in fact, not equal.
    @Test public void equalsFalse() {
           assertEquals(false, eh1.equals(eh2));
    }
  4. Using the given EH objects, write a test that verifies that equals() returns true if the objects are, in fact, equal.
    @Test public void equalsTrue() {
           assertEquals(true, eh1.equals(eh3));
    }
  5. Using the given EH objects, write a test to verify that hashCode() is consistent with equals. This test should fail if hashCode() is commented out (as shown), but pass if hashCode() is implemented.
    @Test public void hashConsistent() {
           assertEquals(true, eh1.hashCode() == eh3.hashCode());
    }