Boost C++ Libraries Home Libraries People FAQ More

PrevUpHomeNext

Sequences and collections comparison

Instead of comparing one singular value against another, there is often the need to compare sequences or sets of values. Most of the time, the test for the full sequences is achieves through loops, or by using the native available operators for which no additional diagnostic is provided.

BOOST_TEST automatically detects containers, and let the user specify the comparison method:

More details about what is considered to be a containers is given here.

Default comparison

The default comparison dispatches to the existing overloaded operator. Given two containers c_a and c_b,

BOOST_TEST(c_a op c_b)

is equivalent, in terms of test success, to

auto result = c_a op c_b;
BOOST_TEST(result);

In the example below, operator== is not defined for std::vector of different types, and the program would fail to compile if the corresponding lines were uncommented. std::vector uses lexicographical compare by default.

Example: BOOST_TEST containers comparison default

Code

#define BOOST_TEST_MODULE boost_test_sequence
#include <boost/test/included/unit_test.hpp>
#include <vector>

BOOST_AUTO_TEST_CASE( test_collections_vectors )
{
  std::vector<int> a{1,2,3}, c{1,5,3,4};
  std::vector<long> b{1,5,3};

  // the following does not compile
  //BOOST_TEST(a == b);
  //BOOST_TEST(a <= b);

  // stl defaults to lexicographical comparison
  BOOST_TEST(a < c);
  BOOST_TEST(a >= c);
  BOOST_TEST(a != c);
}

Output

> ./boost_test_container_default --log_level=all
Running 1 test case...
Entering test module "boost_test_sequence"
test.cpp(13): Entering test case "test_collections_vectors"
test.cpp(23): info: check a < c has passed
test.cpp(24): error: in "test_collections_vectors": check a >= c has failed
test.cpp(25): info: check a != c has passed
test.cpp(13): Leaving test case "test_collections_vectors"; testing time: 208us
Leaving test module "boost_test_sequence"; testing time: 286us

*** 1 failure is detected in the test module "boost_test_container_default"

Element-wise comparison

By specifying the manipulator boost::test_tools::per_element, the comparison of the elements of the containers will be performed per-element, in the order given by the forward iterators of the containers. This is a comparison on the sequences of elements generated by the containers, for which the Unit Test Framework provides advanced diagnostic.

In more details, let c_a = (a_1,... a_n) and c_b = (b_1,... b_n) be two sequences of same length, but not necessarily of same type. Those sequences correspond to the content of the respective containers, in the order given by their iterator. Let op be one of the binary comparison operators.

BOOST_TEST(c_a op c_b, boost::test_tools::per_element() );

is equivalent to

if(c_a.size() == c_b.size())
{
  for(int i=0; i < c_a.size(); i++)
  {
    BOOST_TEST_CONTEXT("index" << i);
    BOOST_TEST(a_i op b_i);
  }
}
else
{
  BOOST_TEST(c_a.size() == c_b.size());
}
[Warning] Warning

this is fundamentally different from using the containers' default comparison operators (default behaviour).

[Warning] Warning

this is not an order relationship on containers. As a side effect, it is possible to have eg.

BOOST_TEST(c_a == c_b)

and

BOOST_TEST(c_a != c_b)

failing at the same time

Sequences are compared using the specified operator op, evaluated on the left and right elements of the respective sequences. The order of elements is given by the iterators of the respective containers [11]. In case of failure, the indices of the elements failing op are returned.

Example: BOOST_TEST sequence comparison

Code

#define BOOST_TEST_MODULE boost_test_sequence_per_element
#include <boost/test/included/unit_test.hpp>
#include <vector>
#include <list>
namespace tt = boost::test_tools;

BOOST_AUTO_TEST_CASE( test_sequence_per_element )
{
  std::vector<int> a{1,2,3};
  std::vector<long> b{1,5,3};
  std::list<short> c{1,5,3,4};

  BOOST_TEST(a == b, tt::per_element()); // nok: a[1] != b[1]

  BOOST_TEST(a != b, tt::per_element()); // nok: a[0] == b[0] ...
  BOOST_TEST(a <= b, tt::per_element()); // ok
  BOOST_TEST(b  < c, tt::per_element()); // nok: size mismatch
  BOOST_TEST(b >= c, tt::per_element()); // nok: size mismatch
  BOOST_TEST(b != c, tt::per_element()); // nok: size mismatch
}

Output

> ./boost_test_sequence_per_element
Running 1 test case...
test.cpp(21): error: in "test_sequence_per_element": check a == b has failed
Mismatch at position 1: 2 != 5.
test.cpp(23): error: in "test_sequence_per_element": check a != b has failed
Mismatch at position 0: 1 == 1.
Mismatch at position 2: 3 == 3.
test.cpp(25): error: in "test_sequence_per_element": check b < c has failed
Collections size mismatch: 3 != 4
test.cpp(26): error: in "test_sequence_per_element": check b >= c has failed
Collections size mismatch: 3 != 4
test.cpp(27): error: in "test_sequence_per_element": check b != c has failed
Collections size mismatch: 3 != 4

*** 5 failures are detected in the test module "boost_test_sequence_per_element"
Requirements

For the sequences to be comparable element-wise, the following conditions should be met:

[Caution] Caution

the resulting type of "c_a == c_b" is an assertion_result. It is not possible to compose more that one comparison on the BOOST_TEST statement:

BOOST_TEST(c_a == c_b == 42, boost::test_tools::per_element() ); // does not compile

Lexicographic comparison

By specifying the manipulator boost::test_tools::lexicographic, the containers are compared using the lexicographic order, for which the Unit Test Framework will provide additional information in case of failure.

BOOST_TEST(c_a op c_b, boost::test_tools::lexicographic() );

The comparison is performed in the order given by forward iterators of the containers.

[Tip] Tip

lexicographic comparison yields a total order on the containers: the statements c_a < c_b and c_b <= c_a are mutually exclusive.

Example: BOOST_TEST container comparison using lexicographical order

Code

#define BOOST_TEST_MODULE boost_test_container_lex
#include <boost/test/included/unit_test.hpp>
#include <vector>

namespace tt = boost::test_tools;

BOOST_AUTO_TEST_CASE( test_collections_vectors_lex )
{
  std::vector<int> a{1,2,3}, b{1,2,2}, c{1,2,3,4};

  BOOST_TEST(a < a, tt::lexicographic());
  BOOST_TEST(a < b, tt::lexicographic());
  BOOST_TEST(a < c, tt::lexicographic());
  BOOST_TEST(a >= c, tt::lexicographic());

  // does not compile
  //BOOST_TEST(a == c, tt::lexicographic());
  //BOOST_TEST(a != c, tt::lexicographic());
}

Output

> ./boost_test_container_lex --log_level=all
Running 1 test case...
Entering test module "boost_test_container_lex"
test.cpp(15): Entering test case "test_collections_vectors_lex"
test.cpp(19): error: in "test_collections_vectors_lex": check a < a has failed
Collections appear to be equal.
test.cpp(20): error: in "test_collections_vectors_lex": check a < b has failed
Failure at position 2: 3 >= 2.
test.cpp(21): info: check a < c has passed
test.cpp(22): error: in "test_collections_vectors_lex": check a >= c has failed
Second collection has extra trailing elements.
test.cpp(15): Leaving test case "test_collections_vectors_lex"; testing time: 267us
Leaving test module "boost_test_container_lex"; testing time: 341us

*** 3 failures are detected in the test module "boost_test_container_lex"
Requirements

What is a sequence?

A sequence is given by the iteration over a forward iterable container. A forward iterable container is a container (C++11):

To that respect, C-arrays are not forward iterable containers:

Example: BOOST_TEST C-arrays

Code

#define BOOST_TEST_MODULE boost_test_container_c
#include <boost/test/included/unit_test.hpp>
#include <sstream>
#include <map>
#include <vector>

BOOST_AUTO_TEST_CASE( test_collections_not_on_c_arrays )
{
  int a[] = {1, 2, 3};
  int b[] = {1, 5, 3, 4};
  BOOST_TEST(a == b);
}

Output

> ./boost_test_macro_container_c_array --log_level=all
Running 1 test case...
Entering test module "boost_test_container_c"
test.cpp(15): Entering test case "test_collections_not_on_c_arrays"
test.cpp(19): error: in "test_collections_not_on_c_arrays": check a == b has failed [0x7fff526e5bc4 != 0x7fff526e5bb0]
test.cpp(15): Leaving test case "test_collections_not_on_c_arrays"; testing time: 323us
Leaving test module "boost_test_container_c"; testing time: 526us

*** 1 failure is detected in the test module "boost_test_container_c"

The detection of the containers is delegated to the class boost::unit_test::is_forward_iterable, which for C++11 detects the required member functions and fields. However for C++03, the types providing the sequences should be explicitly indicated to the Unit Test Framework by a specialization of boost::unit_test::is_forward_iterable [12].



[10] either defined by the container or by the user

[11] the containers should yield the same sequences for a fixed set of elements they contain

[12] Standard containers of the STL are recognised as collections.


PrevUpHomeNext