All the expressions in this benchmark are written in the context of the model Library shown above. They are iterator expressions for which we artificially increase the size of the collections to be iterated upon: namely, the terms p1, p2, and p3 refer, respectively, to the expressions Book.allInstances().author.books, p1.author.books, and p2.author.books. The times we show in this benchmark correspond to those obtained by evaluating each expression using the EOS tool.3 All the expressions are evaluated on the same scenario, namely, an instance MyLibrary of the model Library which contains 103 authors, each author with 10 different books, and each book with a title different from ``Hobbit''. In this scenario, the terms p1, p2, and p3 evaluates to collections with 105, 106, and 107 books, respectively. We run this benchmark on a laptop computer, with Windows XP Professional installed, a processor Intel Pentium M 2.00GHz 600MHz, and 1GB of RAM. Also, we run the JVM with its parameters -Xms and -Xmx set to 1024m.
| p1 | 80ms | |
| p2 | ->collect(x|x.title)->size() | 391ms |
| p3 | 3s 896ms | |
| p1 | 90ms | |
| p2 | ->collect(x|x.title <> 'Hobbit')->size() | 481ms |
| p3 | 4s 516ms | |
| p1 | 240ms | |
| p2 | ->collect(x|x.author.books)->size() | 2s 140ms |
| p3 | 17s 736ms | |
| p1 | 221ms | |
| p2 | ->collect(x|x.author.books->includes(x))->size() | 1s 893ms |
| p3 | 17s 906ms | |
| p1 | 251ms | |
| p2 | ->forAll(x|x.author.books->includes(x)) | 1s 963 |
| p3 | 17s 30ms | |
| p1 | 260ms | |
| p2 | ->select(x|x.author.books->includes(x))->size() | 2s 13ms |
| p3 | 17s 605ms | |
| p1 | 290ms | |
| p2 | ->collect(x|x.author.books.title)->size() | 2s 573ms |
| p3 | 23s 684ms | |
| p1 | 270ms | |
| p2 | ->collect(x|x.author.books.title->size())->sum() | 2s 274ms |
| p3 | 20s 840ms | |
| p1 | 280ms | |
| p2 | ->forAll(x|x.author.books.title->excludes('Hobbit')) | 2s 604ms |
| p3 | 22s 802ms |
1. Interestingly, this quality ---OCL engine efficiency on medium-large scenarios--- was not covered by the benchmark proposed in the paper "A Benchmark for OCL Engine Accuracy, Determinateness, and Efficiency" written by Martin Gogolla, Mirco Kuhlmann and Fabian Büttner, which was presented at the MODELS 2008 Conference: in fact, the largest scenarios scenario proposed by them for testing OCL engine efficiency contains only 42 objects and 42 links among them.
2. A more "natural" approach will be to consider scenarios with larger number of books but, as we discuss in our paper, none of the currently available tools support well the loading of scenarios with more than 106 objects.
3. The paper "Building an Efficient Component for OCL evaluation" also includes the evaluting performance of MDT OCL and the USE tools.
