Allows N instances of tests run in parallel, each running 1/N parts
of the test suite under Valgrind, to speed things up.
* tr/test-v-and-v-subtest-only:
perf-lib: fix start/stop of perf tests
test-lib: support running tests under valgrind in parallel
test-lib: allow prefixing a custom string before "ok N" etc.
test-lib: valgrind for only tests matching a pattern
test-lib: verbose mode for only tests matching a pattern
test-lib: self-test that --verbose works
test-lib: rearrange start/end of test_expect_* and test_skip
test-lib: refactor $GIT_SKIP_TESTS matching
test-lib: enable MALLOC_* for the actual tests
With the new --valgrind-only=<pattern> option, one can enable
--valgrind at a per-test granularity, exactly analogous to
--verbose-only from the previous commit.
The options are wired such that --valgrind implies --verbose (as
before), but --valgrind-only=<pattern> implies
--verbose-only=<pattern> unless --verbose is also in effect.
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When TEST_OUTPUT_DIRECTORY setting is used, it was handled somewhat
inconsistently between the test framework and t/Makefile, and logic
to summarize the results looked at a wrong place.
* jk/test-output:
t/Makefile: don't define TEST_RESULTS_DIRECTORY recursively
test output: respect $TEST_OUTPUT_DIRECTORY
t/Makefile: fix result handling with TEST_OUTPUT_DIRECTORY
Most test results go in $TEST_OUTPUT_DIRECTORY, but the output files for
tests run with --tee or --valgrind just use bare "test-results".
Changes these so that they do respect $TEST_OUTPUT_DIRECTORY.
As a result of this, the valgrind/analyze.sh script may no longer
inspect the correct files so it is also updated to respect
$TEST_OUTPUT_DIRECTORY by adding it to GIT-BUILD-OPTIONS. This may be a
regression for people who have TEST_OUTPUT_DIRECTORY in their config.mak
but want to override it in the environment, but this change merely
brings it into line with GIT_TEST_OPTS which already cannot be
overridden if it is specified in config.mak.
Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With --valgrind=memcheck-fast, the tests run under memcheck but
without the autodetected --track-origins. If you just run valgrind to
see *if* there is any memory issue with your program, the extra
information is not needed, and it comes at a roughly 30% hit in
runtime.
While it is possible to achieve the same through GIT_VALGRIND_OPTIONS,
this should be more discoverable and hopefully encourage more users to
run their tests with valgrind.
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Acked-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Running tests under helgrind and DRD recently proved useful in
tracking down thread interaction issues. This can unfortunately not
be done through GIT_VALGRIND_OPTIONS because any tool other than
memcheck would complain about unknown options.
Let --valgrind take an optional parameter that describes the valgrind
tool to invoke. The default mode is to run memcheck as before.
Signed-off-by: Thomas Rast <trast@inf.ethz.ch>
Acked-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some versions of strlen use SSE to speed up the calculation and load 4
bytes at a time, even if it means reading past the end of the
allocated memory. This read is safe and when the strlen function is
inlined, it is not replaced by valgrind, which reports a
false-possitive.
Tell valgrind to ignore this particular error, as the read is, in
fact, safe. Current upstream-released version 3.6.1 is affected. Some
distributions have this fixed in their latest versions.
Signed-off-by: Carlos Martín Nieto <cmn@elego.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Valgrind 3.4.0 is pretty new, and even if --track-origins is a nice
feature, it is not the end of the world if that is not available. So
play nice and use that option only when only an older version of
valgrind is available.
In the same spirit, refrain from the use of '...' in suppression
files, which is also a feature only valgrind 3.4 and newer understand.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After running the valgrind tests with GIT_TEST_TREE=t, the test output
is in the test-results/$TEST.out files.
Call ./valgrind/analyze.sh in $GIT_ROOT/t/ to group the valgrind errors
by backtrace.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
On some Linux systems, we get a host of Cond and Addr errors
from calls to dlopen that are caused by nss modules. We
should be able to safely ignore anything happening in
ld-*.so as "not our problem."
[Johannes: I added some more... unfortunately using valgrind 3.4.0 syntax]
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch adds the ability to use valgrind's memcheck tool to
diagnose memory problems in Git while running the test scripts.
It requires valgrind 3.4.0 or newer.
It works by creating symlinks to a valgrind script, which have the same
name as our Git binaries, and then putting that directory in front of
the test script's PATH as well as set GIT_EXEC_PATH to that directory.
Git scripts are symlinked from that directory directly.
That way, Git binaries called by Git scripts are valgrinded, too.
Valgrind can be used by specifying "GIT_TEST_OPTS=--valgrind" in the
make invocation. Any invocation of git that finds any errors under
valgrind will exit with failure code 126. Any valgrind output will go
to the usual stderr channel for tests (i.e., /dev/null, unless -v has
been specified).
If you need to pass options to valgrind -- you might want to run
another tool than memcheck, for example -- you can set the environment
variable GIT_VALGRIND_OPTIONS.
A few default suppressions are included, since libz seems to trigger
quite a few false positives. We'll assume that libz works and that we
can ignore any errors which are reported there.
Note: it is safe to run the valgrind tests in parallel, as the links in
t/valgrind/bin/ are created using proper locking.
Initial patch and all the hard work by Jeff King.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
