mirrors/git
1
0
Fork 0
mirror of https://github.com/git/git.git synced 2024-11-01 14:57:52 +01:00
Code Activity
git/t/Makefile

98 lines
2.5 KiB
Makefile
Raw Normal View History

[PATCH 1/2] Test framework take two. This adds t/ directory to host test suite, a test helper library and a basic set of tests. Petr Baudis raised many valid points at the earlier attempts in git mailing list. This round, test-lib.sh has been updated to a bit more modern style, and the default output is made easier to read. Also included is one sample test script that tests the very basics. This test has already found one leftover bug missed when we introduced symlink support, which has been fixed since then. The supplied Makefile is designed to run all the available tests. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Petr Baudis <pasky@ucw.cz>
2005-05-14 07:50:32 +02:00
# Run tests
#
# Copyright (c) 2005 Junio C Hamano
#
t/Makefile: OPTS -> GIT_TEST_OPTS This way, passing this variable through the environment actually makes sense.
2005-05-14 17:58:22 +02:00
test_cmp: do not use "diff -u" on platforms that lack one By default the testsuite calls 'diff -u' whenever a file comparison is called for. Unfortunately that throws a "diff: unknown option '-u'" error for most non-GNU diffs. This patch sets GIT_TEST_CMP to 'cmp' on all the architectures where that happens. The previous version of this patch forgot to export GIT_TEST_CMP from t/Makefile, which is why 'make test' continued to fail most tests on most architectures - test-lib.sh was falling back on its default of `diff -u' for GIT_TEST_CMP. This version of this patch shows a vast improvement in testsuite results where either GNU diff is in the path at configure time, or where Makefile knows that GIT_TEST_CMP=cmp is required. Signed-off-by: Gary V. Vaughan <gary@thewrittenword.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-05-14 11:31:38 +02:00
-include ../config.mak.autogen
t/Makefile: include config.mak This is useful if you want to specify GIT_TEST_OPTS that you always use. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-08-09 10:50:37 +02:00
-include ../config.mak
Makefile: whitespace style fixes in macro definitions Consistently use a single space before and after the "=" (or ":=", "+=", etc.) in assignments to make macros. Granted, this was not a big deal, but I did find the needless inconsistency quite distracting. Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-09 11:36:17 +01:00
#GIT_TEST_OPTS = --verbose --debug
[PATCH] Make the test more shell generic and fix missing Solaris find option This is from Peter Eriksen, but further fixed. Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-09-24 21:50:29 +02:00
SHELL_PATH ?= $(SHELL)
t/Makefile: Can't include GIT-BUILD-OPTIONS, it's a .sh Change the smoke testing portion of t/Makefile not to include GIT-BUILD-OPTIONS. It's a shellscript, not a Makefile snippet, so it had the nasty side-effect of sneaking e.g. SHELL_PATH = '/bin/sh' (with quotes) everywhere. Just add our own PERL_PATH variable as a workaround. The t/Makefile already has e.g. an equivalent SHELL_PATH and TAR option which duplicate the definitions in GIT-BUILD-OPTIONS. Reported-by: Thomas Rast <trast@student.ethz.ch> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-08 16:49:26 +02:00
PERL_PATH ?= /usr/bin/perl
More portability. - The location of openssl development files got customizable. - The location of iconv development files got customizable. - Pass $TAR down to t5000 test so that the user can override with 'gmake TAR=gtar'. - Solaris 'bc' does not seem to grok "define abs()". There is no reason to use bc there -- expr would do. Signed-off-by: Junio C Hamano <junio@twinsun.com>
2005-09-30 22:31:16 +02:00
TAR ?= $(TAR)
Use $(RM) in Makefiles instead of 'rm -f' Signed-off-by: Emil Medve <Emilian.Medve@Freescale.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-07-14 19:51:44 +02:00
RM ?= rm -f
test: allow running the tests under "prove" You can run "make DEFAULT_TEST_TARGET=prove test" to run the test under "prove" (or $(PROVE) if set). The output is a bit easier to read when running many tests in parallel. Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net> Liked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Liked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-10-14 10:53:36 +02:00
PROVE ?= prove
DEFAULT_TEST_TARGET ?= test
t/Makefile: always test all lint targets when running tests Only the two targets "test-lint-duplicates" and "test-lint-executable" are currently executed when running the test target. This was done on purpose when the TEST_LINT variable was added in 81127d74 to avoid twisted shell scripting by developers only to avoid false positives that might result from the rather simple minded tests, e.g. test-lint-shell-syntax. But it looks like it might be better to include all lint tests to help developers to detect non portable shell constructs before the patch is sent to the list and reviewed there. Change the TEST_LINT variable to run all lint test unless the TEST_LINT variable is overridden. If we hit false positives more often than helping developers to avoid non-portable code (or add less accurate or slow tests later) we could still fall back to exclude them like 81127d74 proposed. Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-07-09 21:34:42 +02:00
TEST_LINT ?= test-lint
[PATCH 1/2] Test framework take two. This adds t/ directory to host test suite, a test helper library and a basic set of tests. Petr Baudis raised many valid points at the earlier attempts in git mailing list. This round, test-lib.sh has been updated to a bit more modern style, and the default output is made easier to read. Also included is one sample test script that tests the very basics. This test has already found one leftover bug missed when we introduced symlink support, which has been fixed since then. The supplied Makefile is designed to run all the available tests. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Petr Baudis <pasky@ucw.cz>
2005-05-14 07:50:32 +02:00
t/Makefile: fix result handling with TEST_OUTPUT_DIRECTORY When TEST_OUTPUT_DIRECTORY is set, the test results will be generated in "$TEST_OUTPUT_DIRECTORY/test-results", which may not be the same as "test-results" in t/Makefile. This causes the aggregate-results target to fail as it finds no count files. Fix this by introducing TEST_RESULTS_DIRECTORY and using it wherever test-results is referenced. Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-26 20:55:52 +02:00
ifdef TEST_OUTPUT_DIRECTORY
t/Makefile: don't define TEST_RESULTS_DIRECTORY recursively Commit 54bb901 (t/Makefile: fix result handling with TEST_OUTPUT_DIRECTORY - 2013-04-26) incorrectly defined TEST_RESULTS_DIRECTORY relative to itself, when it should be relative to TEST_OUTPUT_DIRECTORY. Fix this. Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-06 14:35:46 +02:00
TEST_RESULTS_DIRECTORY = $(TEST_OUTPUT_DIRECTORY)/test-results
t/Makefile: fix result handling with TEST_OUTPUT_DIRECTORY When TEST_OUTPUT_DIRECTORY is set, the test results will be generated in "$TEST_OUTPUT_DIRECTORY/test-results", which may not be the same as "test-results" in t/Makefile. This causes the aggregate-results target to fail as it finds no count files. Fix this by introducing TEST_RESULTS_DIRECTORY and using it wherever test-results is referenced. Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-26 20:55:52 +02:00
else
TEST_RESULTS_DIRECTORY = test-results
endif
Deal with $(bindir) and friends with whitespaces. ... using HPA's shellquote macro. Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-10 22:50:01 +02:00
# Shell quote;
Fixes for ancient versions of GNU make Some versions of GNU make do not understand $(call), and have problems to interpret rules like this: some_target: CFLAGS += -Dsome=defs [jc: simplified substitution a bit. ] Signed-off-by: Johannes E. Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-02-18 12:40:22 +01:00
SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH))
test: Add check-non-portable-shell.pl Add the perl script "check-non-portable-shell.pl" to detect non-portable shell syntax. "echo -n" is an example of a shell command working on Linux, but not on Mac OS X. These shell commands are checked and reported as error: - "echo -n" (printf should be used) - "sed -i" (GNUism; use a temp file instead) - "declare" (bashism, often used with arrays) - "which" (unreliable exit status and output; use type instead) - "test a == b" (bashism for "test a = b") "make test-lint-shell-syntax" can be used to run only the check. Helped-By: Jeff King <peff@peff.net> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-01-03 00:20:19 +01:00
PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH))
t/Makefile: fix result handling with TEST_OUTPUT_DIRECTORY When TEST_OUTPUT_DIRECTORY is set, the test results will be generated in "$TEST_OUTPUT_DIRECTORY/test-results", which may not be the same as "test-results" in t/Makefile. This causes the aggregate-results target to fail as it finds no count files. Fix this by introducing TEST_RESULTS_DIRECTORY and using it wherever test-results is referenced. Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-26 20:55:52 +02:00
TEST_RESULTS_DIRECTORY_SQ = $(subst ','\'',$(TEST_RESULTS_DIRECTORY))
Deal with $(bindir) and friends with whitespaces. ... using HPA's shellquote macro. Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-10 22:50:01 +02:00
t/Makefile: Use $(sort ...) explicitly where needed Starting from GNU Make 3.82 $(wildcard ...) no longer sorts the result (from NEWS): * WARNING: Backward-incompatibility! Wildcards were not documented as returning sorted values, but the results have been sorted up until this release.. If your makefiles require sorted results from wildcard expansions, use the $(sort ...) function to request it explicitly. http://repo.or.cz/w/make.git/commitdiff/2a59dc32aaf0681dec569f32a9d7ab88a379d34f I usually watch test progress visually, and if tests are sorted, even with make -j4 they go more or less incrementally by their t number. On the other side, without sorting, tests are executed in seemingly random order even for -j1. Let's please maintain sane tests order for perceived prettyness. Another note is that in GNU Make sort also works as uniq, so after sort being removed, we might expect e.g. $(wildcard *.sh a.*) to produce duplicates for e.g. "a.sh". From this point of view, adding sort could be seen as hardening t/Makefile from accidentally introduced dups. It turned out that prevous releases of GNU Make did not perform full sort in $(wildcard), only sorting results for each pattern, that's why explicit sort-as-uniq is relevant even for older makes. Signed-off-by: Kirill Smelkov <kirr@navytux.spb.ru> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-09-03 22:41:21 +02:00
T = $(sort $(wildcard t[0-9][0-9][0-9][0-9]-*.sh))
TGITWEB = $(sort $(wildcard t95[0-9][0-9]-*.sh))
t/Makefile: check helper scripts for non-portable shell commands too Currently only the "t[0-9][0-9][0-9][0-9]-*.sh" scripts are tested for shell incompatibilities using the check-non-portable-shell.pl script. This makes it easy to miss non-POSIX constructs added to one of the t/*lib*.sh helper scripts, as they aren't automatically detected. Fix that by adding a THELPERS variable containing all shell scripts that aren't tests and add these to the "test-lint-shell-syntax" target too. Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-07-09 21:34:12 +02:00
THELPERS = $(sort $(filter-out $(T),$(wildcard *.sh)))
[PATCH 1/2] Test framework take two. This adds t/ directory to host test suite, a test helper library and a basic set of tests. Petr Baudis raised many valid points at the earlier attempts in git mailing list. This round, test-lib.sh has been updated to a bit more modern style, and the default output is made easier to read. Also included is one sample test script that tests the very basics. This test has already found one leftover bug missed when we introduced symlink support, which has been fixed since then. The supplied Makefile is designed to run all the available tests. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Petr Baudis <pasky@ucw.cz>
2005-05-14 07:50:32 +02:00
test: allow running the tests under "prove" You can run "make DEFAULT_TEST_TARGET=prove test" to run the test under "prove" (or $(PROVE) if set). The output is a bit easier to read when running many tests in parallel. Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net> Liked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Liked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-10-14 10:53:36 +02:00
all: $(DEFAULT_TEST_TARGET)
tests: add some script lint checks There are some common but minor errors we tend to make in writing test scripts: 1. Scripts are left non-executable. This is not usually noticed immediately because "make test" does not need the bit, but it is a matter of git policy to make them executable (and is a slight convenience when running individual scripts). 2. Two scripts are allocated the same number. Usually this happens on separate branches, and the problem only comes about during a merge. But since there is no textual conflict, the merger would have to be very observant to notice. This is also a minor error, but can make GIT_SKIP_TESTS ambiguous. This patch introduces a "test-lint" target which checks both. It is not invoked by default. You can invoke it as "make test-lint", or you can make it a prerequisite of running the tests by specifying "TEST_LINT = test-lint" in your config.mak or on the command line. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-12-13 18:22:38 +01:00
test: pre-clean $(TEST_LINT)
tests: Clarify dependencies between tests, 'aggregate-results' and 'clean' The Makefile targets 'aggregate-results' and 'clean' pretended to be independent. This is not true, of course, since aggregate-results needs the results _before_ they are removed. Likewise, the tests should have been run already when the results are to be aggregated. However, as it is legitimate to run only a few tests, and then aggregate just those results, so another target is introduced, that depends on all tests, then aggregates the results, and only then removes the results. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-08-08 07:59:18 +02:00
$(MAKE) aggregate-results-and-cleanup
make tests ignorable with "make -i" Allow failed tests to be ignored using make's "-i". The patch also disables parallel make in t/. This doesn't make the testing any different as before: the tests were run sequentially before. It also allows to run more tests, ignoring the ones usually failing just to figure out if something else broke. (Or to ignore plainly uninteresting situations because of the testing being done on say... cygwin ;) Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-11-08 10:51:10 +01:00
tests: add some script lint checks There are some common but minor errors we tend to make in writing test scripts: 1. Scripts are left non-executable. This is not usually noticed immediately because "make test" does not need the bit, but it is a matter of git policy to make them executable (and is a slight convenience when running individual scripts). 2. Two scripts are allocated the same number. Usually this happens on separate branches, and the problem only comes about during a merge. But since there is no textual conflict, the merger would have to be very observant to notice. This is also a minor error, but can make GIT_SKIP_TESTS ambiguous. This patch introduces a "test-lint" target which checks both. It is not invoked by default. You can invoke it as "make test-lint", or you can make it a prerequisite of running the tests by specifying "TEST_LINT = test-lint" in your config.mak or on the command line. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-12-13 18:22:38 +01:00
prove: pre-clean $(TEST_LINT)
t/Makefile: stop setting GIT_CONFIG Once upon a time, the setting of GIT_CONFIG in the environment could affect how tests ran. Commit 9c3796f (Fix setting config variables with an alternative GIT_CONFIG, 2006-06-20) unconditionally set GIT_CONFIG in the Makefile when running tests to give us a known starting point. This is insufficient for running the tests outside of the Makefile, however, and 8565d2d (Make tests independent of global config files, 2007-02-15) later set GIT_CONFIG directly in test-lib.sh. At that point the Makefile setting was redundant, but we never removed it. Let's do so now. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-21 00:13:21 +01:00
@echo "*** prove ***"; $(PROVE) --exec '$(SHELL_PATH_SQ)' $(GIT_PROVE_OPTS) $(T) :: $(GIT_TEST_OPTS)
t/Makefile: retain cache t/.prove across prove runs prove(1) can write a summary of its test results and timings into a cache file, t/.prove, then use this information during later runs for various purposes. But deleting t/.prove after every test run defeats this purpose. So do not delete t/.prove as part of "make DEFAILT_TEST_TARGET=prove test". (Continue to delete the file on "make clean".) Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-02 17:31:52 +02:00
$(MAKE) clean-except-prove-cache
test: allow running the tests under "prove" You can run "make DEFAULT_TEST_TARGET=prove test" to run the test under "prove" (or $(PROVE) if set). The output is a bit easier to read when running many tests in parallel. Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net> Liked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Liked-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-10-14 10:53:36 +02:00
make tests ignorable with "make -i" Allow failed tests to be ignored using make's "-i". The patch also disables parallel make in t/. This doesn't make the testing any different as before: the tests were run sequentially before. It also allows to run more tests, ignoring the ones usually failing just to figure out if something else broke. (Or to ignore plainly uninteresting situations because of the testing being done on say... cygwin ;) Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-11-08 10:51:10 +01:00
$(T):
t/Makefile: stop setting GIT_CONFIG Once upon a time, the setting of GIT_CONFIG in the environment could affect how tests ran. Commit 9c3796f (Fix setting config variables with an alternative GIT_CONFIG, 2006-06-20) unconditionally set GIT_CONFIG in the Makefile when running tests to give us a known starting point. This is insufficient for running the tests outside of the Makefile, however, and 8565d2d (Make tests independent of global config files, 2007-02-15) later set GIT_CONFIG directly in test-lib.sh. At that point the Makefile setting was redundant, but we never removed it. Let's do so now. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-21 00:13:21 +01:00
@echo "*** $@ ***"; '$(SHELL_PATH_SQ)' $@ $(GIT_TEST_OPTS)
[PATCH 1/2] Test framework take two. This adds t/ directory to host test suite, a test helper library and a basic set of tests. Petr Baudis raised many valid points at the earlier attempts in git mailing list. This round, test-lib.sh has been updated to a bit more modern style, and the default output is made easier to read. Also included is one sample test script that tests the very basics. This test has already found one leftover bug missed when we introduced symlink support, which has been fixed since then. The supplied Makefile is designed to run all the available tests. Signed-off-by: Junio C Hamano <junkio@cox.net> Signed-off-by: Petr Baudis <pasky@ucw.cz>
2005-05-14 07:50:32 +02:00
Hook up the result aggregation in the test makefile. This patch makes 'make' output the aggregated results at the end of each build. The 'git-test-result' file is removed both before and after each build. Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-08 16:04:35 +02:00
pre-clean:
t/Makefile: fix result handling with TEST_OUTPUT_DIRECTORY When TEST_OUTPUT_DIRECTORY is set, the test results will be generated in "$TEST_OUTPUT_DIRECTORY/test-results", which may not be the same as "test-results" in t/Makefile. This causes the aggregate-results target to fail as it finds no count files. Fix this by introducing TEST_RESULTS_DIRECTORY and using it wherever test-results is referenced. Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-26 20:55:52 +02:00
$(RM) -r '$(TEST_RESULTS_DIRECTORY_SQ)'
Hook up the result aggregation in the test makefile. This patch makes 'make' output the aggregated results at the end of each build. The 'git-test-result' file is removed both before and after each build. Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-08 16:04:35 +02:00
t/Makefile: retain cache t/.prove across prove runs prove(1) can write a summary of its test results and timings into a cache file, t/.prove, then use this information during later runs for various purposes. But deleting t/.prove after every test run defeats this purpose. So do not delete t/.prove as part of "make DEFAILT_TEST_TARGET=prove test". (Continue to delete the file on "make clean".) Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-02 17:31:52 +02:00
clean-except-prove-cache:
t/Makefile: fix result handling with TEST_OUTPUT_DIRECTORY When TEST_OUTPUT_DIRECTORY is set, the test results will be generated in "$TEST_OUTPUT_DIRECTORY/test-results", which may not be the same as "test-results" in t/Makefile. This causes the aggregate-results target to fail as it finds no count files. Fix this by introducing TEST_RESULTS_DIRECTORY and using it wherever test-results is referenced. Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-26 20:55:52 +02:00
$(RM) -r 'trash directory'.* '$(TEST_RESULTS_DIRECTORY_SQ)'
t/Makefile: remove test artifacts upon "make clean" Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-03-13 21:41:20 +01:00
$(RM) -r valgrind/bin
t/Makefile: retain cache t/.prove across prove runs prove(1) can write a summary of its test results and timings into a cache file, t/.prove, then use this information during later runs for various purposes. But deleting t/.prove after every test run defeats this purpose. So do not delete t/.prove as part of "make DEFAILT_TEST_TARGET=prove test". (Continue to delete the file on "make clean".) Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-02 17:31:52 +02:00
clean: clean-except-prove-cache
tests: Ignore the Test::Harness .prove file We document how to run prove with the --state option in t/README. This produces a .prove YAML file in the current directory. Change the t/ gitignore to ignore it, and clean it up on `make clean'. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-24 00:58:44 +02:00
$(RM) .prove
make tests ignorable with "make -i" Allow failed tests to be ignored using make's "-i". The patch also disables parallel make in t/. This doesn't make the testing any different as before: the tests were run sequentially before. It also allows to run more tests, ignoring the ones usually failing just to figure out if something else broke. (Or to ignore plainly uninteresting situations because of the testing being done on say... cygwin ;) Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-11-08 10:51:10 +01:00
t/Makefile: ensure that paths are valid on platforms we care Some pathnames that are okay on ext4 and on HFS+ cannot be checked out on Windows. Tests that want to see operations on such paths on filesystems that support them must do so behind appropriate test prerequisites, and must not include them in the source tree (instead they should create them when they run). Otherwise, the source tree cannot even be checked out. Make sure that double-quotes, asterisk, colon, greater/less-than, question-mark, backslash, tab, vertical-bar, as well as any non-ASCII characters never appear in the pathnames with a new test-lint-* target as part of a `make test`. To that end, we call `git ls-files` (ensuring that the paths are quoted properly), relying on the fact that paths containing non-ASCII characters are quoted within double-quotes. In case that the source code does not actually live in a Git repository (e.g. when extracted from a .zip file), or that the `git` executable cannot be executed, we simply ignore the error for now; In that case, our trusty Continuous Integration will be the last line of defense and catch any problematic file name. Noticed when a topic wanted to add a pathname with '>' in it. A check like this will prevent a similar problems from happening in the future. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-16 17:13:25 +02:00
test-lint: test-lint-duplicates test-lint-executable test-lint-shell-syntax \
test-lint-filenames
tests: add some script lint checks There are some common but minor errors we tend to make in writing test scripts: 1. Scripts are left non-executable. This is not usually noticed immediately because "make test" does not need the bit, but it is a matter of git policy to make them executable (and is a slight convenience when running individual scripts). 2. Two scripts are allocated the same number. Usually this happens on separate branches, and the problem only comes about during a merge. But since there is no textual conflict, the merger would have to be very observant to notice. This is also a minor error, but can make GIT_SKIP_TESTS ambiguous. This patch introduces a "test-lint" target which checks both. It is not invoked by default. You can invoke it as "make test-lint", or you can make it a prerequisite of running the tests by specifying "TEST_LINT = test-lint" in your config.mak or on the command line. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-12-13 18:22:38 +01:00
test-lint-duplicates:
@dups=`echo $(T) | tr ' ' '\n' | sed 's/-.*//' | sort | uniq -d` && \
test -z "$$dups" || { \
echo >&2 "duplicate test numbers:" $$dups; exit 1; }
test-lint-executable:
@bad=`for i in $(T); do test -x "$$i" || echo $$i; done` && \
test -z "$$bad" || { \
echo >&2 "non-executable tests:" $$bad; exit 1; }
test: Add check-non-portable-shell.pl Add the perl script "check-non-portable-shell.pl" to detect non-portable shell syntax. "echo -n" is an example of a shell command working on Linux, but not on Mac OS X. These shell commands are checked and reported as error: - "echo -n" (printf should be used) - "sed -i" (GNUism; use a temp file instead) - "declare" (bashism, often used with arrays) - "which" (unreliable exit status and output; use type instead) - "test a == b" (bashism for "test a = b") "make test-lint-shell-syntax" can be used to run only the check. Helped-By: Jeff King <peff@peff.net> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-01-03 00:20:19 +01:00
test-lint-shell-syntax:
t/Makefile: check helper scripts for non-portable shell commands too Currently only the "t[0-9][0-9][0-9][0-9]-*.sh" scripts are tested for shell incompatibilities using the check-non-portable-shell.pl script. This makes it easy to miss non-POSIX constructs added to one of the t/*lib*.sh helper scripts, as they aren't automatically detected. Fix that by adding a THELPERS variable containing all shell scripts that aren't tests and add these to the "test-lint-shell-syntax" target too. Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-07-09 21:34:12 +02:00
@'$(PERL_PATH_SQ)' check-non-portable-shell.pl $(T) $(THELPERS)
test: Add check-non-portable-shell.pl Add the perl script "check-non-portable-shell.pl" to detect non-portable shell syntax. "echo -n" is an example of a shell command working on Linux, but not on Mac OS X. These shell commands are checked and reported as error: - "echo -n" (printf should be used) - "sed -i" (GNUism; use a temp file instead) - "declare" (bashism, often used with arrays) - "which" (unreliable exit status and output; use type instead) - "test a == b" (bashism for "test a = b") "make test-lint-shell-syntax" can be used to run only the check. Helped-By: Jeff King <peff@peff.net> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-01-03 00:20:19 +01:00
t/Makefile: ensure that paths are valid on platforms we care Some pathnames that are okay on ext4 and on HFS+ cannot be checked out on Windows. Tests that want to see operations on such paths on filesystems that support them must do so behind appropriate test prerequisites, and must not include them in the source tree (instead they should create them when they run). Otherwise, the source tree cannot even be checked out. Make sure that double-quotes, asterisk, colon, greater/less-than, question-mark, backslash, tab, vertical-bar, as well as any non-ASCII characters never appear in the pathnames with a new test-lint-* target as part of a `make test`. To that end, we call `git ls-files` (ensuring that the paths are quoted properly), relying on the fact that paths containing non-ASCII characters are quoted within double-quotes. In case that the source code does not actually live in a Git repository (e.g. when extracted from a .zip file), or that the `git` executable cannot be executed, we simply ignore the error for now; In that case, our trusty Continuous Integration will be the last line of defense and catch any problematic file name. Noticed when a topic wanted to add a pathname with '>' in it. A check like this will prevent a similar problems from happening in the future. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-16 17:13:25 +02:00
test-lint-filenames:
@# We do *not* pass a glob to ls-files but use grep instead, to catch
@# non-ASCII characters (which are quoted within double-quotes)
@bad="$$(git -c core.quotepath=true ls-files 2>/dev/null | \
grep '["*:<>?\\|]')"; \
test -z "$$bad" || { \
echo >&2 "non-portable file name(s): $$bad"; exit 1; }
tests: Clarify dependencies between tests, 'aggregate-results' and 'clean' The Makefile targets 'aggregate-results' and 'clean' pretended to be independent. This is not true, of course, since aggregate-results needs the results _before_ they are removed. Likewise, the tests should have been run already when the results are to be aggregated. However, as it is legitimate to run only a few tests, and then aggregate just those results, so another target is introduced, that depends on all tests, then aggregates the results, and only then removes the results. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-08-08 07:59:18 +02:00
aggregate-results-and-cleanup: $(T)
$(MAKE) aggregate-results
$(MAKE) clean
Hook up the result aggregation in the test makefile. This patch makes 'make' output the aggregated results at the end of each build. The 'git-test-result' file is removed both before and after each build. Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-08 16:04:35 +02:00
aggregate-results:
t/Makefile: fix result handling with TEST_OUTPUT_DIRECTORY When TEST_OUTPUT_DIRECTORY is set, the test results will be generated in "$TEST_OUTPUT_DIRECTORY/test-results", which may not be the same as "test-results" in t/Makefile. This causes the aggregate-results target to fail as it finds no count files. Fix this by introducing TEST_RESULTS_DIRECTORY and using it wherever test-results is referenced. Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-26 20:55:52 +02:00
for f in '$(TEST_RESULTS_DIRECTORY_SQ)'/t*-*.counts; do \
t/aggregate-results: accomodate systems with small max argument list length IRIX 6.5 has a default maximum argument list length of 20480. The file glob that is passed to aggregate-results currently exceeds this length, and so the script cannot run successfully. Work around this issue by passing the file names in via the standard input rather than the argument list. Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-06-02 02:13:44 +02:00
echo "$$f"; \
done | '$(SHELL_PATH_SQ)' ./aggregate-results.sh
Hook up the result aggregation in the test makefile. This patch makes 'make' output the aggregated results at the end of each build. The 'git-test-result' file is removed both before and after each build. Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-08 16:04:35 +02:00
gitweb/Makefile: Add 'test' and 'test-installed' targets The 'test-installed' target in gitweb/Makefile tests installed gitweb, using the same destination directory that 'install' target uses. The 'test' target is just a convenience wrapper invoking 'gitweb-test' target of t/Makefile. Signed-off-by: Jakub Narebski <jnareb@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-09-26 15:02:26 +02:00
gitweb-test:
$(MAKE) $(TGITWEB)
t/Makefile: provide a 'valgrind' target It is easy to forget running valgrinded tests without -v, and it is also easy to forget to redirect the output to "tee" (lest the output scroll out of the terminal's buffer). Running "make valgrind" will take care of all that. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-02-04 00:26:18 +01:00
valgrind:
t/Makefile: pass test opts to valgrind target properly The valgrind target just reinvokes make with GIT_TEST_OPTS set to "--valgrind". However, it does this using an environment variable, which means GIT_TEST_OPTS in your config.mak would override it, and "make valgrind" would simply run the test suite without valgrind on. Instead, we should pass GIT_TEST_OPTS on the command-line, overriding what's in config.mak, and take care to append to whatever the user has there already. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-06-17 10:29:57 +02:00
$(MAKE) GIT_TEST_OPTS="$(GIT_TEST_OPTS) --valgrind"
t/Makefile: provide a 'valgrind' target It is easy to forget running valgrinded tests without -v, and it is also easy to forget to redirect the output to "tee" (lest the output scroll out of the terminal's buffer). Running "make valgrind" will take care of all that. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-02-04 00:26:18 +01:00
Introduce a performance testing framework This introduces a performance testing framework under t/perf/. It tries to be as close to the test-lib.sh infrastructure as possible, and thus should be easy to get used to for git developers. The following points were considered for the implementation: 1. You usually want to compare arbitrary revisions/build trees against each other. They may not have the performance test under consideration, or even the perf-lib.sh infrastructure. To cope with this, the 'run' script lets you specify arbitrary build dirs and revisions. It even automatically builds the revisions if it doesn't have them at hand yet. 2. Usually you would not want to run all tests. It would take too long anyway. The 'run' script lets you specify which tests to run; or you can also do it manually. There is a Makefile for discoverability and 'make clean', but it is not meant for real-world use. 3. Creating test repos from scratch in every test is extremely time-consuming, and shipping or downloading such large/weird repos is out of the question. We leave this decision to the user. Two different sizes of test repos can be configured, and the scripts just copy one or more of those (using hardlinks for the object store). By default it tries to use the build tree's git.git repository. This is fairly fast and versatile. Using a copy instead of a clone preserves many properties that the user may want to test for, such as lots of loose objects, unpacked refs, etc. Signed-off-by: Thomas Rast <trast@student.ethz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-02-17 11:25:09 +01:00
perf:
$(MAKE) -C perf/ all
.PHONY: pre-clean $(T) aggregate-results clean valgrind perf
Copy permalink