The test scripts for "git p4" (but not "git p4" implementation
itself) has been updated so that they would work even on a system
where the installed version of Python is python 3.
* ld/p4-test-py3:
git-p4 tests: time_in_seconds should use $PYTHON_PATH
git-p4 tests: work with python3 as well as python2
git-p4 tests: cd to / before running python
Traditionally cat-file's batch-mode does not do any output
buffering. The reason is that a caller may have pipes
connected to its input and output, and would want to use
cat-file interactively, getting output immediately for each
input it sends.
This may involve a lot of small write() calls, which can be
slow. So we introduced --buffer to improve this, but we
can't turn it on by default, as it would break the
interactive case above.
However, when --batch-all-objects is used, we do not read
stdin at all. We generate the output ourselves as quickly as
possible, and then exit. In this case buffering is a strict
win, and it is simply a hassle for the user to have to
remember to specify --buffer.
This patch makes --buffer the default when --batch-all-objects
is used. Specifying "--buffer" manually is still OK, and you
can even override it with "--no-buffer" if you're a
masochist (or debugging).
For some real numbers, running:
git cat-file --batch-all-objects --batch-check='%(objectname)'
on torvalds/linux goes from:
real 0m1.464s
user 0m1.208s
sys 0m0.252s
to:
real 0m1.230s
user 0m1.172s
sys 0m0.056s
for a 16% speedup.
Suggested-by: Charles Bailey <charles@hashpling.org>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It is not unreasonable to ask cat-file for a batch-check
format of simply "%(objectname)". At first glance this seems
like a noop (you are generally already feeding the object
names on stdin!), but it has a few uses:
1. With --batch-all-objects, you can generate a listing of
the sha1s present in the repository, without any input.
2. You do not have to feed sha1s; you can feed arbitrary
sha1 expressions and have git resolve them en masse.
3. You can even feed a raw sha1, with the result that git
will tell you whether we actually have the object or
not.
In case 3, the call to sha1_object_info is useful; it tells
us whether the object exists or not (technically we could
swap this out for has_sha1_file, but the cost is roughly the
same).
In case 2, the existence check is of debatable value. A
mass-resolution might prefer performance to safety (against
outputting a value for a corrupted ref, for example).
However, the object lookup cost is likely not as noticeable
compared to the resolution cost. And since we have provided
that safety in the past, the conservative choice is to keep
it.
In case 1, though, the object lookup is a definite noop; we
know about the object because we found it in the object
database. There is no new information gained by making the
call.
This patch detects that case and optimizes out the call.
Here are best-of-five timings for linux.git:
[before]
$ time git cat-file --buffer \
--batch-all-objects \
--batch-check='%(objectname)'
real 0m2.117s
user 0m2.044s
sys 0m0.072s
[after]
$ time git cat-file --buffer \
--batch-all-objects \
--batch-check='%(objectname)'
real 0m1.230s
user 0m1.176s
sys 0m0.052s
There are two implementation details to note here.
One is that we detect the noop case by seeing that "struct
object_info" does not request any information. But besides
object existence, there is one other piece of information
which sha1_object_info may fill in: whether the object is
cached, loose, or packed. We don't currently provide that
information in the output, but if we were to do so later,
we'd need to take note and disable the optimization in that
case.
And that leads to the second note. If we were to output
that information, a better implementation would be to
remember where we saw the object in --batch-all-objects in
the first place, and avoid looking it up again by sha1.
In fact, we could probably squeeze out some extra
performance for less-trivial cases, too, by remembering the
pack location where we saw the object, and going directly
there to find its information (like type, size, etc). That
would in theory make this optimization unnecessary.
I didn't pursue that path here for two reasons:
1. It's non-trivial to implement, and has memory
implications. Because we sort and de-dup the list of
output sha1s, we'd have to record the pack information
for each object, too.
2. It doesn't save as much as you might hope. It saves the
find_pack_entry() call, but getting the size and type
for deltified objects requires walking down the delta
chain (for the real type) or reading the delta data
header (for the size). These costs tend to dominate the
non-trivial cases.
By contrast, this optimization is easy and self-contained,
and speeds up a real-world case I've used.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ideally, each test should be responsible for setting up state it needs
rather than relying upon transient global state. Toward this end, teach
test_rev_parse() to accept a "-g <dir>" option to allow callers to
specify the value of the GIT_DIR environment variable explicitly. Take
advantage of this new option to avoid polluting the global scope with
GIT_DIR assignments.
Implementation note: Typically, tests avoid polluting the global state
by wrapping transient environment variable assignments within a
subshell, however, this technique doesn't work here since test_config()
and test_unconfig() need to know GIT_DIR, as well, but neither function
can be used within a subshell. Consequently, GIT_DIR is instead cleared
manually via test_when_finished().
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ideally, each test should be responsible for setting up state it needs
rather than relying upon transient global state. Toward this end, teach
test_rev_parse() to accept a "-b <value>" option to allow callers to set
"core.bare" explicitly or undefine it. Take advantage of this new option
to avoid setting "core.bare" outside of tests.
Under the hood, "-b <value>" invokes "test_config -C <dir>" (or
"test_unconfig -C <dir>"), thus git-config knows explicitly where to
find its configuration file. Consequently, the global GIT_CONFIG
environment variable required by the manual git-config invocations
outside of tests is no longer needed, and is thus dropped.
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ideally, each test should be responsible for setting up state it needs
rather than relying upon transient global state. Toward this end, teach
test_rev_parse() to accept a "-C <dir>" option to allow callers to
instruct it explicitly in which directory its tests should be run. Take
advantage of this new option to avoid changing the working directory
outside of tests.
Implementation note: test_rev_parse() passes "-C <dir>" along to
git-rev-parse with <dir> properly quoted. The natural and POSIX way to
do so is via ${dir:+-C "$dir"}, however, with some older broken shells,
this expression evaluates incorrectly to a single argument ("-C <dir>")
rather than the expected two (-C and "<dir>"). Work around this problem
with the slightly ungainly expression: ${dir:+-C} ${dir:+"$dir"}
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Tests run by test_rev_parse() are nearly identical; each invokes
git-rev-parse with a single option and compares the result against an
expected value. Such duplication makes it onerous to extend the tests
since any change needs to be repeated in each test. Avoid the
duplication by parameterizing the test and driving it via a for-loop.
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Certain lines of the marks file might be corrupted (or the objects
missing due to a garbage collection), but that's no reason to truncate
the file and essentially destroy the rest of it.
Ideally missing objects should not cause a crash, we could just skip
them, but that's another patch.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Documentation for "git merge --verify-signatures" has been updated
to clarify that the signature of only the commit at the tip is
verified. Also the phrasing used for signature and key validity is
adjusted to align with that used by OpenPGP.
* kf/gpg-sig-verification-doc:
Documentation: clarify signature verification
On Windows, .git and optionally any files whose name starts with a
dot are now marked as hidden, with a core.hideDotFiles knob to
customize this behaviour.
* js/windows-dotgit:
mingw: remove unnecessary definition
mingw: introduce the 'core.hideDotFiles' setting
Running tests with '-x' option to trace the individual command
executions is a useful way to debug test scripts, but some tests
that capture the standard error stream and check what the command
said can be broken with the trace output mixed in. When running
our tests under "bash", however, we can redirect the trace output
to another file descriptor to keep the standard error of programs
being tested intact.
* jk/test-send-sh-x-trace-elsewhere:
test-lib: set BASH_XTRACEFD automatically
Update tests for "http.extraHeaders=<header>" to be portable back
to Apache 2.2 (the original depended on <RequireAll/> which is a
more recent feature).
* js/http-custom-headers:
submodule: ensure that -c http.extraheader is heeded
t5551: make the test for extra HTTP headers more robust
tests: adjust the configuration for Apache 2.2
Portability enhancement for "rebase -i" to help platforms whose
shell does not like "for i in <empty>" (which is not POSIX-kosher).
* jk/rebase-interative-eval-fix:
rebase--interactive: avoid empty list in shell for-loop
"git fsck" learned to catch NUL byte in a commit object as
potential error and warn.
* jc/fsck-nul-in-commit:
fsck: detect and warn a commit with embedded NUL
fsck_commit_buffer(): do not special case the last validation
Switching between 'master' and 'next', between which the paths to
test helper binaries have changed, did not update bin-wrappers/*
scripts used in tests, causing false test failures.
* nd/test-helpers:
wrap-for-bin.sh: regenerate bin-wrappers when switching branches
"git rerere" can get confused by conflict markers deliberately left
by the inner merge step, because they are indistinguishable from
the real conflict markers left by the outermost merge which are
what the end user and "rerere" need to look at. This was fixed by
making the conflict markers left by the inner merges a bit longer.
* jc/ll-merge-internal:
t6036: remove pointless test that expects failure
ll-merge: use a longer conflict marker for internal merge
ll-merge: fix typo in comment
The code for warning_errno/die_errno has been refactored and a new
error_errno() reporting helper is introduced.
* nd/error-errno: (41 commits)
wrapper.c: use warning_errno()
vcs-svn: use error_errno()
upload-pack.c: use error_errno()
unpack-trees.c: use error_errno()
transport-helper.c: use error_errno()
sha1_file.c: use {error,die,warning}_errno()
server-info.c: use error_errno()
sequencer.c: use error_errno()
run-command.c: use error_errno()
rerere.c: use error_errno() and warning_errno()
reachable.c: use error_errno()
mailmap.c: use error_errno()
ident.c: use warning_errno()
http.c: use error_errno() and warning_errno()
grep.c: use error_errno()
gpg-interface.c: use error_errno()
fast-import.c: use error_errno()
entry.c: use error_errno()
editor.c: use error_errno()
diff-no-index.c: use error_errno()
...
An earlier addition of "sanitize_submodule_env" with 14111fc4 (git:
submodule honor -c credential.* from command line, 2016-02-29)
turned out to be a convoluted no-op; implement what it wanted to do
correctly, and stop filtering settings given via "git -c var=val".
* jk/submodule-c-credential:
submodule: stop sanitizing config options
submodule: use prepare_submodule_repo_env consistently
submodule--helper: move config-sanitizing to submodule.c
submodule: export sanitized GIT_CONFIG_PARAMETERS
t5550: break submodule config test into multiple sub-tests
t5550: fix typo in $HTTPD_URL
Mark several messages for translation.
* va/i18n-misc-updates:
i18n: unpack-trees: avoid substituting only a verb in sentences
i18n: builtin/pull.c: split strings marked for translation
i18n: builtin/pull.c: mark placeholders for translation
i18n: git-parse-remote.sh: mark strings for translation
i18n: branch: move comment for translators
i18n: branch: unmark string for translation
i18n: builtin/rm.c: remove a comma ',' from string
i18n: unpack-trees: mark strings for translation
i18n: builtin/branch.c: mark option for translation
i18n: index-pack: use plural string instead of normal one
Correct faulty recommendation to use "git submodule deinit ." when
de-initialising all submodules, which would result in a strange
error message in a pathological corner case.
* sb/submodule-deinit-all:
submodule deinit: require '--all' instead of '.' for all submodules
Consolidate description of tilde-expansion that is done to
configuration variables that take pathname to a single place.
* jc/config-pathname-type:
config: describe 'pathname' value type
"http.cookieFile" configuration variable clearly wants a pathname,
but we forgot to treat it as such by e.g. applying tilde expansion.
* bn/http-cookiefile-config:
http: expand http.cookieFile as a path
Documentation: config: improve word ordering for http.cookieFile
A new configuration variable core.hooksPath allows customizing
where the hook directory is.
* ab/hooks:
hooks: allow customizing where the hook directory is
githooks.txt: minor improvements to the grammar & phrasing
githooks.txt: amend dangerous advice about 'update' hook ACL
githooks.txt: improve the intro section
Update of "git submodule" to move pieces of logic to C continues.
* sb/submodule-init:
submodule init: redirect stdout to stderr
submodule--helper update-clone: abort gracefully on missing .gitmodules
submodule init: fail gracefully with a missing .gitmodules file
submodule: port init from shell to C
submodule: port resolve_relative_url from shell to C
The final batch of git-rev-parse tests work against a non-local object
database named repo.git. This is done by renaming .git to repo.git and
pointing GIT_DIR at it, but the name is never restored to .git at the
end of the script, which can be problematic for tests added in the
future. Be more friendly by instead making repo.git a copy of .git.
Furthermore, make it clear that tests in repo.git will be independent
from the results of earlier tests done in .git by initializing repo.git
earlier in the test sequence.
Likewise, bundle remaining preparation (such as directory creation) into
a common setup test consistent with modern practice.
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When files are unmerged they can show up as both unmerged and
modified in the output of `git diff --raw`. This causes
difftool's dir-diff to create filesystem entries for the same
path twice, which fails when it encounters a duplicate path.
Ensure that each worktree path is only processed once.
Add a test to demonstrate the breakage.
Reported-by: Jan Smets <jan@smets.cx>
Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The code always goes into one of the two conditional blocks but make it
clear that not doing so is an error condition by setting $ok to 0.
Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In shell code like:
test -z $foo
test -n $foo
that does not quote its arguments, it's easy to think that
it is actually looking at the contents of $foo in each case.
But if $foo is empty, then "test" does not see any argument
at all! The results are quite subtle.
POSIX specifies that test's behavior depends on the number
of arguments it sees, and if $foo is empty, it sees only
one. The behavior in this case is:
1 argument: Exit true (0) if $1 is not null; otherwise,
exit false.
So in the "-z $foo" case, if $foo is empty, then we check
that "-z" is non-null, and it returns success. Which happens
to match what we expected. But for "-n $foo", if $foo is
empty, we'll see that "-n" is non-null and still return
success. That's the opposite of what we intended!
Furthermore, if $foo contains whitespace, we'll end up with
more than 2 arguments. The results in this case are
generally unspecified (unless the first part of $foo happens
to be a valid binary operator, in which case the results are
specified but certainly not what we intended).
And on top of this, even though "test -z $foo" _should_ work
for the empty case, some older shells (reportedly ksh88)
complain about the missing argument.
So let's make sure we consistently quote our variable
arguments to "test". After this patch, the results of:
git grep 'test -[zn] [^"]'
are empty.
Reported-by: Armin Kunaschik <megabreit@googlemail.com>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The main goal here was to avoid double-quotes for
surrounding the test snippet, since it makes the code hard
to read (and to grep for common problems).
But while we're here, we can fix a few other things:
- use test_path_* helpers, which are more robust and give
better error messages
- only "cd" inside a subshell, which leaves the
environment pristine if further tests are added
- consistently quote shell arguments. These aren't wrong
if we assume find-rev output doesn't have any
whitespace, but it doesn't hurt to be careful.
- replace the old-style 'test x$foo = x' with 'test -z
"$foo"'. Besides the quoting fix, this is the form we
generally use in our test suite.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
One of the test snippets in t9107 is enclosed in double
quotes, but then uses single quotes to surround an
interpolated variable inside the snippet, like:
test_expect_success '...' "
test -n '$head'
"
This happens to work because the variable is interpolated
_before_ the snippet is run, and the result is eval'd. So as
long as the variable does not contain any single quotes, the
two are equivalent. And it doesn't, as we know it is a sha1
from rev-parse above. But this construct is unnecessarily
confusing.
But we can go a step further in cleaning up. The test is
really checking that a particular ref has a value. Rather
than checking if rev-parse produced output, we can just move
rev-parse into the test itself, and rely on the exit code
from --verify. Nobody else cares about the $head variable at
all.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When a test runs a loop, it cannot rely on the usual
&&-chaining to propagate a failure inside the loop; it needs
to break out with a failure signal. However, unless you are
in a subshell, doing so with "exit 1" will exit the entire
test script, not just the test snippet we are in (and cause
the harness to complain that test_done was never reached).
So the fundamental point of this patch is s/exit/return/.
But while we're there, let's fix a number of style and
readability issues:
- snippets in double-quotes need an extra layer of quoting
for their meta-characters; let's avoid that by using
single quotes
- accumulating loop output by appending to a file in each
iteration is brittle, as it can be affected by content
left in the file by earlier tests. Instead, it's better
to redirect stdout for the whole loop, so we know the
output only comes from that loop.
- using "test -z" to check that diff output is empty is
overly verbose; we can just ask diff to use --exit-code.
- we can factor out long lists of refs to make it more
obvious we're using the same ones in each loop
- subshells are unnecessary when ending an &&-chain with
"|| return 1"
- minor style fixups like space-after-redirection, and
"do" and "done" on their own lines
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When running diff commands, a pathspec containing decomposed
unicode code points is not converted to precomposed unicode form
under Mac OS X, but we normalize the paths in the index and the
history to precomposed form on that platform. As a result, the
pathspec would not match and no diff is shown.
Unlike many builtin commands, the "diff" family of commands do
not use parse_options(), which is how other builtin commands
indirectly call precompose_argv() to normalize argv[] into
precomposed form on Mac OSX. Teach these commands to call
precompose_argv() themselves.
Note that precomopose_argv() normalizes not just paths but all
command line arguments, so things like "git diff -G $string"
when $string has the decomposed form would first be normalized
into the precomposed form and would stop hitting the same string
in the decomposed form in the diff output with this change.
It is not a problem per-se, as "log" family of commands already use
parse_options() and call precompose_argv()--we can think of this
change as making the "diff" family of commands behave in a similar
way as the commands in the "log" family.
Signed-off-by: Alexander Rinass <alex@fournova.com>
Helped-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A few tests here use double-quotes around the snippets of
shell code to run the tests. None of these tests wants to do
any interpolation at all, and it just leads to an extra
layer of quoting around all double-quotes and dollar signs
inside the snippet. Let's switch to single quotes, like
most other test scripts.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
These variables were added in 16805d3 (t/t91XX-svn: start
removing use of "git-" from these tests, 2008-09-08) so that
running:
git grep git-
would return fewer hits. At the time, we were transitioning
away from the use of the "dashed" git-foo form.
That transition has been over for years, and grepping for
"git-" in the test suite yields thousands of hits anyway
(all presumably false positives).
With their original purpose gone, these variables serve only
to obfuscate the tests. Let's get rid of them.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
