Handle memory pressure and file descriptor pressure separately when
deciding to release pack windows to honor resource limits.
* bc/unuse-packfile:
Don't close pack fd when free'ing pack windows
sha1_file: introduce close_one_pack() to close packs on fd pressure
* da/darwin:
OS X: Fix redeclaration of die warning
Makefile: Fix APPLE_COMMON_CRYPTO with BLK_SHA1
imap-send: use Apple's Security framework for base64 encoding
* jk/fast-import-empty-ls:
fast-import: allow moving the root tree
fast-import: allow ls or filecopy of the root tree
fast-import: set valid mode on root tree in "ls"
t9300: document fast-import empty path issues
Subversion 1.8.0 that was recently released breaks older subversion
clients coming over http/https in various ways.
* km/svn-1.8-serf-only:
Git.pm: revert _temp_cache use of temp_is_locked
git-svn: allow git-svn fetching to work using serf
Git.pm: add new temp_is_locked function
"git check-ignore -z" applied the NUL termination to both its input
(with --stdin) and its output, but "git check-attr -z" ignored the
option on the output side.
This is potentially a backward incompatible fix. Let's see if
anybody screams before deciding if we want to do anything to help
existing users (there may be none).
* jc/check-x-z:
check-attr -z: a single -z should apply to both input and output
check-ignore -z: a single -z should apply to both input and output
check-attr: the name of the character is NUL, not NULL
check-ignore: the name of the character is NUL, not NULL
Add 'struct ref_update' to encode the information needed to update or
delete a ref (name, new sha1, optional old sha1, no-deref flag). Add
function 'update_refs' accepting an array of updates to perform. First
sort the input array to order locks consistently everywhere and reject
multiple updates to the same ref. Then acquire locks on all refs with
verified old values. Then update or delete all refs accordingly. Fail
if any one lock cannot be obtained or any one old value does not match.
Though the refs themselves cannot be modified together in a single
atomic transaction, this function does enable some useful semantics.
For example, a caller may create a new branch starting from the head of
another branch and rewind the original branch at the same time. This
transfers ownership of commits between branches without risk of losing
commits added to the original branch by a concurrent process, or risk of
a concurrent process creating the new branch first.
Signed-off-by: Brad King <brad.king@kitware.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Generalize repack_without_ref as repack_without_refs to support a list
of refs and implement the former in terms of the latter.
Signed-off-by: Brad King <brad.king@kitware.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Factor loose ref deletion into helper function delete_ref_loose to allow
later use elsewhere.
Signed-off-by: Brad King <brad.king@kitware.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Factor the lock and write steps and error handling into helper functions
update_ref_lock and update_ref_write to allow later use elsewhere.
Expose lock_any_ref_for_update's type_p to update_ref_lock callers.
While at it, drop "static" from the local "lock" variable as it is not
necessary to keep across invocations.
Signed-off-by: Brad King <brad.king@kitware.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 68be2fea (receive-pack, fetch-pack: reject bogus pack that
records objects twice, 2011-11-16) taught index-pack to notice and
reject duplicate objects if --strict is given (which it is for
incoming packs, if transfer.fsckObjects is set). However, it never
tested the code, because we did not have an easy way of generating
such a bogus pack.
Now that we have test infrastructure to handle this, let's confirm
that it works.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Back in 21e9757e (Hack git-add--interactive to make it work with
ActiveState Perl, 2007-08-01), the invocation of external commands was
changed to use qx{} on Windows. The rationale was that the command
interpreter on Windows is not a POSIX shell, but rather Windows's CMD.
That patch was wrong to include 'msys' in the check whether to use qx{}
or not: 'msys' identifies MSYS perl as shipped with Git for Windows,
which does not need the special treatment; qx{} should be used only with
ActiveState perl, which is identified by 'MSWin32'.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git p4" does not support many of the view wildcards, such as * and
%%n. It only knows the common ... mapping, and exclusions.
Redo the entire wildcard code around the idea of directly querying
the p4 server for the mapping. For each commit, invoke "p4 where"
with committed file paths as args and use the client mapping to
decide where the file goes in git.
This simplifies a lot of code, and adds support for all wildcards
supported by p4. Downside is that there is probably a 20%-ish
slowdown with this approach.
[pw: redo code and tests]
Signed-off-by: Kazuki Saitoh <ksaitoh560@gmail.com>
Signed-off-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we are parsing "rev^{foo}", we check "foo" against the
various global type strings, like "commit_type",
"tree_type", etc. This is nicely abstracted, but then we
destroy the abstraction completely by using magic numbers
that must match the length of the type strings.
We could avoid these magic numbers by using skip_prefix. But
taking a step back, we can realize that using the
"commit_type" global is not really buying us anything. It is
not ever going to change from being "commit" without causing
severe breakage to existing uses. And even if it did change
for some crazy reason, we would want to evaluate its effects
on the "rev^{}" syntax, anyway.
Let's just switch these to using a custom string literal, as
we do for "rev^{object}". The resulting code is more robust
to changes in the type strings, and is more readable.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Complete the <rev>^{<type>} family of object descriptors by having
<rev>^{tag} dereference <rev> until a tag object is found (or fail if
unable).
At first glance this may not seem very useful, as commits, trees, and
blobs cannot be peeled to a tag, and a tag would just peel to itself.
However, this can be used to ensure that <rev> names a tag object:
$ git rev-parse --verify v1.8.4^{tag}
04f013dc38
$ git rev-parse --verify master^{tag}
error: master^{tag}: expected tag type, but the object dereferences to tree type
fatal: Needed a single revision
Users can already ensure that <rev> is a tag object by checking the
output of 'git cat-file -t <rev>', but:
* users may expect <rev>^{tag} to exist given that <rev>^{commit},
<rev>^{tree}, and <rev>^{blob} all exist
* this syntax is more convenient/natural in some circumstances
Signed-off-by: Richard Hansen <rhansen@bbn.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Save the reader from learning specialized t6* setup functions
where familiar commands like test_commit, "git checkout --orphan",
and "git merge" will do.
While at it, wrap the setup commands in a test assertion so errors can
be caught and stray output suppressed when running without --verbose
as in other tests.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use test_cmp instead of passing two command substitutions to the
"test" builtin. This way:
- when tests fail, they can print a helpful diff if run with
"--verbose"
- the argument order "test_cmp expect actual" feels natural,
unlike test <known> = <unknown> that seems backwards
- the exit status from invoking git is checked, so if rev-parse
starts segfaulting then the test will notice and fail
Use a custom function for this instead of test_cmp_rev to emphasize
that we are testing the output from "git rev-parse" with certain
arguments, not checking that the revisions are equal in abstract.
Reported-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This way, if rev-parse segfaults then the test will fail instead
of treating it the same way as a controlled failure.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of cramming everything in one line, put the test body in an
indented block after the opening test_expect_success line and quote
and put the closing quote on a line by itself.
Use single-quote instead of double-quote to quote the test body
for more useful --verbose output.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
cmd_summary reads the output of git diff, but reads in the submodule path into a
variable called name. Since this variable does not contain the name of the
submodule, but the path, rename it to be clearer what data it actually holds.
Signed-off-by: Brian M. Carlson <sandals@crustytoothpaste.net>
Acked-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Split into a separate helper function get_commit() so that the part that
finds the relevant commit, and the part that does something with it
(handle tag object, etc.) are in different places.
No functional changes.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There's no need to pass it around everywhere. This would make easier
further refactoring that makes use of this variable.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We used to update the private ref ourselves, but this update is now
done by default since 664059fb (transport-helper: update remote
helper namespace, 2013-04-17).
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since 664059fb (transport-helper: update remote helper namespace,
2013-04-17), a 'push' operation on a remote helper updates the
private ref by default. This is often a good thing, but it can also
be desirable to disable this update to force the next 'pull' to
re-import the pushed revisions.
Allow remote-helpers to disable the automatic update by introducing a new
capability.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is useful to make sure we don't step outside the boundaries of what
we are interpreting at the moment. For example while interpreting
foobar@{u}~1, the job of interpret_branch_name() ends right before ~1,
but there's no way to figure that out inside the function, unless the
len argument is passed.
So let's do that.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This makes reinitializing on a .git file repository work.
This is probably the only case that setup_git_env() (via
set_git_dir()) is called on a .git file. Other cases in
setup_git_dir_gently() and enter_repo() both cover .git file case
explicitly because they need to verify the target repo is valid.
Reported-by: Ximin Luo <infinity0@gmx.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If the user specifies a pager of "cat" (or the empty
string), whether it is in the environment or from config, we
automagically optimize it out to mean "no pager" and avoid
forking at all. We treat an empty pager variable similary.
However, we did not apply this optimization when
DEFAULT_PAGER was set to "cat" (or the empty string). There
is no reason to treat DEFAULT_PAGER any differently. The
optimization should not be user-visible (unless the user has
a bizarre "cat" in their PATH). And even if it is, we are
better off behaving consistently between the compile-time
default and the environment and config settings.
The stray "else" we are removing from this code was
introduced by 402461a (pager: do not fork a pager if PAGER
is set to empty., 2006-04-16). At that time, the line
directly above used:
if (!pager)
pager = "less";
as a fallback, meaning that it could not possibly trigger
the optimization. Later, a3d023d (Provide a build time
default-pager setting, 2009-10-30) turned that constant into
a build-time setting which could be anything, but didn't
loosen the "else" to let DEFAULT_PAGER use the optimization.
Noticed-by: Dale R. Worley <worley@alum.mit.edu>
Suggested-by: Matthieu Moy <Matthieu.Moy@grenoble-inp.fr>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Even though it is not required by POSIX to double-quote the
redirection target in a variable, our code does so because some
versions of bash issue a warning without the quotes.
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
During the review of the main series it was noticed that these test
scripts can use updates to conform to our coding style better, but
fixing the style should be done in a patch separate from the main
series.
This updates the test-*.sh scripts only for style issues:
* We do not leave SP between a redirection operator and the
filename;
* We change line before "then", "do", etc. rather than terminating
the condition for "if"/"while" and list for "for" with a
semicolon;
* When HERE document does not use any expansion, we quote the end
marker (e.g. "cat <<\EOF" not "cat <<EOF") to signal the readers
that there is no funny substitution to worry about when reading
the code.
* We use "test" rather than "[".
Reviewed-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After commit 3fde386 (reset [--mixed]: use diff-based reset whether or
not pathspec was given), some code can be moved to the 'reset_type ==
MIXED' check.
Let's move the code that is specific to MIXED.
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Expose lock_ref_sha1_basic's type_p argument to callers of
lock_any_ref_for_update. Update all call sites to ignore it by passing
NULL for now.
Signed-off-by: Brad King <brad.king@kitware.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function resets refs rather than doing arbitrary updates.
Rename it to allow a future general-purpose update_refs function
to be added.
Signed-off-by: Brad King <brad.king@kitware.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the @author_initials feature Jakub added in
v1.6.4-rc2-14-ga36817b to match non-ASCII author initials as intended.
The regexp Jakub added was intended to match
non-ASCII (/\b([[:upper:]])\B/g). But in Perl this doesn't actually
match non-ASCII upper-case characters unless the string being matched
against has the UTF8 flag.
So when we open a pipe to "git blame" we need to mark the file
descriptor we're opening as utf8 explicitly.
So as a result it abbreviates me to "AB" not "ÆAB", entirely because "Æ"
isn't /[[:upper:]]/ unless the string being matched against has the UTF8
flag.
Here's something that demonstrates the issue:
#!/usr/bin/env perl
use strict;
use warnings;
binmode STDOUT, ':utf8' if $ENV{UTF8};
open my $fd, "-|", "git", "blame", "--incremental", "--", "Makefile" or die "Can't open: $!";
binmode $fd, ":utf8" if $ENV{UTF8};
while (my $line = <$fd>) {
next unless my ($author) = $line =~ /^author (.*)/;
my @author_initials = ($author =~ /\b([[:upper:]])\B/g);
printf "%s (%s)\n", join("", @author_initials), $author;
}
When that's run with and without UTF8 being true in the environment it
gives, on git.git:
$ UTF8=0 perl author-initials.pl | sort | uniq -c |
sort -nr | head -n 5
99 JH (Junio C Hamano)
35 JN (Jonathan Nieder)
35 JK (Jeff King)
20 JS (Johannes Schindelin)
16 AB (Ævar Arnfjörð Bjarmason)
$ UTF8=1 perl author-initials.pl | sort | uniq -c |
sort -nr | head -n 5
99 JH (Junio C Hamano)
35 JN (Jonathan Nieder)
35 JK (Jeff King)
20 JS (Johannes Schindelin)
16 ÆAB (Ævar Arnfjörð Bjarmason)
Acked-by: Jakub Narębski <jnareb@gmail.com>
Tested-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Tested-by: Simon Ruderich <simon@ruderich.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we read a sha1 file, we first look for a packed
version, then a loose version, and then re-check the pack
directory again before concluding that we cannot find it.
This lets us handle a process that is writing to the
repository simultaneously (e.g., receive-pack writing a new
pack followed by a ref update, or git-repack packing
existing loose objects into a new pack).
However, we do not do the same trick with has_sha1_file; we
only check the packed objects once, followed by loose
objects. This means that we might incorrectly report that we
do not have an object, even though we could find it if we
simply re-checked the pack directory.
By itself, this is usually not a big deal. The other process
is running simultaneously, so we may run has_sha1_file
before it writes, anyway. It is a race whether we see the
object or not. However, we may also see other things
the writing process has done (like updating refs); and in
that case, we must be able to also see the new objects.
For example, imagine we are doing a for_each_ref iteration,
and somebody simultaneously pushes. Receive-pack may write
the pack and update a ref after we have examined the
objects/pack directory, but before the iteration gets to the
updated ref. When we do finally see the updated ref,
for_each_ref will call has_sha1_file to check whether the
ref is broken. If has_sha1_file returns the wrong answer, we
erroneously will think that the ref is broken.
For a normal iteration without DO_FOR_EACH_INCLUDE_BROKEN,
this means that the caller does not see the ref at all
(neither the old nor the new value). So not only will we
fail to see the new value of the ref (which is acceptable,
since we are running simultaneously with the writer, and we
might well read the ref before the writer commits its
write), but we will not see the old value either. For
programs that act on reachability like pack-objects or
prune, this can cause data loss, as we may see the objects
referenced by the original ref value as dangling (and either
omit them from the pack, or delete them via prune).
There's no test included here, because the success case is
two processes running simultaneously forever. But you can
replicate the issue with:
# base.sh
# run this in one terminal; it creates and pushes
# repeatedly to a repository
git init parent &&
(cd parent &&
# create a base commit that will trigger us looking at
# the objects/pack directory before we hit the updated ref
echo content >file &&
git add file &&
git commit -m base &&
# set the unpack limit abnormally low, which
# lets us simulate full-size pushes using tiny ones
git config receive.unpackLimit 1
) &&
git clone parent child &&
cd child &&
n=0 &&
while true; do
echo $n >file && git add file && git commit -m $n &&
git push origin HEAD:refs/remotes/child/master &&
n=$(($n + 1))
done
# fsck.sh
# now run this simultaneously in another terminal; it
# repeatedly fscks, looking for us to consider the
# newly-pushed ref broken. We cannot use for-each-ref
# here, as it uses DO_FOR_EACH_INCLUDE_BROKEN, which
# skips the has_sha1_file check (and if it wants
# more information on the object, it will actually read
# the object, which does the proper two-step lookup)
cd parent &&
while true; do
broken=`git fsck 2>&1 | grep remotes/child`
if test -n "$broken"; then
echo $broken
exit 1
fi
done
Without this patch, the fsck loop fails within a few seconds
(and almost instantly if the test repository actually has a
large number of refs). With it, the two can run
indefinitely.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Keep track of Mercurial revisions as Git notes under the 'refs/notes/hg'
ref. This way, the user can easily see which Mercurial revision
corresponds to certain Git commit.
Unfortunately, there's no way to efficiently update the notes after
doing an export (push), so they'll have to be updated when importing
(fetching).
Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It is tentatively called 1.8.5, but it should be an easy matter of
renaming the release-notes file and RelNotes symlink to later call
it 1.9 near the end of the cycle if we wanted to.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some people still use rather old versions of bash, which cannot
grok some constructs like 'printf -v varname' the prompt and
completion code started to use recently.
* bc/completion-for-bash-3.0:
contrib/git-prompt.sh: handle missing 'printf -v' more gracefully
t9902-completion.sh: old Bash still does not support array+=('') notation
git-completion.bash: use correct Bash/Zsh array length syntax
