Running a hook has to make complex set-up to establish web of
communication between child process and multiplexer, which is common
regardless of what kind of data is fed to the hook. Refactor the parts
that is specific to the data fed to the particular set of hooks from the
part that runs the hook, so that the code can be reused to drive hooks
that take different kind of data.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Recent versions of git can be slow to fetch repositories with a
large number of refs (or when they already have a large
number of refs). For example, GitHub makes pull-requests
available as refs, which can lead to a large number of
available refs. This slowness goes away when submodule
recursion is turned off:
$ git ls-remote git://github.com/rails/rails.git | wc -l
3034
[this takes ~10 seconds of CPU time to complete]
git fetch --recurse-submodules=no \
git://github.com/rails/rails.git "refs/*:refs/*"
[this still isn't done after 10 _minutes_ of pegging the CPU]
git fetch \
git://github.com/rails/rails.git "refs/*:refs/*"
You can produce a quicker and simpler test case like this:
doit() {
head=`git rev-parse HEAD`
for i in `seq 1 $1`; do
echo $head refs/heads/ref$i
done >.git/packed-refs
echo "==> $1"
rm -rf dest
git init -q --bare dest &&
(cd dest && time git.compile fetch -q .. refs/*:refs/*)
}
rm -rf repo
git init -q repo && cd repo &&
>file && git add file && git commit -q -m one
doit 100
doit 200
doit 400
doit 800
doit 1600
doit 3200
Which yields timings like:
# refs seconds of CPU
100 0.06
200 0.24
400 0.95
800 3.39
1600 13.66
3200 54.09
Notice that although the number of refs doubles in each
trial, the CPU time spent quadruples.
The problem is that the submodule recursion code works
something like:
- for each ref we fetch
- for each commit in git rev-list $new_sha1 --not --all
- add modified submodules to list
- fetch any newly referenced submodules
But that means if we fetch N refs, we start N revision
walks. Worse, because we use "--all", the number of refs we
must process that constitute "--all" keeps growing, too. And
you end up doing O(N^2) ref resolutions.
Instead, this patch structures the code like this:
- for each sha1 we already have
- add $old_sha1 to list $old
- for each ref we fetch
- add $new_sha1 to list $new
- for each commit in git rev-list $new --not $old
- add modified submodules to list
- fetch any newly referenced submodules
This yields timings like:
# refs seconds of CPU
100 0.00
200 0.04
400 0.04
800 0.10
1600 0.21
3200 0.39
Note that the amount of effort doubles as the number of refs
doubles. Similarly, the fetch of rails.git takes about as
much time as it does with --recurse-submodules=no.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
commit c9bfb953 (want_color: automatically fallback to color.ui,
2011-08-17) introduced a regression where format-patch produces colorized
patches when color.ui is set to "always".
In f3aafa4 (Disable color detection during format-patch, 2006-07-09),
git_format_config was taught to intercept diff.color to avoid passing it
down to git_log_config and later, git_diff_ui_config.
Teach git_format_config to intercept color.ui in the same way.
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Pang Yan Han <pangyanhan@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jn/remote-helpers-doc:
(short) documentation for the testgit remote helper
Documentation/git-remote-helpers: explain how import works with multiple refs
Documentation/remote-helpers: explain capabilities first
* jk/maint-config-param:
config: use strbuf_split_str instead of a temporary strbuf
strbuf: allow strbuf_split to work on non-strbufs
config: avoid segfault when parsing command-line config
config: die on error in command-line config
fix "git -c" parsing of values with equals signs
strbuf_split: add a max parameter
Removing Cogito leaves just git and StGit, which is a rather
incomplete list of git diff tools available. Sidestep the problem
of deciding what tools to mention by not mentioning any.
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Asking fwrite() to write one item of size bytes results in fwrite()
reporting "I wrote zero item", when size is zero. Instead, we could
ask it to write "size" items of 1 byte and expect it to report that
"I wrote size items" when it succeeds, with any value of size,
including zero.
Noticed and reported by BJ Hargrave.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In particular, sparse issues the following warnings:
compat/obstack.c:176:17: warning: Using plain integer as NULL pointer
compat/obstack.c:224:17: warning: Using plain integer as NULL pointer
compat/obstack.c:324:16: warning: Using plain integer as NULL pointer
compat/obstack.c:329:16: warning: Using plain integer as NULL pointer
compat/obstack.c:347:16: warning: Using plain integer as NULL pointer
compat/obstack.c:362:19: warning: Using plain integer as NULL pointer
compat/obstack.c:379:29: warning: Using plain integer as NULL pointer
compat/obstack.c:399:1: error: symbol 'print_and_abort' redeclared with \
different type (originally declared at compat/obstack.c:95) \
- different modifiers
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Extract the helper function and the type definition of the iterator
function it uses out of builtin/fetch.c into a separate source and a
header file.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We will be using the same "rev-list --verify-objects" logic to add a
sanity check to the receiving end of "git push" in the same way, but the
list of commits that are checked come from a structure with a different
shape over there.
Update the function to take an iterator to make it easier to reuse it in
different contexts.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "git fetch" command works in two phases. The remote side tells us what
objects are at the tip of the refs we are fetching from, and transfers the
objects missing from our side. After storing the objects in our repository,
we update our remote tracking branches to point at the updated tips of the
refs.
A broken or malicious remote side could send a perfectly well-formed pack
data during the object transfer phase, but there is no guarantee that the
given data actually fill the gap between the objects we originally had and
the refs we are updating to.
Although this kind of breakage can be caught by running fsck after a
fetch, it is much cheaper to verify that everything that is reachable from
the tips of the refs we fetched are indeed fully connected to the tips of
our current set of refs before we update them.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It makes no sense to do the - possibly very expensive - call to "rev-list
<new-ref-sha1> --not --all" in check_for_new_submodule_commits() when
there aren't any submodules configured.
Leave check_for_new_submodule_commits() early when no name <-> path
mappings for submodules are found in the configuration. To make that work
reading the configuration had to be moved further up in cmd_fetch(), as
doing that after the actual fetch of the superproject was too late.
Reported-by: Martin Fick <mfick@codeaurora.org>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"branch -v" without other options or parameters still works in the list
mode, but that is not because there is "-v" but because there is no
parameter nor option.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The %(body) placeholder returns the whole body of a tag or
commit, including the signature. However, callers may want
to get just the body without signature, or just the
signature.
Rather than change the meaning of %(body), which might break
some scripts, this patch introduces a new set of
placeholders which break down the %(contents) placeholder
into its constituent parts.
[jk: initial patch by mg, rebased on top of my refactoring
and with tests by me]
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Generally the format of a git tag or commit message is:
subject
body body body
body body body
However, we occasionally see multiline subjects like:
subject
with multiple
lines
body body body
body body body
The rest of git treats these multiline subjects as something
to be concatenated and shown as a single line (e.g., "git
log --pretty=format:%s" will do so since f53bd74). For
consistency, for-each-ref should do the same with its
"%(subject)".
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The find_subpos function was a little hard to use, as well
as to read. It would sometimes write into the subject and
body pointers, and sometimes not. The body pointer sometimes
could be compared to subject, and sometimes not. When
actually duplicating the subject, the caller was forced to
figure out again how long the subject is (which is not too
big a deal when the subject is a single line, but hard to
extend).
The refactoring makes the function more straightforward, both
to read and to use. We will always put something into the
subject and body pointers, and we return explicit lengths
for them, too.
This lays the groundwork both for more complex subject
parsing (e.g., multiline), as well as splitting the body
into subparts (like the text versus the signature).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The current tests don't actually check parsing commit and
tag messages that have both a subject and a body (they just
have single-line messages).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Other test scripts may want to look at or verify signed
tags, and the setup is non-trivial. Let's factor this out
into lib-gpg.sh for other tests to use.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When it needs to compare and add an alt object path to the
alt_odb_list, we normalize this path first since comparing normalized
path is easy to get correct result.
Use strbuf to replace some string operations, since it is cleaner and
safer.
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Hui Wang <Hui.Wang@windriver.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A few more parts of this document is stale that needs updating
to reflect the reality, but I do not regularly rebase topics that
are only in "pu" anymore, which may be noteworthy for a commit.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This reverts commit ffa69e61d3, reversing
changes made to 4a13c4d148.
Adding a new command line option to receive-pack and feed it from
send-pack is not an acceptable way to add features, as there is no
guarantee that your updated send-pack will be talking to updated
receive-pack. New features need to be added via the capability mechanism
negotiated over the protocol.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add tests for the new fetch.fsckobjects, and also tests for
receive.fsckobjects we have had for quite some time.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This single variable can be used to set instead of setting fsckobjects
variable for fetch & receive independently.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This corresponds to receive.fsckobjects configuration variable added (a
lot) earlier in 20dc001 (receive-pack: allow using --strict mode for
unpacking objects, 2008-02-25).
Signed-off-by: Junio C Hamano <gitster@pobox.com>
