* sb/atomic-push:
Document receive.advertiseatomic
t5543-atomic-push.sh: add basic tests for atomic pushes
push.c: add an --atomic argument
send-pack.c: add --atomic command line argument
send-pack: rename ref_update_to_be_sent to check_to_send_update
receive-pack.c: negotiate atomic push support
receive-pack.c: add execute_commands_atomic function
receive-pack.c: move transaction handling in a central place
receive-pack.c: move iterating over all commands outside execute_commands
receive-pack.c: die instead of error in case of possible future bug
receive-pack.c: shorten the execute_commands loop over all commands
* mh/reflog-expire: (24 commits)
refs.c: let fprintf handle the formatting
refs.c: don't expose the internal struct ref_lock in the header file
lock_any_ref_for_update(): inline function
refs.c: remove unlock_ref/close_ref/commit_ref from the refs api
reflog_expire(): new function in the reference API
expire_reflog(): treat the policy callback data as opaque
Move newlog and last_kept_sha1 to "struct expire_reflog_cb"
expire_reflog(): move rewrite to flags argument
expire_reflog(): move verbose to flags argument
expire_reflog(): pass flags through to expire_reflog_ent()
struct expire_reflog_cb: a new callback data type
Rename expire_reflog_cb to expire_reflog_policy_cb
expire_reflog(): move updateref to flags argument
expire_reflog(): move dry_run to flags argument
expire_reflog(): add a "flags" argument
expire_reflog(): extract two policy-related functions
Extract function should_expire_reflog_ent()
expire_reflog(): use a lock_file for rewriting the reflog file
expire_reflog(): return early if the reference has no reflog
expire_reflog(): rename "ref" parameter to "refname"
...
"git remote add $name $URL" is now allowed when "url.$URL.insteadOf"
is already defined.
* js/remote-add-with-insteadof:
Add a regression test for 'git remote add <existing> <same-url>'
git remote: allow adding remotes agreeing with url.<...>.insteadOf
In cat_one_file(), "type" and "size" variables are defined in the
function scope, and then two variables of the same name are defined
in a block in one of the if/else statement, hiding the definitions
in the outer scope.
Because the values of the outer variables before the control enters
this scope, however, do not have to be preserved, we can remove
useless definitions of variables from the inner scope safely without
breaking anything.
Signed-off-by: Alexander Kuleshov <kuleshovmail@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The code handling %(upstream:track) and %(upstream:trackshort)
assumed that it always had a valid branch that had been sanitized
earlier in populate_value(), and thus did not check the return value
of the call to stat_tracking_info().
While there is indeed some sanitization code that basically
corresponds to stat_tracking_info() returning 0 (no base branch
set), the function can also return -1 when the base branch did exist
but has since then been deleted.
In this case, num_ours and num_theirs had undefined values and a
call to `git for-each-ref --format="%(upstream:track)"` could print
spurious values such as
[behind -111794512]
[ahead 38881640, behind 5103867]
even for repositories with one single commit.
Verify stat_tracking_info()'s return value and do not print anything
if it returns -1. This behavior also matches the documentation ("has
no effect if the ref does not have tracking information associated
with it").
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Raphael Kubo da Costa <raphael.kubo.da.costa@intel.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Earlier we made "rev-list --object-edge" more aggressively list the
objects at the edge commits, in order to reduce number of objects
fetched into a shallow repository, but the change affected cases
other than "fetching into a shallow repository" and made it
unusably slow (e.g. fetching into a normal repository should not
have to suffer the overhead from extra processing). Limit it to a
more specific case by introducing --objects-edge-aggressive, a new
option to rev-list.
* bc/fetch-thin-less-aggressive-in-normal-repository:
pack-objects: use --objects-edge-aggressive for shallow repos
rev-list: add an option to mark fewer edges as uninteresting
Documentation: add missing article in rev-list-options.txt
"git checkout-index --temp=$target $path" did not work correctly
for paths outside the current subdirectory in the project.
* es/checkout-index-temp:
checkout-index: fix --temp relative path mangling
t2004: demonstrate broken relative path printing
t2004: standardize file naming in symlink test
t2004: drop unnecessary write-tree/read-tree
t2004: modernize style
- "exec_cmd.h" became unnecessary at b931aa5a (Call builtin ls-tree
in git-cat-file -p, 2006-05-26), when it changed an earlier code
that delegated tree display to "ls-tree" via the run_command()
API (hence needing "exec_cmd.h") to call cmd_ls_tree() directly.
We should have removed the include in the same commit, but we
forgot to do so.
- "diff.h" was added at e5fba602 (textconv: support for cat_file,
2010-06-15), together with "userdiff.h", but "userdiff.h" can be
included without including "diff.h"; the header was unnecessary
from the beginning.
- "tag.h" and "tree.h" were necessary since 8e440259 (Use blob_,
commit_, tag_, and tree_type throughout., 2006-04-02) to check
the type of object by comparing typename with tree_type and
tag_type (pointers to extern strings).
21666f1a (convert object type handling from a string to a number,
2007-02-26) made these <type>_type strings unnecessary, and it
could have switched to include "object.h", which is necessary to
use typename(), but it forgot to do so. Because "tag.h" and
"tree.h" include "object.h", it did not need to explicitly
include "object.h" in order to start using typename() itself.
We do not even have to include "object.h" after removing these
two #includes, because "builtin.h" includes "commit.h" which in
turn includes "object.h" these days. This happened at 7b9c0a69
(git-commit-tree: make it usable from other builtins,
2008-07-01).
Signed-off-by: Alexander Kuleshov <kuleshovmail@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a command line argument to the git push command to request atomic
pushes.
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This adds support to send-pack to negotiate and use atomic pushes
iff the server supports it. Atomic pushes are activated by a new command
line flag --atomic.
In order to do this we also need to change the semantics for send_pack()
slightly. The existing send_pack() function actually doesn't send all the
refs back to the server when multiple refs are involved, for example
when using --all. Several of the failure modes for pushes can already be
detected locally in the send_pack client based on the information from the
initial server side list of all the refs as generated by receive-pack.
Any such refs that we thus know would fail to push are thus pruned from
the list of refs we send to the server to update.
For atomic pushes, we have to deal thus with both failures that are detected
locally as well as failures that are reported back from the server. In order
to do so we treat all local failures as push failures too.
We introduce a new status code REF_STATUS_ATOMIC_PUSH_FAILED so we can
flag all refs that we would normally have tried to push to the server
but we did not due to local failures. This is to improve the error message
back to the end user to flag that "these refs failed to update since the
atomic push operation failed."
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This adds the atomic protocol option to allow
receive-pack to inform the client that it has
atomic push capability.
This commit makes the functionality introduced
in the previous commits go live for the serving
side. The changes in documentation reflect the
protocol capabilities of the server.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This introduces the new function execute_commands_atomic which will use
one atomic transaction for all updates. The default behavior is still
the old non atomic way, one ref at a time. This is to cause as little
disruption as possible to existing clients. It is unknown if there are
client scripts that depend on the old non-atomic behavior so we make it
opt-in for now.
A later patch will add the possibility to actually use the functionality
added by this patch. For now use_atomic is always 0.
Inspired-by: Ronnie Sahlberg <sahlberg@google.com>
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This moves all code related to transactions into the
execute_commands_non_atomic function. This includes
beginning and committing the transaction as well as
dealing with the errors which may occur during the
begin and commit phase of a transaction.
No functional changes intended.
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This commit allows us in a later patch to easily distinguish between
the non atomic way to update the received refs and the atomic way which
is introduced in a later patch.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Discussion on the previous patch revealed we rather want to err on the
safe side. To do so we need to stop receive-pack in case of the possible
future bug when connectivity is not checked on a shallow push.
Also while touching that code we considered that removing the reported
refs may be harmful in some situations. Sound the message more like a
"This Cannot Happen, Please Investigate!" instead of giving advice to
remove refs.
Suggested-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make the main "execute_commands" loop in receive-pack easier to read
by splitting out some steps into helper functions. The new helper
'should_process_cmd' checks if a ref update is unnecessary, whether
due to an error having occurred or for another reason. The helper
'warn_if_skipped_connectivity_check' warns if we have forgotten to
run a connectivity check on a ref which is shallow for the client
which would be a bug.
This will help us to duplicate less code in a later patch when we make
a second copy of the "execute_commands" loop.
No functional change intended.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The get_merge_bases*() API was easy to misuse by careless
copy&paste coders, leaving object flags tainted in the commits that
needed to be traversed.
* jc/merge-bases:
get_merge_bases(): always clean-up object flags
bisect: clean flags after checking merge bases
The parse_options API expects an array of alternative usage lines
to which it automatically ads the language-appropriate "or" when
displaying. Each of these options is marked for translation with N_
and then later translated when gettext is called on each element
of the array.
Since the N_ macro just expands to its argument, if two N_-marked
strings appear next to each other without being separated by anything
else such as a comma, the preprocessor will join them into one string.
In that case two separate strings get marked for translation, but at
runtime they have been joined into a single string passed to gettext
which then fails to get translated because the combined string was
never marked for translation.
Fix this by properly separating the two N_ marked strings with
a comma and removing the embedded "\n" and " or:" that are
properly supplied by the parse_options API.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
checkout-index --temp only properly prints relative paths which are
descendants of the current directory. Paths in ancestor or sibling
directories (or their children) are often printed in mangled form. For
example:
mkdir a bbb &&
>file &&
>bbb/file &&
git update-index --add file bbb/file &&
cd a &&
git checkout-index --temp ../file ../bbb/file
prints:
.merge_file_ooblek le
.merge_file_igloo0 b/file
rather than the correct:
.merge_file_ooblek ../file
.merge_file_igloo0 ../bbb/file
Internally, given the above example, checkout-index prefixes each input
argument with the name of the current directory ("a/", in this case),
and then assumes that it can simply skip forward by strlen("a/") bytes
to recover the original name. This works for files in the current
directory or its descendants, but fails for files in ancestors or
siblings (or their children) due to path normalization.
For instance, given "../file", "a/" is prepended, giving "a/../file".
Path normalization folds out "a/../", resulting in "file". Attempting
to recover the original name by skipping strlen("a/") bytes gives the
incorrect "le" rather than the desired "../file".
Fix this by taking advantage of write_name_quoted_relative() to recover
the original name properly, rather than assuming that it can be
recovered by skipping strlen(prefix) bytes.
As a bonus, this also fixes a bug in which checkout-index --temp
accessed and printed memory beyond the end-of-string. For instance,
within a subdirectory named "subdirectory", and given argument
"../file", prefixing would give "subdirectory/../file", which would
become "file" after normalization. checkout-index would then attempt to
recover the original name by skipping strlen("subdirectory/") bytes of
"file", which placed it well beyond end-of-string. Despite this error,
it often appeared to give the correct result, but only due to an
accident of implementation which left an apparently correct copy of the
path in memory following the normalized value. In particular, handed
"subdirectory/../file", in-place processing by normalize_path_copy_len()
resulted in "file\0rectory/../file". When checkout-index skipped
strlen("subdirectory/") bytes, it ended up back at "../file" and thus
appeared to give the correct answer, despite being past end-of-string.
Reported-by: Russ Cox <rsc@golang.org>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When fetching into or pushing from a shallow repository, we want to
aggressively mark edges as uninteresting, since this decreases the pack
size. However, aggressively marking edges can negatively affect
performance on large non-shallow repositories with lots of refs.
Teach pack-objects a --shallow option to indicate that we're pushing
from or fetching into a shallow repository. Use
--objects-edge-aggressive only for shallow repositories and otherwise
use --objects-edge, which performs better in the general case. Update
the callers to pass the --shallow option when they are dealing with a
shallow repository.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In commit fbd4a70 (list-objects: mark more commits as edges in
mark_edges_uninteresting - 2013-08-16), we marked an increasing number
of edges uninteresting. This change, and the subsequent change to make
this conditional on --objects-edge, are used by --thin to make much
smaller packs for shallow clones.
Unfortunately, they cause a significant performance regression when
pushing non-shallow clones with lots of refs (23.322 seconds vs.
4.785 seconds with 22400 refs). Add an option to git rev-list,
--objects-edge-aggressive, that preserves this more aggressive behavior,
while leaving --objects-edge to provide more performant behavior.
Preserve the current behavior for the moment by using the aggressive
option.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git update-ref --stdin"'s verify command did not work well when
<oldvalue>, which is documented as optional, was missing.
* mh/update-ref-verify:
update-ref: fix "verify" command with missing <oldvalue>
t1400: add some more tests of "update-ref --stdin"'s verify command
When adding a remote, we make sure that the remote does not exist
already. However, this test was not quite correct: when the
url.<...>.insteadOf config variable was set to the remote name to be
added, the code would assume that the remote exists already.
Let's allow adding remotes when there is a url.<...>.insteadOf setting
when both the name and the URL agree with the remote to be added.
It might seem like a mistake to compare against remote->url[0] without
verifying that remote->url_nr >=1, but at this point a missing URL has
been filled by the name already, therefore url_nr cannot be zero.
Noticed by Anastas Dancha.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
New tag object format validation added in 2.2 showed garbage
after a tagname it reported in its error message.
* js/fsck-tag-validation:
index-pack: terminate object buffers with NUL
fsck: properly bound "invalid tag name" error message
"git branch -d" (delete) and "git branch -m" (move) learned to
honor "-f" (force) flag; unlike many other subcommands, the way to
force these have been with separate "-D/-M" options, which was
inconsistent.
* mg/branch-d-m-f:
branch: allow -f with -m and -d
t3200-branch: test -M
"git ls-tree" does not support path selection based on negative
pathspecs, but did not error out when negative pathspecs are given.
* nd/ls-tree-pathspec:
t3102: style modernization
t3102: document that ls-tree does not yet support negated pathspec
ls-tree: disable negative pathspec because it's not supported
ls-tree: remove path filtering logic in show_tree
tree.c: update read_tree_recursive callback to pass strbuf as base
"git push" into a repository with a working tree normally refuses
to modify the branch that is checked out. The command learned to
optionally do an equivalent of "git reset --hard" only when there
is no change to the working tree and the index instead, which would
be useful to "deploy" by pushing into a repository.
* js/push-to-deploy:
t5516: more tests for receive.denyCurrentBranch=updateInstead
receive-pack: add another option for receive.denyCurrentBranch
The function sometimes returned a non-freeable memory and some
other times returned a piece of memory that must be freed.
* jc/exec-cmd-system-path-leak-fix:
system_path(): always return free'able memory to the caller
"git am" learned "--message-id" option to copy the message ID of
the incoming e-mail to the log message of resulting commit.
* pb/am-message-id-footer:
git-am: add --message-id/--no-message-id
git-mailinfo: add --message-id
"git remote update --prune" to drop many refs has been optimized.
* mh/simplify-repack-without-refs:
sort_string_list(): rename to string_list_sort()
prune_remote(): iterate using for_each_string_list_item()
prune_remote(): rename local variable
repack_without_refs(): make the refnames argument a string_list
prune_remote(): sort delete_refs_list references en masse
prune_remote(): initialize both delete_refs lists in a single loop
prune_remote(): exit early if there are no stale references
Some filesystems assign filemodes in a strange way, fooling then
automatic "filemode trustability" check done during a new
repository creation.
* tb/config-core-filemode-check-on-broken-fs:
init-db: improve the filemode trustability check
