When running a required clean filter, we do not have to mmap the
original before feeding the filter. Instead, stream the file
contents directly to the filter and process its output.
* sp/stream-clean-filter:
sha1_file: don't convert off_t to size_t too early to avoid potential die()
convert: stream from fd to required clean filter to reduce used address space
copy_fd(): do not close the input file descriptor
mmap_limit: introduce GIT_MMAP_LIMIT to allow testing expected mmap size
memory_limit: use git_env_ulong() to parse GIT_ALLOC_LIMIT
config.c: add git_env_ulong() to parse environment variable
convert: drop arguments other than 'path' from would_convert_to_git()
Allow "git push" request to be signed, so that it can be verified and
audited, using the GPG signature of the person who pushed, that the
tips of branches at a public repository really point the commits
the pusher wanted to, without having to "trust" the server.
* jc/push-cert: (24 commits)
receive-pack::hmac_sha1(): copy the entire SHA-1 hash out
signed push: allow stale nonce in stateless mode
signed push: teach smart-HTTP to pass "git push --signed" around
signed push: fortify against replay attacks
signed push: add "pushee" header to push certificate
signed push: remove duplicated protocol info
send-pack: send feature request on push-cert packet
receive-pack: GPG-validate push certificates
push: the beginning of "git push --signed"
pack-protocol doc: typofix for PKT-LINE
gpg-interface: move parse_signature() to where it should be
gpg-interface: move parse_gpg_output() to where it should be
send-pack: clarify that cmds_sent is a boolean
send-pack: refactor inspecting and resetting status and sending commands
send-pack: rename "new_refs" to "need_pack_data"
receive-pack: factor out capability string generation
send-pack: factor out capability string generation
send-pack: always send capabilities
send-pack: refactor decision to send update per ref
send-pack: move REF_STATUS_REJECT_NODELETE logic a bit higher
...
FreeBSD's printf(1) doesn't accept empty strings for numerical format
specifiers:
$ printf "%d\n" "" >/dev/null; echo $?
printf: : expected numeric value
1
Initialize the AWK variable c to make sure the shell variable
subtree_count always contains a numerical value, in order to keep the
subsequently called printf happy.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We used to get confused when a process called us with SIGPIPE
ignored; we do want to die with SIGPIPE when the output is not
read by default, and do ignore the signal when appropriate.
* pr/use-default-sigpipe-setting:
mingw.h: add dummy functions for sigset_t operations
unblock and unignore SIGPIPE
Some MUAs mangled a line in a message that begins with "From " to
">From " when writing to a mailbox file and feeding such an input
to "git am" used to lose such a line.
* jk/mbox-from-line:
mailinfo: work around -Wstring-plus-int warning
mailinfo: make ">From" in-body header check more robust
"rev-parse --verify --quiet $name" is meant to quietly exit with a
non-zero status when $name is not a valid object name, but still
gave error messages in some cases.
* da/rev-parse-verify-quiet:
stash: prefer --quiet over shell redirection of the standard error stream
refs: make rev-parse --quiet actually quiet
t1503: use test_must_be_empty
Documentation: a note about stdout for git rev-parse --verify --quiet
The pretty-format specifier "%d", which expanded to " (tagname)"
for a tagged commit, gained a cousin "%D" that just gives the
"tagname" without frills.
* hj/pretty-naked-decoration:
pretty: add %D format specifier
Use write_script to create the helper "askpass" script, instead of
hand-creating it with hardcoded "#!/bin/sh" to make sure we use the
shell the user told us to use.
Signed-off-by: Ben Walton <bdwalton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
pre- and post-receive hooks are no longer required to read all
their inputs.
* jc/ignore-sigpipe-while-running-hooks:
receive-pack: allow hooks to ignore its standard input stream
Using "hash-object --literally", test one of the new breakages
js/fsck-tag-validation topic teaches "fsck" to catch is caught.
* jc/hash-object-fsck-tag:
t1450: make sure fsck detects a malformed tagger line
Teach "git fsck" to inspect the contents of annotated tag objects.
* js/fsck-tag-validation:
Make sure that index-pack --strict checks tag objects
Add regression tests for stricter tag fsck'ing
fsck: check tag objects' headers
Make sure fsck_commit_buffer() does not run out of the buffer
fsck_object(): allow passing object data separately from the object itself
Refactor type_from_string() to allow continuing after detecting an error
Optimize the check to see if a ref $F can be created by making sure
no existing ref has $F/ as its prefix, which especially matters in
a repository with a large number of existing refs.
* jk/faster-name-conflicts:
refs: speed up is_refname_available
It was reported that the allocated stack space was too small for
some archs openSUSE buildfarm runs the tests on. Double it while
also doubling the amount of data to be handled.
Reported-by: Andreas Schwab <schwab@linux-m68k.org>
Suggested-by: Jeff King <peff@peff.net>
Tested-by: Andreas Schwab <schwab@linux-m68k.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Windows does not have POSIX-like signals, and so we ignore all
operations on the non-existent signal mask machinery.
Do not turn sigemptyset into a function, but leave it a macro that
erases the code in the argument because it is used to set sa_mask
of a struct sigaction, but our dummy in mingw.h does not have that
member.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch fixes two problems with using :(glob) (or even "*.c"
without ":(glob)").
The first one is we forgot to turn on the 'recursive' flag in struct
pathspec. Without that, tree_entry_interesting() will not mark
potential directories "interesting" so that it can confirm whether
those directories have anything matching the pathspec.
The marking directories interesting has a side effect that we need to
walk inside a directory to realize that there's nothing interested in
there. By that time, 'archive' code has already written the (empty)
directory down. That means lots of empty directories in the result
archive.
This problem is fixed by lazily writing directories down when we know
they are actually needed. There is a theoretical bug in this
implementation: we can't write empty trees/directories that match that
pathspec.
path_exists() is also made stricter in order to detect non-matching
pathspec because when this 'recursive' flag is on, we most likely
match some directories. The easiest way is not consider any
directories "matched".
Noticed-by: Peter Wu <peter@lekensteyn.nl>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jn/unpack-trees-checkout-m-carry-deletion:
checkout -m: attempt merge when deletion of path was staged
unpack-trees: use 'cuddled' style for if-else cascade
unpack-trees: simplify 'all other failures' case
* jc/apply-ws-prefix:
apply: omit ws check for excluded paths
apply: hoist use_patch() helper for path exclusion up
apply: use the right attribute for paths in non-Git patches
Conflicts:
builtin/apply.c
"git fsck" failed to report that it found corrupt objects via its
exit status in some cases.
* jk/fsck-exit-code-fix:
fsck: return non-zero status on missing ref tips
fsck: exit with non-zero status upon error from fsck_obj()
"git config --add section.var val" used to lose existing
section.var whose value was an empty string.
* ta/config-add-to-empty-or-true-fix:
config: avoid a funny sentinel value "a^"
make config --add behave correctly for empty and NULL values
Add checks for a common programming mistake to assign the same
short option name to two separate options to help developers.
* jc/parseopt-verify-short-name:
parse-options: detect attempt to add a duplicate short option name
* tb/crlf-tests:
MinGW: update tests to handle a native eol of crlf
Makefile: propagate NATIVE_CRLF to C
t0027: Tests for core.eol=native, eol=lf, eol=crlf
An attempt to remove the entire tree in the "git fast-import" input
stream caused it to misbehave.
* mb/fast-import-delete-root:
fast-import: fix segfault in store_tree()
t9300: test filedelete command
"log --date=iso" uses a slight variant of ISO 8601 format that is
made more human readable. A new "--date=iso-strict" option gives
datetime output that is more strictly conformant.
* bb/date-iso-strict:
pretty: provide a strict ISO 8601 date format
Sometimes users want to report a bug they experience on their
repository, but they are not at liberty to share the contents of
the repository. "fast-export" was taught an "--anonymize" option
to replace blob contents, names of people and paths and log
messages with bland and simple strings to help them.
* jk/fast-export-anonymize:
docs/fast-export: explain --anonymize more completely
teach fast-export an --anonymize option
The number of refs that can be pushed at once over smart HTTP was
limited by the command line length. The limitation has been lifted
by passing these refs from the standard input of send-pack.
* jk/send-pack-many-refspecs:
send-pack: take refspecs over stdin
When a reflog is deleted, e.g. when "git stash" clears its stashes,
"git rev-parse --verify --quiet" dies:
fatal: Log for refs/stash is empty.
The reason is that the get_sha1() code path does not allow us
to suppress this message.
Pass the flags bitfield through get_sha1_with_context() so that
read_ref_at() can suppress the message.
Use get_sha1_with_context1() instead of get_sha1() in rev-parse
so that the --quiet flag is honored.
Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a new format specifier, '%D' that is identical in behaviour to '%d',
except that it does not include the ' (' prefix or ')' suffix provided
by '%d'.
Signed-off-by: Harry Jeffery <harry@exec64.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Blocked and ignored signals -- but not caught signals -- are inherited
across exec. Some callers with sloppy signal-handling behavior can call
git with SIGPIPE blocked or ignored, even non-deterministically. When
SIGPIPE is blocked or ignored, several git commands can run indefinitely,
ignoring EPIPE returns from write() calls, even when the process that
called them has gone away. Our specific case involved a pipe of git
diff-tree output to a script that reads a limited amount of diff data.
In an ideal world, git would never be called with SIGPIPE blocked or
ignored. But in the real world, several real potential callers, including
Perl, Apache, and Unicorn, sometimes spawn subprocesses with SIGPIPE
ignored. It is easier and more productive to harden git against this
mistake than to clean it up in every potential parent process.
Signed-off-by: Patrick Reynolds <patrick.reynolds@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we run `branch --merged`, we use prepare_revision_walk
with the merge-filter marked as UNINTERESTING. Any branch
tips that are marked UNINTERESTING after it returns must be
ancestors of that commit. As we iterate through the list of
refs to show, we check item->commit->object.flags to see
whether it was marked.
This interacts badly with --verbose, which will do a
separate walk to find the ahead/behind information for each
branch. There are two bad things that can happen:
1. The ahead/behind walk may get the wrong results,
because it can see a bogus UNINTERESTING flag leftover
from the merge-filter walk.
2. We may omit some branches if their tips are involved in
the ahead/behind traversal of a branch shown earlier.
The ahead/behind walk carefully cleans up its commit
flags, meaning it may also erase the UNINTERESTING
flag that we expect to check later.
We can solve this by moving the merge-filter state for each
ref into its "struct ref_item" as soon as we finish the
merge-filter walk. That fixes (2). Then we are free to clear
the commit flags we used in the walk, fixing (1).
Note that we actually do away with the matches_merge_filter
helper entirely here, and inline it between the revision
walk and the flag-clearing. This ensures that nobody
accidentally calls it at the wrong time (it is only safe to
check in that instant between the setting and clearing of
the global flag).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When operating with the stateless RPC mode, we will receive a nonce
issued by another instance of us that advertised our capability and
refs some time ago. Update the logic to check received nonce to
detect this case, compute how much time has passed since the nonce
was issued and report the status with a new environment variable
GIT_PUSH_CERT_NONCE_SLOP to the hooks.
GIT_PUSH_CERT_NONCE_STATUS will report "SLOP" in such a case. The
hooks are free to decide how large a slop it is willing to accept.
Strictly speaking, the "nonce" is not really a "nonce" anymore in
the stateless RPC mode, as it will happily take any "nonce" issued
by it (which is protected by HMAC and its secret key) as long as it
is fresh enough. The degree of this security degradation, relative
to the native protocol, is about the same as the "we make sure that
the 'git push' decided to update our refs with new objects based on
the freshest observation of our refs by making sure the values they
claim the original value of the refs they ask us to update exactly
match the current state" security is loosened to accomodate the
stateless RPC mode in the existing code without this series, so
there is no need for those who are already using smart HTTP to push
to their repositories to be alarmed any more than they already are.
In addition, the server operator can set receive.certnonceslop
configuration variable to specify how stale a nonce can be (in
seconds). When this variable is set, and if the nonce received in
the certificate that passes the HMAC check was less than that many
seconds old, hooks are given "OK" in GIT_PUSH_CERT_NONCE_STATUS
(instead of "SLOP") and the received nonce value is given in
GIT_PUSH_CERT_NONCE, which makes it easier for a simple-minded
hook to check if the certificate we received is recent enough.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The "--signed" option received by "git push" is first passed to the
transport layer, which the native transport directly uses to notice
that a push certificate needs to be sent. When the transport-helper
is involved, however, the option needs to be told to the helper with
set_helper_option(), and the helper needs to take necessary action.
For the smart-HTTP helper, the "necessary action" involves spawning
the "git send-pack" subprocess with the "--signed" option.
Once the above all gets wired in, the smart-HTTP transport now can
use the push certificate mechanism to authenticate its pushes.
Add a test that is modeled after tests for the native transport in
t5534-push-signed.sh to t5541-http-push-smart.sh. Update the test
Apache configuration to pass GNUPGHOME environment variable through.
As PassEnv would trigger warnings for an environment variable that
is not set, export it from test-lib.sh set to a harmless value when
GnuPG is not being used in the tests.
Note that the added test is deliberately loose and does not check
the nonce in this step. This is because the stateless RPC mode is
inevitably flaky and a nonce that comes back in the actual push
processing is one issued by a different process; if the two
interactions with the server crossed a second boundary, the nonces
will not match and such a check will fail. A later patch in the
series will work around this shortcoming.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In order to prevent a valid push certificate for pushing into an
repository from getting replayed in a different push operation, send
a nonce string from the receive-pack process and have the signer
include it in the push certificate. The receiving end uses an HMAC
hash of the path to the repository it serves and the current time
stamp, hashed with a secret seed (the secret seed does not have to
be per-repository but can be defined in /etc/gitconfig) to generate
the nonce, in order to ensure that a random third party cannot forge
a nonce that looks like it originated from it.
The original nonce is exported as GIT_PUSH_CERT_NONCE for the hooks
to examine and match against the value on the "nonce" header in the
certificate to notice a replay, but returned "nonce" header in the
push certificate is examined by receive-pack and the result is
exported as GIT_PUSH_CERT_NONCE_STATUS, whose value would be "OK"
if the nonce recorded in the certificate matches what we expect, so
that the hooks can more easily check.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The pre-receive and post-receive hooks were designed to be an
improvement over old style update and post-update hooks, which take
the update information on their command line and are limited by the
command line length limit. The same information is fed from the
standard input to pre/post-receive hooks instead to lift this
limitation. It has been mandatory for these new style hooks to
consume the update information fully from the standard input stream.
Otherwise, they would risk killing the receive-pack process via
SIGPIPE.
If a hook does not want to look at all the information, it is easy
to send its standard input to /dev/null (perhaps a niche use of hook
might need to know only the fact that a push was made, without
having to know what objects have been pushed to update which refs),
and this has already been done by existing hooks that are written
carefully.
However, because there is no good way to consistently fail hooks
that do not consume the input fully (a small push may result in a
short update record that may fit within the pipe buffer, to which
the receive-pack process may manage to write before the hook has a
chance to exit without reading anything, which will not result in a
death-by-SIGPIPE of receive-pack), it can lead to a hard to diagnose
"once in a blue moon" phantom failure.
Lift this "hooks must consume their input fully" mandate. A mandate
that is not enforced strictly is not helping us to catch mistakes in
hooks. If a hook has a good reason to decide the outcome of its
operation without reading the information we feed it, let it do so
as it pleases.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use `test_must_be_be_empty <file>` instead of `test -z "$(cat <file>)"`.
Suggested-by: Fabian Ruch <bafain@gmail.com>
Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
