2013-03-25 21:21:34 +01:00
|
|
|
#!/bin/sh
|
|
|
|
|
|
|
|
|
|
test_description='see how we handle various forms of corruption'
|
|
|
|
|
. ./test-lib.sh
|
|
|
|
|
|
|
|
|
|
# convert "1234abcd" to ".git/objects/12/34abcd"
|
|
|
|
|
obj_to_file() {
|
|
|
|
|
echo "$(git rev-parse --git-dir)/objects/$(git rev-parse "$1" | sed 's,..,&/,')"
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
# Convert byte at offset "$2" of object "$1" into '\0'
|
|
|
|
|
corrupt_byte() {
|
|
|
|
|
obj_file=$(obj_to_file "$1") &&
|
|
|
|
|
chmod +w "$obj_file" &&
|
|
|
|
|
printf '\0' | dd of="$obj_file" bs=1 seek="$2" conv=notrunc
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
test_expect_success 'setup corrupt repo' '
|
|
|
|
|
git init bit-error &&
|
|
|
|
|
(
|
|
|
|
|
cd bit-error &&
|
|
|
|
|
test_commit content &&
|
|
|
|
|
corrupt_byte HEAD:content.t 10
|
index-pack: detect local corruption in collision check
When we notice that we have a local copy of an incoming
object, we compare the two objects to make sure we haven't
found a collision. Before we get to the actual object
bytes, though, we compare the type and size from
sha1_object_info().
If our local object is corrupted, then the type will be
OBJ_BAD, which obviously will not match the incoming type,
and we'll report "SHA1 COLLISION FOUND" (with capital
letters and everything). This is confusing, as the problem
is not a collision but rather local corruption. We should
report that instead (just like we do if reading the rest of
the object content fails a few lines later).
Note that we _could_ just ignore the error and mark it as a
non-collision. That would let you "git fetch" to replace a
corrupted object. But it's not a very reliable method for
repairing a repository. The earlier want/have negotiation
tries to get the other side to omit objects we already have,
and it would not realize that we are "missing" this
corrupted object. So we're better off complaining loudly
when we see corruption, and letting the user take more
drastic measures to repair (like making a full clone
elsewhere and copying the pack into place).
Note that the test sets transfer.unpackLimit in the
receiving repository so that we use index-pack (which is
what does the collision check). Normally for such a small
push we'd use unpack-objects, which would simply try to
write the loose object, and discard the new one when we see
that there's already an old one.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-01 10:09:32 +02:00
|
|
|
) &&
|
|
|
|
|
git init no-bit-error &&
|
|
|
|
|
(
|
|
|
|
|
# distinct commit from bit-error, but containing a
|
|
|
|
|
# non-corrupted version of the same blob
|
|
|
|
|
cd no-bit-error &&
|
|
|
|
|
test_tick &&
|
|
|
|
|
test_commit content
|
2013-03-25 21:21:34 +01:00
|
|
|
)
|
|
|
|
|
'
|
|
|
|
|
|
2013-03-25 22:49:36 +01:00
|
|
|
test_expect_success 'setup repo with missing object' '
|
|
|
|
|
git init missing &&
|
|
|
|
|
(
|
|
|
|
|
cd missing &&
|
|
|
|
|
test_commit content &&
|
|
|
|
|
rm -f "$(obj_to_file HEAD:content.t)"
|
|
|
|
|
)
|
|
|
|
|
'
|
|
|
|
|
|
2013-03-25 21:22:29 +01:00
|
|
|
test_expect_success 'setup repo with misnamed object' '
|
|
|
|
|
git init misnamed &&
|
|
|
|
|
(
|
|
|
|
|
cd misnamed &&
|
|
|
|
|
test_commit content &&
|
|
|
|
|
good=$(obj_to_file HEAD:content.t) &&
|
|
|
|
|
blob=$(echo corrupt | git hash-object -w --stdin) &&
|
|
|
|
|
bad=$(obj_to_file $blob) &&
|
|
|
|
|
rm -f "$good" &&
|
|
|
|
|
mv "$bad" "$good"
|
|
|
|
|
)
|
|
|
|
|
'
|
|
|
|
|
|
2013-03-25 21:21:34 +01:00
|
|
|
test_expect_success 'streaming a corrupt blob fails' '
|
|
|
|
|
(
|
|
|
|
|
cd bit-error &&
|
|
|
|
|
test_must_fail git cat-file blob HEAD:content.t
|
|
|
|
|
)
|
|
|
|
|
'
|
|
|
|
|
|
sha1_loose_object_info: return error for corrupted objects
When sha1_loose_object_info() finds that a loose object file
cannot be stat(2)ed or mmap(2)ed, it returns -1 to signal an
error to the caller. However, if it found that the loose
object file is corrupt and the object data cannot be used
from it, it stuffs OBJ_BAD into "type" field of the
object_info, but returns zero (i.e., success), which can
confuse callers.
This is due to 052fe5eac (sha1_loose_object_info: make type
lookup optional, 2013-07-12), which switched the return to a
strict success/error, rather than returning the type (but
botched the return).
Callers of regular sha1_object_info() don't notice the
difference, as that function returns the type (which is
OBJ_BAD in this case). However, direct callers of
sha1_object_info_extended() see the function return success,
but without setting any meaningful values in the object_info
struct, leading them to access potentially uninitialized
memory.
The easiest way to see the bug is via "cat-file -s", which
will happily ignore the corruption and report whatever
value happened to be in the "size" variable.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-01 10:05:21 +02:00
|
|
|
test_expect_success 'getting type of a corrupt blob fails' '
|
|
|
|
|
(
|
|
|
|
|
cd bit-error &&
|
|
|
|
|
test_must_fail git cat-file -s HEAD:content.t
|
|
|
|
|
)
|
|
|
|
|
'
|
|
|
|
|
|
2013-03-25 22:49:36 +01:00
|
|
|
test_expect_success 'read-tree -u detects bit-errors in blobs' '
|
|
|
|
|
(
|
|
|
|
|
cd bit-error &&
|
|
|
|
|
rm -f content.t &&
|
|
|
|
|
test_must_fail git read-tree --reset -u HEAD
|
|
|
|
|
)
|
|
|
|
|
'
|
|
|
|
|
|
|
|
|
|
test_expect_success 'read-tree -u detects missing objects' '
|
|
|
|
|
(
|
|
|
|
|
cd missing &&
|
|
|
|
|
rm -f content.t &&
|
|
|
|
|
test_must_fail git read-tree --reset -u HEAD
|
|
|
|
|
)
|
|
|
|
|
'
|
|
|
|
|
|
2013-03-25 21:22:29 +01:00
|
|
|
# We use --bare to make sure that the transport detects it, not the checkout
|
|
|
|
|
# phase.
|
|
|
|
|
test_expect_success 'clone --no-local --bare detects corruption' '
|
|
|
|
|
test_must_fail git clone --no-local --bare bit-error corrupt-transport
|
|
|
|
|
'
|
|
|
|
|
|
|
|
|
|
test_expect_success 'clone --no-local --bare detects missing object' '
|
|
|
|
|
test_must_fail git clone --no-local --bare missing missing-transport
|
|
|
|
|
'
|
|
|
|
|
|
2013-03-25 21:26:27 +01:00
|
|
|
test_expect_success 'clone --no-local --bare detects misnamed object' '
|
2013-03-25 21:22:29 +01:00
|
|
|
test_must_fail git clone --no-local --bare misnamed misnamed-transport
|
|
|
|
|
'
|
|
|
|
|
|
|
|
|
|
# We do not expect --local to detect corruption at the transport layer,
|
|
|
|
|
# so we are really checking the checkout() code path.
|
|
|
|
|
test_expect_success 'clone --local detects corruption' '
|
|
|
|
|
test_must_fail git clone --local bit-error corrupt-checkout
|
|
|
|
|
'
|
|
|
|
|
|
2013-03-26 23:22:09 +01:00
|
|
|
test_expect_success 'error detected during checkout leaves repo intact' '
|
|
|
|
|
test_path_is_dir corrupt-checkout/.git
|
|
|
|
|
'
|
|
|
|
|
|
clone: die on errors from unpack_trees
When clone is populating the working tree, it ignores the
return status from unpack_trees; this means we may report a
successful clone, even when the checkout fails.
When checkout fails, we may want to leave the $GIT_DIR in
place, as it might be possible to recover the data through
further use of "git checkout" (e.g., if the checkout failed
due to a transient error, disk full, etc). However, we
already die on a number of other checkout-related errors, so
this patch follows that pattern.
In addition to marking a now-passing test, we need to adjust
t5710, which blindly assumed it could make bogus clones of
very deep alternates hierarchies. By using "--bare", we can
avoid it actually touching any objects.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-03-25 21:23:59 +01:00
|
|
|
test_expect_success 'clone --local detects missing objects' '
|
2013-03-25 21:22:29 +01:00
|
|
|
test_must_fail git clone --local missing missing-checkout
|
|
|
|
|
'
|
|
|
|
|
|
|
|
|
|
test_expect_failure 'clone --local detects misnamed objects' '
|
|
|
|
|
test_must_fail git clone --local misnamed misnamed-checkout
|
|
|
|
|
'
|
|
|
|
|
|
index-pack: detect local corruption in collision check
When we notice that we have a local copy of an incoming
object, we compare the two objects to make sure we haven't
found a collision. Before we get to the actual object
bytes, though, we compare the type and size from
sha1_object_info().
If our local object is corrupted, then the type will be
OBJ_BAD, which obviously will not match the incoming type,
and we'll report "SHA1 COLLISION FOUND" (with capital
letters and everything). This is confusing, as the problem
is not a collision but rather local corruption. We should
report that instead (just like we do if reading the rest of
the object content fails a few lines later).
Note that we _could_ just ignore the error and mark it as a
non-collision. That would let you "git fetch" to replace a
corrupted object. But it's not a very reliable method for
repairing a repository. The earlier want/have negotiation
tries to get the other side to omit objects we already have,
and it would not realize that we are "missing" this
corrupted object. So we're better off complaining loudly
when we see corruption, and letting the user take more
drastic measures to repair (like making a full clone
elsewhere and copying the pack into place).
Note that the test sets transfer.unpackLimit in the
receiving repository so that we use index-pack (which is
what does the collision check). Normally for such a small
push we'd use unpack-objects, which would simply try to
write the loose object, and discard the new one when we see
that there's already an old one.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-01 10:09:32 +02:00
|
|
|
test_expect_success 'fetch into corrupted repo with index-pack' '
|
2018-10-30 19:43:31 +01:00
|
|
|
cp -R bit-error bit-error-cp &&
|
|
|
|
|
test_when_finished "rm -rf bit-error-cp" &&
|
index-pack: detect local corruption in collision check
When we notice that we have a local copy of an incoming
object, we compare the two objects to make sure we haven't
found a collision. Before we get to the actual object
bytes, though, we compare the type and size from
sha1_object_info().
If our local object is corrupted, then the type will be
OBJ_BAD, which obviously will not match the incoming type,
and we'll report "SHA1 COLLISION FOUND" (with capital
letters and everything). This is confusing, as the problem
is not a collision but rather local corruption. We should
report that instead (just like we do if reading the rest of
the object content fails a few lines later).
Note that we _could_ just ignore the error and mark it as a
non-collision. That would let you "git fetch" to replace a
corrupted object. But it's not a very reliable method for
repairing a repository. The earlier want/have negotiation
tries to get the other side to omit objects we already have,
and it would not realize that we are "missing" this
corrupted object. So we're better off complaining loudly
when we see corruption, and letting the user take more
drastic measures to repair (like making a full clone
elsewhere and copying the pack into place).
Note that the test sets transfer.unpackLimit in the
receiving repository so that we use index-pack (which is
what does the collision check). Normally for such a small
push we'd use unpack-objects, which would simply try to
write the loose object, and discard the new one when we see
that there's already an old one.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-01 10:09:32 +02:00
|
|
|
(
|
2018-10-30 19:43:31 +01:00
|
|
|
cd bit-error-cp &&
|
index-pack: detect local corruption in collision check
When we notice that we have a local copy of an incoming
object, we compare the two objects to make sure we haven't
found a collision. Before we get to the actual object
bytes, though, we compare the type and size from
sha1_object_info().
If our local object is corrupted, then the type will be
OBJ_BAD, which obviously will not match the incoming type,
and we'll report "SHA1 COLLISION FOUND" (with capital
letters and everything). This is confusing, as the problem
is not a collision but rather local corruption. We should
report that instead (just like we do if reading the rest of
the object content fails a few lines later).
Note that we _could_ just ignore the error and mark it as a
non-collision. That would let you "git fetch" to replace a
corrupted object. But it's not a very reliable method for
repairing a repository. The earlier want/have negotiation
tries to get the other side to omit objects we already have,
and it would not realize that we are "missing" this
corrupted object. So we're better off complaining loudly
when we see corruption, and letting the user take more
drastic measures to repair (like making a full clone
elsewhere and copying the pack into place).
Note that the test sets transfer.unpackLimit in the
receiving repository so that we use index-pack (which is
what does the collision check). Normally for such a small
push we'd use unpack-objects, which would simply try to
write the loose object, and discard the new one when we see
that there's already an old one.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-01 10:09:32 +02:00
|
|
|
test_must_fail git -c transfer.unpackLimit=1 \
|
|
|
|
|
fetch ../no-bit-error 2>stderr &&
|
|
|
|
|
test_i18ngrep ! -i collision stderr
|
|
|
|
|
)
|
|
|
|
|
'
|
|
|
|
|
|
rev-list: allow cached objects in existence check
This fixes a regression in 7c0fe330d5 (rev-list: handle missing tree
objects properly, 2018-10-05) where rev-list will now complain about the
empty tree when it doesn't physically exist on disk.
Before that commit, we relied on the traversal code in list-objects.c to
walk through the trees. Since it uses parse_tree(), we'd do a normal
object lookup that includes looking in the set of "cached" objects
(which is where our magic internal empty-tree kicks in).
After that commit, we instead tell list-objects.c not to die on any
missing trees, and we check them ourselves using has_object_file(). But
that function uses OBJECT_INFO_SKIP_CACHED, which means we won't use our
internal empty tree.
This normally wouldn't come up. For most operations, Git will try to
write out the empty tree object as it would any other object. And
pack-objects in a push or fetch will send the empty tree (even if it's
virtual on the sending side). However, there are cases where this can
matter. One I found in the wild:
1. The root tree of a commit became empty by deleting all files,
without using an index. In this case it was done using libgit2's
tree builder API, but as the included test shows, it can easily be
done with regular git using hash-object.
The resulting repo works OK, as we'd avoid walking over our own
reachable commits for a connectivity check.
2. Cloning with --reference pointing to the repository from (1) can
trigger the problem, because we tell the other side we already have
that commit (and hence the empty tree), but then walk over it
during the connectivity check (where we complain about it missing).
Arguably the workflow in step (1) should be more careful about writing
the empty tree object if we're referencing it. But this workflow did
work prior to 7c0fe330d5, so let's restore it.
This patch makes the minimal fix, which is to swap out a direct call to
oid_object_info_extended(), minus the SKIP_CACHED flag, instead of
calling has_object_file(). This is all that has_object_file() is doing
under the hood. And there's little danger of unrelated fallout from
other unexpected "cached" objects, since there's only one call site that
ends such a cached object, and it's in git-blame.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-04 18:40:54 +01:00
|
|
|
test_expect_success 'internal tree objects are not "missing"' '
|
|
|
|
|
git init missing-empty &&
|
|
|
|
|
(
|
|
|
|
|
cd missing-empty &&
|
|
|
|
|
empty_tree=$(git hash-object -t tree /dev/null) &&
|
|
|
|
|
commit=$(echo foo | git commit-tree $empty_tree) &&
|
|
|
|
|
git rev-list --objects $commit
|
|
|
|
|
)
|
|
|
|
|
'
|
|
|
|
|
|
2013-03-25 21:21:34 +01:00
|
|
|
test_done
|