mirrors/git
1
0
Fork 0
mirror of https://github.com/git/git.git synced 2024-11-16 06:03:44 +01:00
Code Activity
git/t/t7701-repack-unpack-unreachable.sh

113 lines
3.3 KiB
Bash
Raw Normal View History

repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 06:01:55 +02:00
#!/bin/sh
tests: use "git xyzzy" form (t7200 - t9001) Converts tests between t7201-t9001. Signed-off-by: Nanako Shiraishi <nanako3@lavabit.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-09-03 10:59:33 +02:00
test_description='git repack works correctly'
repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 06:01:55 +02:00
. ./test-lib.sh
t7701-repack-unpack-unreachable.sh: check timestamp of unpacked objects Unpacked objects should receive the timestamp of the pack they were unpacked from. Check. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-29 02:25:05 +02:00
fsha1=
csha1=
tsha1=
repack: only unpack-unreachable if we are deleting redundant packs The -A option calls pack-objects with the --unpack-unreachable option so that the unreachable objects in local packs are left in the local object store loose. But if the -d option to repack was _not_ used, then these unpacked loose objects are redundant and unnecessary. Update tests in t7701. Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-13 21:11:46 +01:00
test_expect_success '-A with -d option leaves unreachable objects unpacked' '
repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 06:01:55 +02:00
echo content > file1 &&
git add . &&
more war on "sleep" in tests Two more tests that sleep only to waste tick can be converted to use test_tick and take expiry parameters relative to $test_tick. The basic idea is to replace "sleep 1" with "test_tick" to cause the "time" to pass. These tests are interested in expiring things with "now" as the timestamp, soo use a timestamp relative to $test_tick to give them more stability and reproducibility. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-15 00:09:57 +02:00
test_tick &&
repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 06:01:55 +02:00
git commit -m initial_commit &&
# create a transient branch with unique content
git checkout -b transient_branch &&
echo more content >> file1 &&
# record the objects created in the database for file, commit, tree
fsha1=$(git hash-object file1) &&
more war on "sleep" in tests Two more tests that sleep only to waste tick can be converted to use test_tick and take expiry parameters relative to $test_tick. The basic idea is to replace "sleep 1" with "test_tick" to cause the "time" to pass. These tests are interested in expiring things with "now" as the timestamp, soo use a timestamp relative to $test_tick to give them more stability and reproducibility. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-15 00:09:57 +02:00
test_tick &&
repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 06:01:55 +02:00
git commit -a -m more_content &&
csha1=$(git rev-parse HEAD^{commit}) &&
tsha1=$(git rev-parse HEAD^{tree}) &&
git checkout master &&
echo even more content >> file1 &&
more war on "sleep" in tests Two more tests that sleep only to waste tick can be converted to use test_tick and take expiry parameters relative to $test_tick. The basic idea is to replace "sleep 1" with "test_tick" to cause the "time" to pass. These tests are interested in expiring things with "now" as the timestamp, soo use a timestamp relative to $test_tick to give them more stability and reproducibility. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-15 00:09:57 +02:00
test_tick &&
repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 06:01:55 +02:00
git commit -a -m even_more_content &&
# delete the transient branch
git branch -D transient_branch &&
# pack the repo
git repack -A -d -l &&
# verify objects are packed in repository
test 3 = $(git verify-pack -v -- .git/objects/pack/*.idx |
tests: grep portability fixes We try to avoid using the "-q" or "-e" options, as they are largely useless, as explained in aadbe44f. There is one exception for "-e" here, which is in t7701 used to produce an "or" of patterns. This can be rewritten as an egrep pattern. This patch also removes use of "grep -F" in favor of the more widely available "fgrep". [sp: Tested on AIX 5.3 by Mike Ralphson, Tested on MinGW by Johannes Sixt] Signed-off-by: Jeff King <peff@peff.net> Tested-by: Mike Ralphson <mike@abacus.co.uk> Tested-by: Johannes Sixt <johannes.sixt@telecom.at> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2008-09-30 10:03:55 +02:00
egrep "^($fsha1|$csha1|$tsha1) " |
repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 06:01:55 +02:00
sort | uniq | wc -l) &&
git show $fsha1 &&
git show $csha1 &&
git show $tsha1 &&
more war on "sleep" in tests Two more tests that sleep only to waste tick can be converted to use test_tick and take expiry parameters relative to $test_tick. The basic idea is to replace "sleep 1" with "test_tick" to cause the "time" to pass. These tests are interested in expiring things with "now" as the timestamp, soo use a timestamp relative to $test_tick to give them more stability and reproducibility. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-15 00:09:57 +02:00
# now expire the reflog, while keeping reachable ones but expiring
# unreachables immediately
test_tick &&
sometimeago=$(( $test_tick - 10000 )) &&
git reflog expire --expire=$sometimeago --expire-unreachable=$test_tick --all &&
repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 06:01:55 +02:00
# and repack
git repack -A -d -l &&
# verify objects are retained unpacked
test 0 = $(git verify-pack -v -- .git/objects/pack/*.idx |
tests: grep portability fixes We try to avoid using the "-q" or "-e" options, as they are largely useless, as explained in aadbe44f. There is one exception for "-e" here, which is in t7701 used to produce an "or" of patterns. This can be rewritten as an egrep pattern. This patch also removes use of "grep -F" in favor of the more widely available "fgrep". [sp: Tested on AIX 5.3 by Mike Ralphson, Tested on MinGW by Johannes Sixt] Signed-off-by: Jeff King <peff@peff.net> Tested-by: Mike Ralphson <mike@abacus.co.uk> Tested-by: Johannes Sixt <johannes.sixt@telecom.at> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2008-09-30 10:03:55 +02:00
egrep "^($fsha1|$csha1|$tsha1) " |
repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 06:01:55 +02:00
sort | uniq | wc -l) &&
git show $fsha1 &&
git show $csha1 &&
git show $tsha1
'
t7701-repack-unpack-unreachable.sh: check timestamp of unpacked objects Unpacked objects should receive the timestamp of the pack they were unpacked from. Check. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-29 02:25:05 +02:00
compare_mtimes ()
{
Windows: Fix intermittent failures of t7701 The last test case checks whether unpacked objects receive the time stamp of the pack file. Due to different implementations of stat(2) by MSYS and our version in compat/mingw.c, the test fails in about half of the test runs. Note the following facts: - The test uses perl's -M operator to compare the time stamps. Since we depend on MSYS perl, the result of this operator is based on MSYS's implementation of the stat(2) call. - NTFS on Windows records fractional seconds. - The MSYS implementation of stat(2) *rounds* fractional seconds to full seconds instead of truncating them. This becomes obvious by comparing the modification times reported by 'ls --full-time $f' and 'stat $f' for various files $f. - Our implementation of stat(2) in compat/mingw.c *truncates* to full seconds. The consequence of this is that - add_packed_git() picks up a truncated whole second modification time from the pack file time stamp, which is then used for the loose objects, while the pack file retains its time stamp in fractional seconds; - but the test case compared the pack file's rounded modification times to the loose objects' truncated modification times. And half of the time the rounded modification time is not the same as its truncated modification time. The fix is that we replace perl by 'test-chmtime -v +0', which prints the truncated whole-second mtime without modifying it. Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-01-28 10:52:26 +01:00
read tref rest &&
while read t rest; do
test "$tref" = "$t" || break
done
t7701-repack-unpack-unreachable.sh: check timestamp of unpacked objects Unpacked objects should receive the timestamp of the pack they were unpacked from. Check. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-29 02:25:05 +02:00
}
repack: only unpack-unreachable if we are deleting redundant packs The -A option calls pack-objects with the --unpack-unreachable option so that the unreachable objects in local packs are left in the local object store loose. But if the -d option to repack was _not_ used, then these unpacked loose objects are redundant and unnecessary. Update tests in t7701. Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-13 21:11:46 +01:00
test_expect_success '-A without -d option leaves unreachable objects packed' '
t7701-repack-unpack-unreachable.sh: check timestamp of unpacked objects Unpacked objects should receive the timestamp of the pack they were unpacked from. Check. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-29 02:25:05 +02:00
fsha1path=$(echo "$fsha1" | sed -e "s|\(..\)|\1/|") &&
fsha1path=".git/objects/$fsha1path" &&
csha1path=$(echo "$csha1" | sed -e "s|\(..\)|\1/|") &&
csha1path=".git/objects/$csha1path" &&
tsha1path=$(echo "$tsha1" | sed -e "s|\(..\)|\1/|") &&
tsha1path=".git/objects/$tsha1path" &&
git branch transient_branch $csha1 &&
git repack -a -d -l &&
test ! -f "$fsha1path" &&
test ! -f "$csha1path" &&
test ! -f "$tsha1path" &&
test 1 = $(ls -1 .git/objects/pack/pack-*.pack | wc -l) &&
packfile=$(ls .git/objects/pack/pack-*.pack) &&
git branch -D transient_branch &&
more war on "sleep" in tests Two more tests that sleep only to waste tick can be converted to use test_tick and take expiry parameters relative to $test_tick. The basic idea is to replace "sleep 1" with "test_tick" to cause the "time" to pass. These tests are interested in expiring things with "now" as the timestamp, soo use a timestamp relative to $test_tick to give them more stability and reproducibility. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-04-15 00:09:57 +02:00
test_tick &&
t7701-repack-unpack-unreachable.sh: check timestamp of unpacked objects Unpacked objects should receive the timestamp of the pack they were unpacked from. Check. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-29 02:25:05 +02:00
git repack -A -l &&
repack: only unpack-unreachable if we are deleting redundant packs The -A option calls pack-objects with the --unpack-unreachable option so that the unreachable objects in local packs are left in the local object store loose. But if the -d option to repack was _not_ used, then these unpacked loose objects are redundant and unnecessary. Update tests in t7701. Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-11-13 21:11:46 +01:00
test ! -f "$fsha1path" &&
test ! -f "$csha1path" &&
test ! -f "$tsha1path" &&
git show $fsha1 &&
git show $csha1 &&
git show $tsha1
'
test_expect_success 'unpacked objects receive timestamp of pack file' '
tmppack=".git/objects/pack/tmp_pack" &&
ln "$packfile" "$tmppack" &&
git repack -A -l -d &&
Windows: Fix intermittent failures of t7701 The last test case checks whether unpacked objects receive the time stamp of the pack file. Due to different implementations of stat(2) by MSYS and our version in compat/mingw.c, the test fails in about half of the test runs. Note the following facts: - The test uses perl's -M operator to compare the time stamps. Since we depend on MSYS perl, the result of this operator is based on MSYS's implementation of the stat(2) call. - NTFS on Windows records fractional seconds. - The MSYS implementation of stat(2) *rounds* fractional seconds to full seconds instead of truncating them. This becomes obvious by comparing the modification times reported by 'ls --full-time $f' and 'stat $f' for various files $f. - Our implementation of stat(2) in compat/mingw.c *truncates* to full seconds. The consequence of this is that - add_packed_git() picks up a truncated whole second modification time from the pack file time stamp, which is then used for the loose objects, while the pack file retains its time stamp in fractional seconds; - but the test case compared the pack file's rounded modification times to the loose objects' truncated modification times. And half of the time the rounded modification time is not the same as its truncated modification time. The fix is that we replace perl by 'test-chmtime -v +0', which prints the truncated whole-second mtime without modifying it. Signed-off-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-01-28 10:52:26 +01:00
test-chmtime -v +0 "$tmppack" "$fsha1path" "$csha1path" "$tsha1path" \
> mtimes &&
compare_mtimes < mtimes
t7701-repack-unpack-unreachable.sh: check timestamp of unpacked objects Unpacked objects should receive the timestamp of the pack they were unpacked from. Check. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-29 02:25:05 +02:00
'
gc: do not explode objects which will be immediately pruned When we pack everything into one big pack with "git repack -Ad", any unreferenced objects in to-be-deleted packs are exploded into loose objects, with the intent that they will be examined and possibly cleaned up by the next run of "git prune". Since the exploded objects will receive the mtime of the pack from which they come, if the source pack is old, those loose objects will end up pruned immediately. In that case, it is much more efficient to skip the exploding step entirely for these objects. This patch teaches pack-objects to receive the expiration information and avoid writing these objects out. It also teaches "git gc" to pass the value of gc.pruneexpire to repack (which in turn learns to pass it along to pack-objects) so that this optimization happens automatically during "git gc" and "git gc --auto". Signed-off-by: Jeff King <peff@peff.net> Acked-by: Nicolas Pitre <nico@fluxnic.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-07 12:30:09 +02:00
test_expect_success 'do not bother loosening old objects' '
obj1=$(echo one | git hash-object -w --stdin) &&
obj2=$(echo two | git hash-object -w --stdin) &&
pack1=$(echo $obj1 | git pack-objects .git/objects/pack/pack) &&
pack2=$(echo $obj2 | git pack-objects .git/objects/pack/pack) &&
git prune-packed &&
git cat-file -p $obj1 &&
git cat-file -p $obj2 &&
test-chmtime =-86400 .git/objects/pack/pack-$pack2.pack &&
git repack -A -d --unpack-unreachable=1.hour.ago &&
git cat-file -p $obj1 &&
test_must_fail git cat-file -p $obj2
'
repack: modify behavior of -A option to leave unreferenced objects unpacked The previous behavior of the -A option was to retain any previously packed objects which had become unreferenced, and place them into the newly created pack file. Since git-gc, when run automatically with the --auto option, calls repack with the -A option, this had the effect of retaining unreferenced packed objects indefinitely. To avoid this scenario, the user was required to run git-gc with the little known --prune option or to manually run repack with the -a option. This patch changes the behavior of the -A option so that unreferenced objects that exist in any pack file being replaced, will be unpacked into the repository. The unreferenced loose objects can then be garbage collected by git-gc (i.e. git-prune) based on the gc.pruneExpire setting. Also add new tests for checking whether unreferenced objects which were previously packed are properly left in the repository unpacked after repacking. Signed-off-by: Brandon Casey <drafnel@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-10 06:01:55 +02:00
test_done
Copy permalink