mirrors/git
1
0
Fork 0
mirror of https://github.com/git/git.git synced 2024-11-14 13:13:01 +01:00
Code Activity
git/builtin-prune-packed.c

91 lines
1.9 KiB
C
Raw Normal View History

use declarations from builtin.h for builtin commands Signed-off-by: Matthias Kestenholz <matthias@spinlock.ch> Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-08-02 23:52:00 +02:00
#include "builtin.h"
Add "git-prune-packed" that removes objects that exist in a pack. This, together with "git repack" can be used to clean up unpacked git archives.
2005-07-03 23:27:34 +02:00
#include "cache.h"
Teach prune-packed to use the standard progress meter Rather than reimplementing the progress meter logic and always showing 100 lines of output while pruning already packed objects we now use a delayed progress meter and only show it if there are enough objects to make us take a little while. Most users won't see the message anymore as it usually doesn't take very long to delete the already packed loose objects. This neatens the output of a git-gc or git-repack execution, which is especially important for a `git gc --auto` triggered from within another command. We perform the display_progress() call from within the very innermost loop in case we spend more than 1 second within any single object directory. This ensures that a progress_update event from the timer will still trigger in a timely fashion and allow the user to see the progress meter. While I'm in here I changed the message to be more descriptive of its actual task. "Removing unused objects" is a little scary for new users as they wonder where these unused objects came from and how they should avoid them. Truth is these objects aren't unused in the sense of what git-prune would call a dangling object, these are used but are just duplicates of things we have already stored in a packfile. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2007-10-19 06:08:37 +02:00
#include "progress.h"
Add "git-prune-packed" that removes objects that exist in a pack. This, together with "git repack" can be used to clean up unpacked git archives.
2005-07-03 23:27:34 +02:00
Call prune-packed from "git prune" as well. Add -n (dryrun) flag to git-prune-packed, and call it from "git prune". Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-08-20 06:38:36 +02:00
static const char prune_packed_usage[] =
Make usage strings dash-less When you misuse a git command, you are shown the usage string. But this is currently shown in the dashed form. So if you just copy what you see, it will not work, when the dashed form is no longer supported. This patch makes git commands show the dash-less version. For shell scripts that do not specify OPTIONS_SPEC, git-sh-setup.sh generates a dash-less usage string now. Signed-off-by: Stephan Beyer <s-beyer@gmx.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-07-13 15:36:15 +02:00
"git prune-packed [-n] [-q]";
Call prune-packed from "git prune" as well. Add -n (dryrun) flag to git-prune-packed, and call it from "git prune". Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-08-20 06:38:36 +02:00
Make git-prune-packed a bit more chatty. Steven Grimm noticed that git-repack's verbosity is inconsistent because pack-objects is chatty and prune-packed is not. This makes the latter a bit more chatty and gives -q option to squelch it. Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-01-13 00:00:13 +01:00
#define DRY_RUN 01
#define VERBOSE 02
make struct progress an opaque type This allows for better management of progress "object" existence, as well as making the progress display implementation more independent from its callers. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-10-30 19:57:32 +01:00
static struct progress *progress;
Teach prune-packed to use the standard progress meter Rather than reimplementing the progress meter logic and always showing 100 lines of output while pruning already packed objects we now use a delayed progress meter and only show it if there are enough objects to make us take a little while. Most users won't see the message anymore as it usually doesn't take very long to delete the already packed loose objects. This neatens the output of a git-gc or git-repack execution, which is especially important for a `git gc --auto` triggered from within another command. We perform the display_progress() call from within the very innermost loop in case we spend more than 1 second within any single object directory. This ensures that a progress_update event from the timer will still trigger in a timely fashion and allow the user to see the progress meter. While I'm in here I changed the message to be more descriptive of its actual task. "Removing unused objects" is a little scary for new users as they wonder where these unused objects came from and how they should avoid them. Truth is these objects aren't unused in the sense of what git-prune would call a dangling object, these are used but are just duplicates of things we have already stored in a packfile. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2007-10-19 06:08:37 +02:00
Make git-prune-packed a bit more chatty. Steven Grimm noticed that git-repack's verbosity is inconsistent because pack-objects is chatty and prune-packed is not. This makes the latter a bit more chatty and gives -q option to squelch it. Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-01-13 00:00:13 +01:00
static void prune_dir(int i, DIR *dir, char *pathname, int len, int opts)
Add "git-prune-packed" that removes objects that exist in a pack. This, together with "git repack" can be used to clean up unpacked git archives.
2005-07-03 23:27:34 +02:00
{
struct dirent *de;
char hex[40];
sprintf(hex, "%02x", i);
while ((de = readdir(dir)) != NULL) {
unsigned char sha1[20];
if (strlen(de->d_name) != 38)
continue;
memcpy(hex+2, de->d_name, 38);
if (get_sha1_hex(hex, sha1))
continue;
pack-objects --unpacked=<existing pack> option. Incremental repack without -a essentially boils down to: rev-list --objects --unpacked --all | pack-objects $new_pack which picks up all loose objects that are still live and creates a new pack. This implements --unpacked=<existing pack> option to tell the revision walking machinery to pretend as if objects in such a pack are unpacked for the purpose of object listing. With this, we could say: rev-list --objects --unpacked=$active_pack --all | pack-objects $new_pack instead, to mean "all live loose objects but pretend as if objects that are in this pack are also unpacked". The newly created pack would be perfect for updating $active_pack by replacing it. Since pack-objects now knows how to do the rev-list's work itself internally, you can also write the above example by: pack-objects --unpacked=$active_pack --all $new_pack </dev/null Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-09-06 11:12:09 +02:00
if (!has_sha1_pack(sha1, NULL))
Add "git-prune-packed" that removes objects that exist in a pack. This, together with "git repack" can be used to clean up unpacked git archives.
2005-07-03 23:27:34 +02:00
continue;
memcpy(pathname + len, de->d_name, 38);
Make git-prune-packed a bit more chatty. Steven Grimm noticed that git-repack's verbosity is inconsistent because pack-objects is chatty and prune-packed is not. This makes the latter a bit more chatty and gives -q option to squelch it. Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-01-13 00:00:13 +01:00
if (opts & DRY_RUN)
Call prune-packed from "git prune" as well. Add -n (dryrun) flag to git-prune-packed, and call it from "git prune". Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-08-20 06:38:36 +02:00
printf("rm -f %s\n", pathname);
else if (unlink(pathname) < 0)
Add "git-prune-packed" that removes objects that exist in a pack. This, together with "git repack" can be used to clean up unpacked git archives.
2005-07-03 23:27:34 +02:00
error("unable to unlink %s", pathname);
return the prune-packed progress display to the inner loop This reverts commit 0e549137966feb016927a827fb6e359aec8264a3 so to return to the same state as commit b5d72f0a4cd3cce945ca0d37e4fa0ebbfcdcdb52. On Wed, 31 Oct 2007, Shawn O. Pearce wrote: > During my testing with a 40,000 loose object case (yea, I fully > unpacked a git.git clone I had laying around) my system stalled > hard in the first object directory. A *lot* longer than 1 second. > So I got no progress meter for a long time, and then a progress > meter appeared on the second directory. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-11-01 21:59:55 +01:00
display_progress(progress, i + 1);
Add "git-prune-packed" that removes objects that exist in a pack. This, together with "git repack" can be used to clean up unpacked git archives.
2005-07-03 23:27:34 +02:00
}
Create object subdirectories on demand This makes it possible to have a "sparse" git object subdirectory structure, something that has become much more attractive now that people use pack-files all the time. As a result of pack-files, a git object directory doesn't necessarily have any individual objects lying around, and in that case it's just wasting space to keep the empty first-level object directories around: on many filesystems the 256 empty directories will be aboue 1MB of diskspace. Even more importantly, after you re-pack a project that _used_ to be unpacked, you could be left with huge directories that no longer contain anything, but that waste space and take time to look through. With this change, "git prune-packed" can just do an rmdir() on the directories, and they'll get removed if empty, and re-created on demand. This patch also tries to fix up "write_sha1_from_fd()" to use the new common infrastructure for creating the object files, closing a hole where we might otherwise leave half-written objects in the object database. [jc: I unoptimized the part that really removes the fan-out directories to ease transition. init-db still wastes 1MB of diskspace to hold 256 empty fan-outs, and prune-packed rmdir()'s the grown but empty directories, but runs mkdir() immediately after that -- reducing the saving from 150KB to 146KB. These parts will be re-introduced when everybody has the on-demand capability.] Signed-off-by: Linus Torvalds <torvalds@osdl.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-09 00:54:01 +02:00
pathname[len] = 0;
Create object subdirectories on demand (phase II) This removes the unoptimization. The previous round does not mind missing fan-out directories, but still makes sure they exist, lest older versions choke on a repository created/packed by it. This round does not play that nicely anymore -- empty fan-out directories are not created by init-db, and will stay removed by prune-packed. The prune command also removes empty fan-out directories. Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-09 11:30:17 +02:00
rmdir(pathname);
Add "git-prune-packed" that removes objects that exist in a pack. This, together with "git repack" can be used to clean up unpacked git archives.
2005-07-03 23:27:34 +02:00
}
Make git-prune-packed a bit more chatty. Steven Grimm noticed that git-repack's verbosity is inconsistent because pack-objects is chatty and prune-packed is not. This makes the latter a bit more chatty and gives -q option to squelch it. Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-01-13 00:00:13 +01:00
void prune_packed_objects(int opts)
Add "git-prune-packed" that removes objects that exist in a pack. This, together with "git repack" can be used to clean up unpacked git archives.
2005-07-03 23:27:34 +02:00
{
int i;
static char pathname[PATH_MAX];
const char *dir = get_object_directory();
int len = strlen(dir);
Teach prune-packed to use the standard progress meter Rather than reimplementing the progress meter logic and always showing 100 lines of output while pruning already packed objects we now use a delayed progress meter and only show it if there are enough objects to make us take a little while. Most users won't see the message anymore as it usually doesn't take very long to delete the already packed loose objects. This neatens the output of a git-gc or git-repack execution, which is especially important for a `git gc --auto` triggered from within another command. We perform the display_progress() call from within the very innermost loop in case we spend more than 1 second within any single object directory. This ensures that a progress_update event from the timer will still trigger in a timely fashion and allow the user to see the progress meter. While I'm in here I changed the message to be more descriptive of its actual task. "Removing unused objects" is a little scary for new users as they wonder where these unused objects came from and how they should avoid them. Truth is these objects aren't unused in the sense of what git-prune would call a dangling object, these are used but are just duplicates of things we have already stored in a packfile. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2007-10-19 06:08:37 +02:00
if (opts == VERBOSE)
make struct progress an opaque type This allows for better management of progress "object" existence, as well as making the progress display implementation more independent from its callers. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-10-30 19:57:32 +01:00
progress = start_progress_delay("Removing duplicate objects",
Teach prune-packed to use the standard progress meter Rather than reimplementing the progress meter logic and always showing 100 lines of output while pruning already packed objects we now use a delayed progress meter and only show it if there are enough objects to make us take a little while. Most users won't see the message anymore as it usually doesn't take very long to delete the already packed loose objects. This neatens the output of a git-gc or git-repack execution, which is especially important for a `git gc --auto` triggered from within another command. We perform the display_progress() call from within the very innermost loop in case we spend more than 1 second within any single object directory. This ensures that a progress_update event from the timer will still trigger in a timely fashion and allow the user to see the progress meter. While I'm in here I changed the message to be more descriptive of its actual task. "Removing unused objects" is a little scary for new users as they wonder where these unused objects came from and how they should avoid them. Truth is these objects aren't unused in the sense of what git-prune would call a dangling object, these are used but are just duplicates of things we have already stored in a packfile. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2007-10-19 06:08:37 +02:00
256, 95, 2);
Add "git-prune-packed" that removes objects that exist in a pack. This, together with "git repack" can be used to clean up unpacked git archives.
2005-07-03 23:27:34 +02:00
if (len > PATH_MAX - 42)
die("impossible object directory");
memcpy(pathname, dir, len);
if (len && pathname[len-1] != '/')
pathname[len++] = '/';
for (i = 0; i < 256; i++) {
DIR *d;
sprintf(pathname + len, "%02x/", i);
d = opendir(pathname);
if (!d)
Create object subdirectories on demand This makes it possible to have a "sparse" git object subdirectory structure, something that has become much more attractive now that people use pack-files all the time. As a result of pack-files, a git object directory doesn't necessarily have any individual objects lying around, and in that case it's just wasting space to keep the empty first-level object directories around: on many filesystems the 256 empty directories will be aboue 1MB of diskspace. Even more importantly, after you re-pack a project that _used_ to be unpacked, you could be left with huge directories that no longer contain anything, but that waste space and take time to look through. With this change, "git prune-packed" can just do an rmdir() on the directories, and they'll get removed if empty, and re-created on demand. This patch also tries to fix up "write_sha1_from_fd()" to use the new common infrastructure for creating the object files, closing a hole where we might otherwise leave half-written objects in the object database. [jc: I unoptimized the part that really removes the fan-out directories to ease transition. init-db still wastes 1MB of diskspace to hold 256 empty fan-outs, and prune-packed rmdir()'s the grown but empty directories, but runs mkdir() immediately after that -- reducing the saving from 150KB to 146KB. These parts will be re-introduced when everybody has the on-demand capability.] Signed-off-by: Linus Torvalds <torvalds@osdl.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-09 00:54:01 +02:00
continue;
Make git-prune-packed a bit more chatty. Steven Grimm noticed that git-repack's verbosity is inconsistent because pack-objects is chatty and prune-packed is not. This makes the latter a bit more chatty and gives -q option to squelch it. Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-01-13 00:00:13 +01:00
prune_dir(i, d, pathname, len + 3, opts);
Add "git-prune-packed" that removes objects that exist in a pack. This, together with "git repack" can be used to clean up unpacked git archives.
2005-07-03 23:27:34 +02:00
closedir(d);
}
relax usage of the progress API Since it is now OK to pass a null pointer to display_progress() and stop_progress() resulting in a no-op, then we can simplify the code and remove a bunch of lines by not making those calls conditional all the time. Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-10-30 19:57:33 +01:00
stop_progress(&progress);
Add "git-prune-packed" that removes objects that exist in a pack. This, together with "git repack" can be used to clean up unpacked git archives.
2005-07-03 23:27:34 +02:00
}
use declarations from builtin.h for builtin commands Signed-off-by: Matthias Kestenholz <matthias@spinlock.ch> Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-08-02 23:52:00 +02:00
int cmd_prune_packed(int argc, const char **argv, const char *prefix)
Add "git-prune-packed" that removes objects that exist in a pack. This, together with "git repack" can be used to clean up unpacked git archives.
2005-07-03 23:27:34 +02:00
{
int i;
Make git-prune-packed a bit more chatty. Steven Grimm noticed that git-repack's verbosity is inconsistent because pack-objects is chatty and prune-packed is not. This makes the latter a bit more chatty and gives -q option to squelch it. Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-01-13 00:00:13 +01:00
int opts = VERBOSE;
Add "git-prune-packed" that removes objects that exist in a pack. This, together with "git repack" can be used to clean up unpacked git archives.
2005-07-03 23:27:34 +02:00
for (i = 1; i < argc; i++) {
const char *arg = argv[i];
if (*arg == '-') {
Call prune-packed from "git prune" as well. Add -n (dryrun) flag to git-prune-packed, and call it from "git prune". Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-08-20 06:38:36 +02:00
if (!strcmp(arg, "-n"))
Make git-prune-packed a bit more chatty. Steven Grimm noticed that git-repack's verbosity is inconsistent because pack-objects is chatty and prune-packed is not. This makes the latter a bit more chatty and gives -q option to squelch it. Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-01-13 00:00:13 +01:00
opts |= DRY_RUN;
else if (!strcmp(arg, "-q"))
opts &= ~VERBOSE;
Call prune-packed from "git prune" as well. Add -n (dryrun) flag to git-prune-packed, and call it from "git prune". Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-08-20 06:38:36 +02:00
else
usage(prune_packed_usage);
continue;
Add "git-prune-packed" that removes objects that exist in a pack. This, together with "git repack" can be used to clean up unpacked git archives.
2005-07-03 23:27:34 +02:00
}
/* Handle arguments here .. */
usage(prune_packed_usage);
}
Make git-prune-packed a bit more chatty. Steven Grimm noticed that git-repack's verbosity is inconsistent because pack-objects is chatty and prune-packed is not. This makes the latter a bit more chatty and gives -q option to squelch it. Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-01-13 00:00:13 +01:00
prune_packed_objects(opts);
Add "git-prune-packed" that removes objects that exist in a pack. This, together with "git repack" can be used to clean up unpacked git archives.
2005-07-03 23:27:34 +02:00
return 0;
}
Copy permalink