1
0
Fork 0
mirror of https://github.com/git/git.git synced 2024-10-30 13:57:54 +01:00
git/builtin
Vicent Marti aa32939fea rev-list: add bitmap mode to speed up object lists
The bitmap reachability index used to speed up the counting objects
phase during `pack-objects` can also be used to optimize a normal
rev-list if the only thing required are the SHA1s of the objects during
the list (i.e., not the path names at which trees and blobs were found).

Calling `git rev-list --objects --use-bitmap-index [committish]` will
perform an object iteration based on a bitmap result instead of actually
walking the object graph.

These are some example timings for `torvalds/linux` (warm cache,
best-of-five):

    $ time git rev-list --objects master > /dev/null

    real    0m34.191s
    user    0m33.904s
    sys     0m0.268s

    $ time git rev-list --objects --use-bitmap-index master > /dev/null

    real    0m1.041s
    user    0m0.976s
    sys     0m0.064s

Likewise, using `git rev-list --count --use-bitmap-index` will speed up
the counting operation by building the resulting bitmap and performing a
fast popcount (number of bits set on the bitmap) on the result.

Here are some sample timings of different ways to count commits in
`torvalds/linux`:

    $ time git rev-list master | wc -l
        399882

        real    0m6.524s
        user    0m6.060s
        sys     0m3.284s

    $ time git rev-list --count master
        399882

        real    0m4.318s
        user    0m4.236s
        sys     0m0.076s

    $ time git rev-list --use-bitmap-index --count master
        399882

        real    0m0.217s
        user    0m0.176s
        sys     0m0.040s

This also respects negative refs, so you can use it to count
a slice of history:

        $ time git rev-list --count v3.0..master
        144843

        real    0m1.971s
        user    0m1.932s
        sys     0m0.036s

        $ time git rev-list --use-bitmap-index --count v3.0..master
        real    0m0.280s
        user    0m0.220s
        sys     0m0.056s

Though note that the closer the endpoints, the less it helps. In the
traversal case, we have fewer commits to cross, so we take less time.
But the bitmap time is dominated by generating the pack revindex, which
is constant with respect to the refs given.

Note that you cannot yet get a fast --left-right count of a symmetric
difference (e.g., "--count --left-right master...topic"). The slow part
of that walk actually happens during the merge-base determination when
we parse "master...topic". Even though a count does not actually need to
know the real merge base (it only needs to take the symmetric difference
of the bitmaps), the revision code would require some refactoring to
handle this case.

Additionally, a `--test-bitmap` flag has been added that will perform
the same rev-list manually (i.e. using a normal revwalk) and using
bitmaps, and verify that the results are the same. This can be used to
exercise the bitmap code, and also to verify that the contents of the
.bitmap file are sane.

Signed-off-by: Vicent Marti <tanoku@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-30 12:19:22 -08:00
..
add.c Merge branch 'fc/trivial' 2013-09-17 11:42:34 -07:00
annotate.c
apply.c Replace deprecated OPT_BOOLEAN by OPT_BOOL 2013-08-05 11:32:19 -07:00
archive.c
bisect--helper.c Replace deprecated OPT_BOOLEAN by OPT_BOOL 2013-08-05 11:32:19 -07:00
blame.c Merge branch 'jl/submodule-mv' 2013-09-09 14:36:15 -07:00
branch.c Merge branch 'jx/branch-vv-always-compare-with-upstream' 2013-09-20 12:26:57 -07:00
bundle.c
cat-file.c Merge branch 'mg/more-textconv' 2013-10-23 13:21:31 -07:00
check-attr.c Merge branch 'sb/parseopt-boolean-removal' 2013-09-04 12:39:03 -07:00
check-ignore.c Merge branch 'dw/check-ignore-sans-index' 2013-09-20 12:37:32 -07:00
check-mailmap.c builtin: add git-check-mailmap command 2013-07-13 10:19:37 -07:00
check-ref-format.c
checkout-index.c checkout-index: fix negations of even numbers of -n 2013-08-07 08:37:02 -07:00
checkout.c Merge branch 'nd/magic-pathspec' 2013-09-09 14:50:44 -07:00
clean.c Merge branch 'jl/submodule-mv' 2013-09-09 14:36:15 -07:00
clone.c Merge branch 'jk/clone-progress-to-stderr' 2013-10-18 13:49:51 -07:00
column.c
commit-tree.c
commit.c Merge branch 'mm/commit-template-squelch-advice-messages' 2013-09-20 12:36:32 -07:00
config.c Merge branch 'jk/config-int-range-check' 2013-09-12 14:41:00 -07:00
count-objects.c
credential.c
describe.c Merge branch 'rh/ishes-doc' 2013-09-17 11:42:51 -07:00
diff-files.c convert read_cache_preload() to take struct pathspec 2013-07-15 10:56:08 -07:00
diff-index.c convert read_cache_preload() to take struct pathspec 2013-07-15 10:56:08 -07:00
diff-tree.c
diff.c pathspec: support :(literal) syntax for noglob pathspec 2013-07-15 10:56:09 -07:00
fast-export.c Merge branch 'fc/fast-export' 2013-09-17 11:42:31 -07:00
fetch-pack.c Merge branch 'jc/push-cas' 2013-09-09 14:30:29 -07:00
fetch.c Merge branch 'bk/refs-multi-update' 2013-09-20 12:36:12 -07:00
fmt-merge-msg.c
for-each-ref.c for-each-ref, quote: convert *_quote_print -> *_quote_buf 2013-07-30 08:06:27 -07:00
fsck.c Merge branch 'jk/free-tree-buffer' 2013-09-17 11:37:33 -07:00
gc.c Merge branch 'sb/parseopt-boolean-removal' 2013-09-04 12:39:03 -07:00
grep.c Merge branch 'mg/more-textconv' 2013-10-23 13:21:31 -07:00
hash-object.c hash-object: replace stdin parsing OPT_BOOLEAN by OPT_COUNTUP 2013-08-07 08:30:55 -07:00
help.c Merge branch 'ph/builtin-srcs-are-in-subdir-these-days' into maint 2013-07-21 22:51:29 -07:00
index-pack.c Merge branch 'jk/free-tree-buffer' 2013-09-17 11:37:33 -07:00
init-db.c
log.c Merge branch 'mg/more-textconv' 2013-10-23 13:21:31 -07:00
ls-files.c Merge branch 'jc/ls-files-killed-optim' 2013-09-11 15:03:28 -07:00
ls-remote.c
ls-tree.c Merge branch 'jl/submodule-mv' 2013-09-09 14:36:15 -07:00
mailinfo.c
mailsplit.c
merge-base.c Replace deprecated OPT_BOOLEAN by OPT_BOOL 2013-08-05 11:32:19 -07:00
merge-file.c Replace deprecated OPT_BOOLEAN by OPT_BOOL 2013-08-05 11:32:19 -07:00
merge-index.c Convert "struct cache_entry *" to "const ..." wherever possible 2013-07-09 09:12:48 -07:00
merge-ours.c
merge-recursive.c
merge-tree.c merge-tree: handle directory/empty conflict correctly 2013-05-06 22:17:00 -07:00
merge.c Merge branch 'rh/ishes-doc' 2013-09-17 11:42:51 -07:00
mktag.c
mktree.c
mv.c Merge branch 'jl/submodule-mv' 2013-09-09 14:36:15 -07:00
name-rev.c branch, commit, name-rev: ease up boolean conditions 2013-08-07 08:30:30 -07:00
notes.c Replace deprecated OPT_BOOLEAN by OPT_BOOL 2013-08-05 11:32:19 -07:00
pack-objects.c pack-objects: use bitmaps when packing objects 2013-12-30 12:19:22 -08:00
pack-redundant.c
pack-refs.c
patch-id.c
prune-packed.c prune-packed: avoid implying "1" is DRY_RUN in prune_packed_objects() 2013-05-28 09:20:54 -07:00
prune.c Merge branch 'nd/prune-packed-dryrun-verbose' 2013-06-06 12:17:52 -07:00
push.c Merge branch 'nd/push-no-thin' 2013-09-11 14:56:59 -07:00
read-tree.c diff-lib, read-tree, unpack-trees: mark cache_entry array paramters const 2013-06-02 15:31:14 -07:00
receive-pack.c Merge branch 'bk/refs-multi-update' 2013-09-20 12:36:12 -07:00
reflog.c Merge branch 'bk/refs-multi-update' 2013-09-20 12:36:12 -07:00
remote-ext.c
remote-fd.c
remote.c Merge branch 'po/remote-set-head-usage' 2013-10-14 11:07:29 -07:00
repack.c repack: improve warnings about failure of renaming and removing files 2013-09-17 13:34:57 -07:00
replace.c Merge branch 'cc/replace-with-the-same-type' 2013-09-24 23:35:24 -07:00
rerere.c rerere: convert to use parse_pathspec 2013-07-15 10:56:07 -07:00
reset.c Merge branch 'jk/trailing-slash-in-pathspec' 2013-10-17 15:55:14 -07:00
rev-list.c rev-list: add bitmap mode to speed up object lists 2013-12-30 12:19:22 -08:00
rev-parse.c Merge branch 'sb/parseopt-boolean-removal' 2013-09-04 12:39:03 -07:00
revert.c Merge branch 'hu/cherry-pick-previous-branch' 2013-10-23 13:21:35 -07:00
rm.c rm: re-use parse_pathspec's trailing-slash removal 2013-09-13 12:37:35 -07:00
send-pack.c send-pack: fix parsing of --force-with-lease option 2013-08-02 16:07:45 -07:00
shortlog.c Merge branch 'jk/shortlog-tolerate-broken-commit' 2013-09-24 23:29:00 -07:00
show-branch.c Replace deprecated OPT_BOOLEAN by OPT_BOOL 2013-08-05 11:32:19 -07:00
show-ref.c Replace deprecated OPT_BOOLEAN by OPT_BOOL 2013-08-05 11:32:19 -07:00
stripspace.c builtin/stripspace.c: fix broken indentation 2013-09-06 13:33:17 -07:00
symbolic-ref.c
tag.c Merge branch 'bk/refs-multi-update' 2013-09-20 12:36:12 -07:00
tar-tree.c tar-tree: remove dependency on sq_quote_print() 2013-07-30 08:10:35 -07:00
unpack-file.c
unpack-objects.c
update-index.c Merge branch 'jl/submodule-mv' 2013-09-09 14:36:15 -07:00
update-ref.c Merge branch 'bk/refs-multi-update' 2013-09-20 12:36:12 -07:00
update-server-info.c
upload-archive.c
var.c
verify-pack.c
verify-tag.c
write-tree.c