This is just an update for people being interested. Alex and me were
busy with that project for a few days now. While it has progressed nicely,
there are quite a couple TODOs in merge-recursive.c, just search for "TODO".
For impatient people: yes, it passes all the tests, and yes, according
to the evil test Alex did, it is faster than the Python script.
But no, it is not yet finished. Biggest points are:
- there are still three external calls
- in the end, it should not be necessary to write the index more than once
(just before exiting)
- a lot of things can be refactored to make the code easier and shorter
BTW we cannot just plug in git-merge-tree yet, because git-merge-tree
does not handle renames at all.
This patch is meant for testing, and as such,
- it compile the program to git-merge-recur
- it adjusts the scripts and tests to use git-merge-recur instead of
git-merge-recursive
- it provides "TEST", a script to execute the tests regarding -recursive
- it inlines the changes to read-cache.c (read_cache_from(), discard_cache()
and refresh_cache_entry())
Brought to you by Alex Riesen and Dscho
Signed-off-by: Junio C Hamano <junkio@cox.net>
* lt/merge-tree:
Improved three-way blob merging code
Prepare "git-merge-tree" for future work
xdiff: generate "anti-diffs" aka what is common to two files
* ew/svn:
Fix some doubled word typos
Typofix in Makefile comment.
Makefile: export NO_SVN_TESTS
git-svn: migrate out of contrib (follow-up)
git-svn: migrate out of contrib
This actually removes the objects to be pruned, unless you specify "-n"
(at which point it will just tell you which files it would prune).
This doesn't do the pack-file pruning that the shell-script used to do,
but if somebody really wants to, they could add it easily enough. I wonder
how useful it is, though, considering that "git repack -a -d" is just a
lot more efficient and generates a better end result.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Some GIT's shell script are using bare 'perl' for perl invocation.
Use @@PERL@@ symbol and replace it with PERL_PATH_SQ everywhere.
Signed-off-by: Michal Rokos <michal.rokos@nextsoft.cz>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Some implementations of sed (like HP-UX one) mandate a space between 'r'
and 'rfile'.
Signed-off-by: Michal Rokos <michal.rokos@nextsoft.cz>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Without this patch, it really is not sufficient to define NO_SVN_TESTS
in config.mak or the Makefile.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Allow NO_SVN_TESTS to be defined to skip git-svn tests. These
tests are time-consuming due to SVN being slow, and even more so
if SVN Perl libraries are not available.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* ew/instaweb:
instaweb: fix unportable ';' usage in sed
Makefile: replace ugly and unportable sed invocation
Add git-instaweb, instantly browse the working repo with gitweb
gitweb: Declare global variables with "our"
gitweb: Enable tree (directory) history display
gitweb: optimize per-file history generation
* jc/sha1:
A better-scheduled PPC SHA-1 implementation.
test-sha1: test hashing large buffer
Makefile: add framework to verify and bench sha1 implementations.
Although our "git-%$X:" implicit target had dependency on
$(GITLIBS) which included xdiff/lib.a, git-http-{fetch,push} had
their own building rules and with an obsolete dependency on
$(LIB_FILES). Update the rules to depend on $(GITLIBS), to make
parallel build work correctly.
Signed-off-by: Junio C Hamano <junkio@cox.net>
I got tired of having to configure gitweb for every repository
I work on. I sometimes prefer gitweb to standard GUIs like gitk
or gitview; so this lets me automatically configure gitweb to
browse my working repository and also opens my browser to it.
Updates from the original patch:
Added Apache/mod_perl2 compatibility if Dennis Stosberg's gitweb
has been applied, too: <20060621130708.Gcbc6e5c@leonov.stosberg.net>
General cleanups in shell code usage.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This fleshes out the code that generates a three-way merge of a set of
blobs.
It still actually does the three-way merge using an external executable
(ie just calling "merge"), but the interfaces have been cleaned up a lot
and are now fully based on the 'mmfile_t' interface, so if libxdiff were
to ever grow a compatible three-way-merge, it could probably be directly
plugged in.
It also uses the previous XDL_EMIT_COMMON functionality extension to
libxdiff to generate a made-up base file for the merge for the case where
no base file previously existed. This should be equivalent to what we
currently do in git-merge-one-file.sh:
diff -u -La/$orig -Lb/$orig $orig $src2 | git-apply --no-add
except it should be much simpler and can be done using the direct libxdiff
interfaces.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This cleans up the use of safe_strncpy() even more. Since it has the
same semantics as strlcpy() use this name instead. Also move the
definition from inside path.c to its own file compat/strlcpy.c, and use
it conditionally at compile time, since some platforms already has
strlcpy(). It's included in the same way as compat/setenv.c.
Signed-off-by: Peter Eriksen <s022018@student.dtu.dk>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* js/lsfix:
Initialize lock_file struct to all zero.
Make git-update-ref a builtin
Make git-update-index a builtin
Make git-stripspace a builtin
Make git-mailinfo a builtin
Make git-mailsplit a builtin
Make git-write-tree a builtin
* jc/upload-corrupt:
daemon: send stderr to /dev/null instead of closing.
upload-pack/fetch-pack: support side-band communication
Retire git-clone-pack
upload-pack: prepare for sideband message support.
upload-pack: avoid sending an incomplete pack upon failure
Before this patch, -DDEFAULT_GIT_TEMPLATE_DIR was passed on compilation
command line to all and every .c file compiled. In fact the macro
is used by only one .c file, and unused by all other .c files.
Remove -DDEFAULT_GIT_TEMPLATE_DIR where unused. Follow the example of
exec_cmd.o. Pass -DDEFAULT_GIT_TEMPLATE_DIR only where actually used.
Signed-off-by: Yakov Lerner <iler.ml@gmail.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The program is not used by git-clone since git-fetch-pack was extended
to allow its caller do what git-clone-pack alone did, and git-clone was
updated to use it.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This creates a simple specialized object allocator for basic
objects.
This avoids wasting space with malloc overhead (metadata and
extra alignment), since the specialized allocator knows the
alignment, and that objects, once allocated, are never freed.
It also allows us to track some basic statistics about object
allocations. For example, for the mozilla import, it shows
object usage as follows:
blobs: 627629 (14710 kB)
trees: 1119035 (34969 kB)
commits: 196423 (8440 kB)
tags: 1336 (46 kB)
and the simpler allocator shaves off about 2.5% off the memory
footprint off a "git-rev-list --all --objects", and is a bit
faster too.
[ Side note: this concludes the series of "save memory in object storage".
The thing is, there simply isn't much more to be saved on the objects.
Doing "git-rev-list --all --objects" on the mozilla archive has a final
total RSS of 131498 pages for me: that's about 513MB. Of that, the
object overhead is now just 56MB, the rest is going somewhere else (put
another way: the fact that this patch shaves off 2.5% of the total
memory overhead, considering that objects are now not much more than 10%
of the total shows how big the wasted space really was: this makes
object allocations much more memory- and time-efficient).
I haven't looked at where the rest is, but I suspect the bulk of it is
just the pack-file loading. It may be that we should pack the tree
objects separately from the blob objects: for git-rev-list --objects, we
don't actually ever need to even look at the blobs, but since trees and
blobs are interspersed in the pack-file, we end up not being dense in
the tree accesses, so we end up looking at more pages than we strictly
need to.
So with a 535MB pack-file, it's entirely possible - even likely - that
most of the remaining RSS is just the mmap of the pack-file itself. We
don't need to map in _all_ of it, but we do end up mapping a fair
amount. ]
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This shrinks "struct object" to the absolutely minimal size possible.
It now contains /only/ the object flags and the SHA1 hash name of the
object.
The "refs" field, which is really needed only for fsck, is maintained in
a separate hashed lookup-table, allowing all normal users to totally
ignore it.
This helps memory usage, although not as much as I hoped: it looks like
the allocation overhead of malloc (and the alignment constraints in
particular) means that while the structure size shrinks, the actual
allocation overhead mostly does not.
[ That said: memory usage is actually down, but not as much as it should
be: I suspect just one of the object types actually ended up shrinking
its effective allocation size.
To get to the next level, we probably need specialized allocators that
don't pad the allocation more than necessary. ]
The separation makes for some code cleanup, though, and makes the ref
tracking that fsck wants a clearly separate thing.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
git-tar-tree adds an extended pax header to archives if its first
parameter points to a commit. It confuses older tars and isn't
very useful in the case of git anyway, so stop doing it.
Idea: Junio, implementation: Junio. I just wrote it up. :-)
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Detect changed prefix and/or changed build flags in the middle
of the build (or between 'make' and 'make install'), and if change
is detected, make sure all objects are compiled with same build
flags and same prefix, thus avoiding inconsistent/broken build.
[jc: removed otherwise unnecessary Makefile target to test the
change this patch introduces. ]
Signed-off-by: Yakov Lerner <iler.ml@gmail.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
By being an internal command git-get-commit-id can make use of
struct ustar_header and other stuff and stops wasting precious
disk space.
Note: I recycled one of the two "tar-tree" entries instead of
splitting that cleanup into a separate patch.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The framework to create lockfiles that are removed at exit is
first used to reliably write the index file, but it is
applicable to other things, so stop calling it "cache_file".
This also rewords a few remaining error message that called the
index file "cache file".
Signed-off-by: Junio C Hamano <junkio@cox.net>
This ifdef's out more functions that are not used while !USE_MULTI
in http code. Also the dependency of http related objects on http.h
header file was missing in the Makefile.
Signed-off-by: Junio C Hamano <junkio@cox.net>
* jc/cache-tree: (26 commits)
builtin-rm: squelch compiler warnings.
git-write-tree writes garbage on sparc64
Fix crash when reading the empty tree
fsck-objects: do not segfault on missing tree in cache-tree
cache-tree: a bit more debugging support.
read-tree: invalidate cache-tree entry when a new index entry is added.
Fix test-dump-cache-tree in one-tree disappeared case.
fsck-objects: mark objects reachable from cache-tree
cache-tree: replace a sscanf() by two strtol() calls
cache-tree.c: typefix
test-dump-cache-tree: validate the cached data as well.
cache_tree_update: give an option to update cache-tree only.
read-tree: teach 1-way merege and plain read to prime cache-tree.
read-tree: teach 1 and 2 way merges about cache-tree.
update-index: when --unresolve, smudge the relevant cache-tree entries.
test-dump-cache-tree: report number of subtrees.
cache-tree: sort the subtree entries.
Teach fsck-objects about cache-tree.
index: make the index file format extensible.
cache-tree: protect against "git prune".
...
Conflicts:
Makefile, builtin.h, git.c: resolved the same way as in next.
Otherwise, if make is suspended, or killed with prejudice, or if the
system crashes, you could be left with an up-to-date, yet corrupt,
generated file.
I left off the `clean' addition, because I believe "make clean" should
not remove wildcard patterns like "*+", on the off-chance that someone
uses names like that for files they care about. Besides, in practice,
those temporary files are left behind so rarely that they're not a bother,
and they're removed again as part of the next build.
[jc: sign-off?]
Signed-off-by: Junio C Hamano <junkio@cox.net>
As both DESTDIR and the prefix are supposed to be absolute pathnames
they can simply be concatenated without an extra / (like in the main Makefile).
The extra slash may even break installation on Windows.
[jc: adjusted an earlier workaround for this problem in the dist-doc
target in the main Makefile as well. ]
Signed-off-by: Martin Waitz <tali@admingilde.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This makes "git format-patch" a built-in.
* js/fmt-patch:
git-rebase: use canonical A..B syntax to format-patch
git-format-patch: now built-in.
fmt-patch: Support --attach
fmt-patch: understand old <his> notation
Teach fmt-patch about --keep-subject
Teach fmt-patch about --numbered
fmt-patch: implement -o <dir>
fmt-patch: output file names to stdout
Teach fmt-patch to write individual files.
Use RFC2822 dates from "git fmt-patch".
git-fmt-patch: thinkofix to show [PATCH] properly.
rename internal format-patch wip
Minor tweak on subject line in --pretty=email
Tentative built-in format-patch.
This makes 'git add' and 'git rm' built-ins.
* lt/dirwalk:
Add builtin "git rm" command
Move pathspec matching from builtin-add.c into dir.c
Prevent bogus paths from being added to the index.
builtin-add: fix unmatched pathspec warnings.
Remove old "git-add.sh" remnants
builtin-add: warn on unmatched pathspecs
Do "git add" as a builtin
Clean up git-ls-file directory walking library interface
libify git-ls-files directory traversal
