Tree reloading allows fast-import to swap out the least-recently used
branch by simply deallocating the data structures from memory that
were associated with that branch. Later if the branch becomes active
again it can lazily recreate those structures on demand by reloading
the necessary trees from the pack file it originally wrote them to.
The reloading process is implemented by mmap'ing the pack into
memory and using a much tighter variant of the pack reading code
contained in sha1_file.c. This was a blatent copy from sha1_file.c
but the unpacking functions were significantly simplified and are
actually now in a form that should make it easier to map only the
necessary regions of a pack rather than the entire file.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Tags received from the frontend are generated in memory in a simple
linked list in the order that the tag commands were sent by the
frontend. If multiple different tag objects for the same tag name
get generated the last one sent by the frontend will be the one
that gets written out at termination. Multiple tag objects for
the same name will cause all older tags of the same name to be lost.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If the branch load count exceeds the number of branches created then
the frontend is causing fast-import to page branches into and out of
memory due to the way its ordering its commits. Performance can
likely be increased if the frontend were to alter its commit
sequence such that it stays on one branch before switching to another
branch, then never returns to the prior branch.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Marks are now saved when the mark directive gets used by the frontend
and may be used in place of a SHA1 expression to locate a previous
SHA1 which fast-import may have generated. This is particularly
useful with commits where the frontend does not (easily) have the
ability to compute the SHA1 for an arbitrary commit but needs it
to generate a branch or tag from that commit.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The following command line options are now accepted before the
pack name:
--objects=n # replaces the object count after the pack name
--depth=n # delta chain depth to use (default is 10)
--active-branches=n # maximum number of branches to keep in memory
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Growing a tree caused all subtrees to be deallocated and put back
into the free list yet those subtree's contents were still actively
in use. Consequently they were doled out again and got stomped
on elsewhere. Releasing a tree is now performed in two parts,
either releasing only the content array or releasing the content
array and recursively releasing the subtree(s).
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If a frontend is smart enough to import a symlink then we should
let them do so. We'll assume that they were smart enough to first
generate a blob to hold the link target, as that's how symlinks
get represented in GIT.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Frontend clients can now send a text stream to fast-import rather
than a binary stream. This should facilitate developing frontend
software as the data stream is easier to view, manipulate and debug
my hand and Mark-I eyeball.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When accepting revision SHA1 IDs from the frontend verify the SHA1
actually refers to a blob and is known to exist. Its an error
to use a SHA1 in a tree if the blob doesn't exist as this would
cause git-fsck-objects to report a missing blob should the pack get
closed without the blob being appended into it or a subsequent pack.
So right now we'll just ask that the frontend "pre-declare" any
blobs it wants to use in a tree before it can use them.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The tree of the current commit can be altered by file_change commands
before the commit gets written to the pack. The file changes are
rather primitive as they simply allow removal of a tree entry or
setting/adding a tree entry.
Currently trees and commits aren't being deltafied when written to
the pack and branch reloading from the current pack doesn't work,
so at most 5 branches can be worked with at any one time.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This provides the basic data structures needed to store trees in
memory while we are processing them for a branch. What we are
attempting to do is track one complete tree for each branch that
the frontend has registered with us through the 'newb' (new_branch)
command. When the frontend edits that tree through 'updf' or 'delf'
commands we'll mark the affected tree(s) as being dirty and recompute
their objects during 'comt' (commit).
Currently the protocol is decidedly _not_ user friendly. I crashed
fast-import by giving it bad input data from Perl. I may try to
improve upon it, or at least upon its error handling.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Moved the new_blob logic off into a new subroutine and
invoked it when getting the 'blob' command.
Added statistics dump to STDERR when the program terminates listing
what it did at a high level. This is somewhat interesting.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Too many globals variables were being used not not enough
code was resuable to process trees and commits so this is
a simple refactoring of the existing blob processing code
to get into a state that will be easier to handle trees
and commits in.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Although its easy to ask the user to tell us how many objects they
will need, its probably better to dynamically grow the object table
in large units. But if the user can give us a hint as to roughly
how many objects then we can still use it during startup.
Also stopped printing the SHA1 strings to stdout as no user is
currently making use of that facility.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Some of the recent changes and shortcuts to the tests broke
things for people using older versions of svn:
t9104-git-svn-follow-parent.sh:
v1.2.3 (from SuSE 10.0 as reported by riddochc on #git
(thanks!)) required an extra 'svn up'. I was also able to
reproduce this with v1.1.4 (Debian Sarge).
lib-git-svn.sh:
SVN::Repos bindings in versions up to and including 1.1.4
(Sarge again) do not pass fs-config options to the underlying
library. BerkeleyDB repositories also seem completely broken
on all my Sarge machines; so not using FSFS does not seem to
be an option for most people.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Steven Grimm noticed that git-repack's verbosity is inconsistent
because pack-objects is chatty and prune-packed is not. This
makes the latter a bit more chatty and gives -q option to
squelch it.
Signed-off-by: Junio C Hamano <junkio@cox.net>
While 'init-db' still is and probably will always remain a valid git
command for obvious backward compatibility reasons, it would be a good
idea to move shipped tools and docs to using 'init' instead.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Andy Parkins noticed that the error message some "whole tree"
oriented commands emit is stated misleadingly when they refused
to run from a subdirectory.
We could probably allow some of them to work from a subdirectory
but that is a semantic change that could have unintended side
effects, so let's start at first by rewording the error message
to be easier to read without doing anything else to be safe.
Signed-off-by: Junio C Hamano <junkio@cox.net>
It is not available in the outermost merge, and it is only
useful for debugging merge-recursive in the inner merges.
Sergey Vlasov noticed that the old code accesses an
uninitialized location.
Signed-off-by: Junio C Hamano <junkio@cox.net>
The code previously checked it's own name and called 'die' upon
an error. However 'die' was not yet defined because git-sh-setup
had not been sourced yet. Instead simply write the error message
to stderr and exit with an error as was originally desired.
Signed-off-by: Bob Proulx <bob@proulx.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Despite what the documentation claims, git-commit does not check commit
for suspicious lines: all hooks are disabled by default,
and the pre-comit hook could be changed to do something else.
Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The "read_or_die()" function would silently NOT die for a partial read,
and since it was of type "void" it obviously couldn't even return the
partial number of bytes read.
IOW, it was totally broken. This hopefully fixes it up.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
With the new-and-improved write_in_full() semantics, where a partial write
simply always returns a real error (and always sets 'errno' when that
happens, including for the disk full case), a lot of the callers of
write_in_full() were just unnecessarily complex.
In particular, there's no reason to ever check for a zero length or
return: if the length was zero, we'll return zero, otherwise, if a disk
full resulted in the actual write() system call returning zero the
write_in_full() logic would have correctly turned that into a negative
return value, with 'errno' set to ENOSPC.
I really wish every "write_in_full()" user would just check against "<0"
now, but this fixes the nasty and stupid ones.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Recently git.git itself encountered a situation on its master and
next branches where git-describe stopped reporting 'v1.5.0-rc0-gN'
and instead started reporting 'v1.4.4.4-gN'. This appeared to be
a backward jump in version numbering.
maint o-------------------4
\ \
master o-o-o-o-o-o-o-5-o-C-o-W
The issue is that commit C in the diagram claims it is version
1.5.0, as the tag v1.5.0 is placed on commit 5. Yet commit W
claims it is version 1.4.4.4 as the tag v1.5.0 has an older tag
date than the v1.4.4.4 tag.
As it turns out this situation is very common. A bug fix applied
to maint and later merged into master occurs frequently enough that
it should Just Work Right(tm).
Rather than taking the first tag that gets found git-describe will
now generate a list of all possible tags and select the one which
has the most number of commits in common with HEAD (or whatever
revision the user requested the description of).
This rule is based on the principle shown in the diagram above.
There are a large number of commits on the primary development branch
'master' which do not appear in the 'maint' branch, and many of
these are already tagged as part of v1.5.0-rc0. Additionally these
commits are not in v1.4.4.4, as they are part of the v1.5.0 release
still being developed. The v1.5.0-rc0 tag is more descriptive of
W than v1.4.4.4 is, and therefore should be used.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* jc/bare:
Disallow working directory commands in a bare repository.
git-fetch: allow updating the current branch in a bare repository.
Introduce is_bare_repository() and core.bare configuration variable
Move initialization of log_all_ref_updates
* jc/detached-head:
git-checkout: handle local changes sanely when detaching HEAD
git-checkout: safety check for detached HEAD checks existing refs
git-checkout: fix branch name output from the command
git-checkout: safety when coming back from the detached HEAD state.
git-checkout: rewording comments regarding detached HEAD.
git-checkout: do not warn detaching HEAD when it is already detached.
Detached HEAD (experimental)
git-branch: show detached HEAD
git-status: show detached HEAD
If you do:
$ /bin/rm foo
$ git status
we used to say "git add ... to add content to commit". But
suggsting "git add" to record the deletion of a file is simply
insane.
So this rewords various things:
- The section header is the old "Changed but not updated",
instead of "Changed but not added";
- Suggestion is "git add ... to update what will be committed",
instead of "... to add content to commit";
- If there are removed paths, the above suggestion becomes "git
add/rm ... to update what will be committed";
- For untracked files, the suggestion is "git add ... to
include in what will be committed".
Signed-off-by: Junio C Hamano <junkio@cox.net>
Often the user would do "/bin/rm foo" before telling git, but
then want to tell git about it. "git rm foo" however would fail
because it cannot unlink(2) foo.
Treat ENOENT error return from unlink(2) as if a successful
removal happened.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Unfortunately, while {read,write}_in_full do take into account
zero-sized reads/writes; their die and whine variants do not.
I have a repository where there are zero-sized files in
the history that was triggering these things.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This fixes another problem that Andy's case showed: git-fsck-objects
reports nonsensical results for corrupt objects.
There were actually two independent and confusing problems:
- when we had a zero-sized file and used map_sha1_file, mmap() would
return EINVAL, and git-fsck-objects would report that as an insane and
confusing error. I don't know when this was introduced, it might have
been there forever.
- when "parse_object()" returned NULL, fsck would say "object not found",
which can be very confusing, since obviously the object might "exist",
it's just unparseable because it's totally corrupt.
So this just makes "xmmap()" return NULL for a zero-sized object (which is
a valid thing pointer, exactly the same way "malloc()" can return NULL for
a zero-sized allocation). That fixes the first problem (but we could have
fixed it in the caller too - I don't personally much care whichever way it
goes, but maybe somebody should check that the NO_MMAP case does
something sane in this case too?).
And the second problem is solved by just making the error message slightly
clearer - the failure to parse an object may be because it's missing or
corrupt, not necessarily because it's not "found".
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
These days, the command does a lot more than just initialise the
object database (such as setting default config-variables,
installing template hooks...), and "git init" is actually a more
sensible name nowadays.
Signed-off-by: Junio C Hamano <junkio@cox.net>
If the user tries to run a porcelainish command which requires
a working directory in a bare repository they may get unexpected
results which are difficult to predict and may differ from command
to command.
Instead we should detect that the current repository is a bare
repository and refuse to run the command there, as there is no
working directory associated with it.
[jc: updated Shawn's original somewhat -- bugs are mine.]
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This revamps the merge-recursive implementation following the
outline in:
Message-ID: <7v8xgileza.fsf@assigned-by-dhcp.cox.net>
There is no need to write out the index until the very end just
once from merge-recursive. Also there is no need to write out
the resulting tree object for the simple case of merging with a
single merge base.
Signed-off-by: Junio C Hamano <junkio@cox.net>
The merge-recursive implementation in C inherited the invariant
that the on-file index file is written out and later read back
after any index operations and writing trees from the original
Python implementation. But it was only because the original
implementation worked at the scripting level.
There is no need to write out the index file after handling
every path.
Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Since 98bf8a47c2 status would claim that
git-commit could be useful even if there are no changes except untracked files.
Since wt-status is already computing all the information needed go the whole
way and actually track the (non-)emptiness of all three sections separately,
unify the code, and provide useful messages for each individual case.
Thanks to Junio and Michael Loeffler for suggestions.
Signed-off-by: Jürgen Rühle <j-r@online.de>
