Sometimes an import frontend may need to work with a temporary branch
which will actually contain many different branches over the life
of the import. This is especially useful when the frontend needs
to create a tag from a set of file versions which are otherwise
never a commit.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When generating a very large pack file (for example close to 1 GB
in size) it may be impossible for the kernel to find a contiguous
free range within a 32 bit address space for the mapping to be
located at. This is especially problematic on large imports where
there is a lot of malloc activity occuring within the same process
and the malloc'd regions may straddle the previously mapped regions,
thereby creating large holes in the address space.
So instead we map only 128 MB of the pack at any given time.
This will likely increase the number of times the file gets mapped
(with additional system time required to update the page tables
more frequently) but will allow the program to handle packs up to
4 GB in size.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
fast-import was encounting a GPF when it ran out of free tree_entry
objects but didn't know this was the cause because the last
tree_entry wasn't terminated with a NULL pointer. The missing NULL
pointer occurred when we allocated additional entries via xmalloc
but didn't set the last tree_entry's "next" pointer to NULL.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This option can be used to have a record of every commit, the mark
(if supplied) and branch name of the commit recorded into a log file
when the commit is generated. This log can be useful to verify the
results of an import as the commits can be compared to some source
repository matching commits through the mark value.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The marks table can be used by the frontend to load any commit after
the import and compare it to whatever data the frontend knows about
that commit. If the mark idnums can be easily correlated to some
reference source then its relatively trivial to compare the GIT
tree to the reference to verify the accuracy of the import.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
cvs2svn has three phases: begin_commit, middle_commit, end_commit.
The ancester is computed in the middle_commit phase. So its easier
to generate a stream if the from command appears after the commit
message itself but before the file change commands.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Jon Smirl was finding it difficult to alter cvs2svn to generate
branch commands prior to the first commit of the same branch.
This change moves the 'from' command to be an optional parameter of
the 'commit' command, thereby allowing a new branch to be defined
at the moment it gets used to create the first commit on that branch.
This change makes it impossible to create a branch with no commits
on it as at least one commit is needed to register the branch.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Some architectures (e.g. SPARC) would require that we access pointers
only on pointer-sized alignments. So ensure the pool allocator
rounds out non-pointer sized allocations to the next pointer so we
don't generate bad memory addresses. This could have occurred if
we had previously allocated an atom whose string was not a whole
multiple of the pointer size, for example.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Tree reloading allows fast-import to swap out the least-recently used
branch by simply deallocating the data structures from memory that
were associated with that branch. Later if the branch becomes active
again it can lazily recreate those structures on demand by reloading
the necessary trees from the pack file it originally wrote them to.
The reloading process is implemented by mmap'ing the pack into
memory and using a much tighter variant of the pack reading code
contained in sha1_file.c. This was a blatent copy from sha1_file.c
but the unpacking functions were significantly simplified and are
actually now in a form that should make it easier to map only the
necessary regions of a pack rather than the entire file.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Tags received from the frontend are generated in memory in a simple
linked list in the order that the tag commands were sent by the
frontend. If multiple different tag objects for the same tag name
get generated the last one sent by the frontend will be the one
that gets written out at termination. Multiple tag objects for
the same name will cause all older tags of the same name to be lost.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If the branch load count exceeds the number of branches created then
the frontend is causing fast-import to page branches into and out of
memory due to the way its ordering its commits. Performance can
likely be increased if the frontend were to alter its commit
sequence such that it stays on one branch before switching to another
branch, then never returns to the prior branch.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Marks are now saved when the mark directive gets used by the frontend
and may be used in place of a SHA1 expression to locate a previous
SHA1 which fast-import may have generated. This is particularly
useful with commits where the frontend does not (easily) have the
ability to compute the SHA1 for an arbitrary commit but needs it
to generate a branch or tag from that commit.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The following command line options are now accepted before the
pack name:
--objects=n # replaces the object count after the pack name
--depth=n # delta chain depth to use (default is 10)
--active-branches=n # maximum number of branches to keep in memory
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Growing a tree caused all subtrees to be deallocated and put back
into the free list yet those subtree's contents were still actively
in use. Consequently they were doled out again and got stomped
on elsewhere. Releasing a tree is now performed in two parts,
either releasing only the content array or releasing the content
array and recursively releasing the subtree(s).
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If a frontend is smart enough to import a symlink then we should
let them do so. We'll assume that they were smart enough to first
generate a blob to hold the link target, as that's how symlinks
get represented in GIT.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Frontend clients can now send a text stream to fast-import rather
than a binary stream. This should facilitate developing frontend
software as the data stream is easier to view, manipulate and debug
my hand and Mark-I eyeball.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When accepting revision SHA1 IDs from the frontend verify the SHA1
actually refers to a blob and is known to exist. Its an error
to use a SHA1 in a tree if the blob doesn't exist as this would
cause git-fsck-objects to report a missing blob should the pack get
closed without the blob being appended into it or a subsequent pack.
So right now we'll just ask that the frontend "pre-declare" any
blobs it wants to use in a tree before it can use them.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The tree of the current commit can be altered by file_change commands
before the commit gets written to the pack. The file changes are
rather primitive as they simply allow removal of a tree entry or
setting/adding a tree entry.
Currently trees and commits aren't being deltafied when written to
the pack and branch reloading from the current pack doesn't work,
so at most 5 branches can be worked with at any one time.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This provides the basic data structures needed to store trees in
memory while we are processing them for a branch. What we are
attempting to do is track one complete tree for each branch that
the frontend has registered with us through the 'newb' (new_branch)
command. When the frontend edits that tree through 'updf' or 'delf'
commands we'll mark the affected tree(s) as being dirty and recompute
their objects during 'comt' (commit).
Currently the protocol is decidedly _not_ user friendly. I crashed
fast-import by giving it bad input data from Perl. I may try to
improve upon it, or at least upon its error handling.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Moved the new_blob logic off into a new subroutine and
invoked it when getting the 'blob' command.
Added statistics dump to STDERR when the program terminates listing
what it did at a high level. This is somewhat interesting.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Too many globals variables were being used not not enough
code was resuable to process trees and commits so this is
a simple refactoring of the existing blob processing code
to get into a state that will be easier to handle trees
and commits in.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Although its easy to ask the user to tell us how many objects they
will need, its probably better to dynamically grow the object table
in large units. But if the user can give us a hint as to roughly
how many objects then we can still use it during startup.
Also stopped printing the SHA1 strings to stdout as no user is
currently making use of that facility.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Suggesting the use of [-a|-i|-o] with git-commit is unnecessarily
complex and confusing. In this context -o is totally useless and -i
requires extra arguments which are not mentioned. The only sensible
hint (besides reading the man page but let's not go there) is
"commit -a".
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This adds tests for "cherry-pick" and "rebase --merge" (and
indirectly "commit -C" since it is used in the latter) to make
sure they create a new commit with correct encoding.
Signed-off-by: Junio C Hamano <junkio@cox.net>
The following commands can reuse log message from an existing
commit while creating a new commit:
git-cherry-pick
git-rebase (both with and without --merge)
git-commit (-c and -C)
When the original commit was made in a different encoding from
the current i18n.commitencoding, "cat-file commit" would give a
string that is inconsistent with what the resulting commit will
claim to be in. Replace them with "git show -s --encoding".
"git-rebase" without --merge is "git format-patch" piped to "git
am" in essence, and has been taken care of before this commit.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This checks combinations of i18n.commitencoding (declares what
encoding you are feeding commit-tree to make commits) and
i18n.logoutputencoding (instructs what encoding to emit the
commit message out to log output, including e-mail format) to
make sure the "format-patch | am" pipe used in git-rebase works
correctly.
I suspect "git cherry-pick" and "git rebase --merge" may fail
similar tests. We'll see.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Some of the recent changes and shortcuts to the tests broke
things for people using older versions of svn:
t9104-git-svn-follow-parent.sh:
v1.2.3 (from SuSE 10.0 as reported by riddochc on #git
(thanks!)) required an extra 'svn up'. I was also able to
reproduce this with v1.1.4 (Debian Sarge).
lib-git-svn.sh:
SVN::Repos bindings in versions up to and including 1.1.4
(Sarge again) do not pass fs-config options to the underlying
library. BerkeleyDB repositories also seem completely broken
on all my Sarge machines; so not using FSFS does not seem to
be an option for most people.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Make gitk use git-rev-parse --git-dir to find the repository.
Signed-off-by: Peter Baumann <siprbaum@stud.informatik.uni-erlangen.de>
Signed-off-by: Paul Mackerras <paulus@samba.org>
This can help people keep track of which gitk is which, when they
have several on the screen.
Signed-off-by: Doug Maxey <dwm@enoyolf.org>
Signed-off-by: Paul Mackerras <paulus@samba.org>
This updates five commands (merge, pull, rebase, revert and cherry-pick)
so that they can be started from a subdirectory.
This may not actually be what we want to do. These commands are
inherently whole-tree operations, and an inexperienced user may
mistakenly expect a "git pull" from a subdirectory would merge
only the subdirectory the command started from.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This converts scripts that do "cd $(rev-parse --show-cdup)" by
hand to use cd_to_toplevel.
I think git-fetch does not have to go to the toplevel, but that
should be dealt with in a separate patch.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Steven Grimm noticed that git-repack's verbosity is inconsistent
because pack-objects is chatty and prune-packed is not. This
makes the latter a bit more chatty and gives -q option to
squelch it.
Signed-off-by: Junio C Hamano <junkio@cox.net>
While 'init-db' still is and probably will always remain a valid git
command for obvious backward compatibility reasons, it would be a good
idea to move shipped tools and docs to using 'init' instead.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Andy Parkins noticed that the error message some "whole tree"
oriented commands emit is stated misleadingly when they refused
to run from a subdirectory.
We could probably allow some of them to work from a subdirectory
but that is a semantic change that could have unintended side
effects, so let's start at first by rewording the error message
to be easier to read without doing anything else to be safe.
Signed-off-by: Junio C Hamano <junkio@cox.net>
It is not available in the outermost merge, and it is only
useful for debugging merge-recursive in the inner merges.
Sergey Vlasov noticed that the old code accesses an
uninitialized location.
Signed-off-by: Junio C Hamano <junkio@cox.net>
The code previously checked it's own name and called 'die' upon
an error. However 'die' was not yet defined because git-sh-setup
had not been sourced yet. Instead simply write the error message
to stderr and exit with an error as was originally desired.
Signed-off-by: Bob Proulx <bob@proulx.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Despite what the documentation claims, git-commit does not check commit
for suspicious lines: all hooks are disabled by default,
and the pre-comit hook could be changed to do something else.
Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The "read_or_die()" function would silently NOT die for a partial read,
and since it was of type "void" it obviously couldn't even return the
partial number of bytes read.
IOW, it was totally broken. This hopefully fixes it up.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
With the new-and-improved write_in_full() semantics, where a partial write
simply always returns a real error (and always sets 'errno' when that
happens, including for the disk full case), a lot of the callers of
write_in_full() were just unnecessarily complex.
In particular, there's no reason to ever check for a zero length or
return: if the length was zero, we'll return zero, otherwise, if a disk
full resulted in the actual write() system call returning zero the
write_in_full() logic would have correctly turned that into a negative
return value, with 'errno' set to ENOSPC.
I really wish every "write_in_full()" user would just check against "<0"
now, but this fixes the nasty and stupid ones.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
