This means we'll have a loose object when we encounter a symlink
but that's not the common case.
We also don't have to worry about svn:eol-style when using the
SVN libraries, either. So remove the code to deal with that.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Share the repack counter between branches when doing
multi-fetch.
Pass the -d flag to git repack by default. That's the
main reason we will want automatic pack generation, to
save space and improve disk cache performance. I won't
add -a by default since it can generate extremely large
packs that make RAM-starved systems unhappy.
We no longer generate the .git/svn/$GIT_SVN_ID/info/uuid
file, either. It was never read in the first place.
Check for and create .rev_db if we need to during fetch (in case
somebody manually blew away their .rev_db and wanted to start
over. Mainly makes debugging easier).
Croak with $? instead of $! if there's an error closing pipes
Quiet down some of the chatter, too.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
repo_path_split() is already pretty fast, and is already
optimized via caching.
We also don't need to create an exclude file if we're
relying on the SVN libraries.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
This is a very intrusive change, so I've beefed up the tests
significantly. Added 'full-test' a target to the Makefile,
to test different possible configurations. This is intended
for maintainers only. Users should only be concerned with
'test' succeeding.
We now have a very simple custom database format for handling
mapping of svn revisions => git commits. Of course, we're
not really using it yet, either.
Also disabled automatic branch-finding on new trees for now.
It's too easily broken. revisions_eq() function should be
helpful for branch detection.
Also removed an extra assertion in fetch_cmd() that wasn't
correctly done. This bug was found by full-test.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
This means we no longer have to deal with having bloated SVN
working copies around and we get a nice performance increase as
well because we don't have to exec the SVN binary and start a
new server connection each time.
Of course we have to manually manage memory with SVN::Pool
whenever we can, and hack around cases where SVN just eats
memory despite pools (I blame Perl, too). I would like to
keep memory usage as stable as possible during long fetch/commit
processes since I still use computers with only 256-512M RAM.
commit should always be faster with the SVN library code. The
SVN::Delta interface is leaky (or I'm not using it with pools
correctly), so I'm forking on every commit, but that doesn't
seem to hurt performance too much (at least on normal Unix/Linux
systems where fork() is pretty cheap).
fetch should be faster in most common cases, but probably not all.
fetches will be faster where client/server delta generation is
the bottleneck and not bandwidth. Of course, full-files are
generated server-side via deltas, too. Full files are always
transferred when they're updated, just like git-svnimport and
unlike command-line svn. I'm also hacking around memory leaks
(see comments) here by using some more forks.
I've tested fetch with http://, https://, file://, and svn://
repositories, so we should be reasonably covered in terms of
error handling for fetching.
Of course, we'll keep plain command-line svn compatibility as a
fallback for people running SVN 1.1 (I'm looking into library
support for 1.1.x SVN, too). If you want to force command-line
SVN usage, set GIT_SVN_NO_LIB=1 in your environment.
We also require two simultaneous connections (just like
git-svnimport), but this shouldn't be a problem for most
servers.
Less important commands:
show-ignore is slower because it requires repository
access, but -r/--revision <num> can be specified.
graft-branches may use more memory, but it's a
short-term process and is funky-filename-safe.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
This quick feature should make it easy to look up svn log
messages when svn users refer to -r/--revision numbers.
The following features from `svn log' are supported:
--revision=<n>[:<n>] - is supported, non-numeric args are not:
HEAD, NEXT, BASE, PREV, etc ...
-v/--verbose - just maps to --raw (in git log), so
it's completely incompatible with
the --verbose output in svn log
--limit=<n> - is NOT the same as --max-count,
doesn't count merged/excluded commits
--incremental - supported (trivial :P)
New features:
--show-commit - shows the git commit sha1, as well
--oneline - our version of --pretty=oneline
Any other arguments are passed directly to `git log'
Signed-off-by: Eric Wong <normalperson@yhbt.net>
New commands:
graft-branches - The most interesting command of the bunch. It
detects branches in SVN via various techniques (currently
regexes and file copies). It can be later extended to handle
svk and other properties people may use to track merges in svk.
Basically, merge tracking is not standardized at all in the SVN
world, and git grafts are perfect for dealing with this
situation.
Existing branch support (via tree matches) is only handled at
fetch time.
The following tow were originally implemented as shell scripts
several months ago, but I just decided to streamline things a
bit and added them to the main script.
multi-init - supports git-svnimport-like command-line syntax for
importing repositories that are layed out as recommended by the
SVN folks. This is a bit more tolerant than the git-svnimport
command-line syntax and doesn't require the user to figure out
where the repository URL ends and where the repository path
begins.
multi-fetch - runs fetch on all known SVN branches we're
tracking. This will NOT discover new branches (unlike
git-svnimport), so multi-init will need to be re-run (it's
idempotent).
Consider these three to be auxilliary commands (like
show-ignore, and rebuild) so their behavior won't receive as
much testing or scrutiny as the core commands (fetch and
commit).
Signed-off-by: Eric Wong <normalperson@yhbt.net>
This should help keep disk usage sane for large imports.
--repack takes an optional argument for the interval, it
defaults to 1000 if no argument is specified.
Arguments to --repack-flags are passed directly to git-repack.
No arguments are passed by default.
Idea stolen from git-cvsimport :)
Signed-off-by: Eric Wong <normalperson@yhbt.net>
While we're at it, read_repo_config has been added and
expanded to handle case where command-line arguments are
optional to Getopt::Long
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Since GIT_SVN_ID usage is probably going to become more
widespread <evil grin>, we won't run the chance of somebody
having a GIT_SVN_ID name that conflicts with one of the default
directories that already exist in $GIT_DIR (branches/tags).
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Sometimes I don't feel like downloading an entire tree again when
I actually decide a branch is worth tracking, so some users can
get around it more easily with this.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
By breaking the pipe read once we've seen a commit twice.
This should make -B/--branch-all-ref faster and usable on a
frequent basis.
We use topological order now for calling git-rev-list, and any
commit we've seen before should imply that all parents have been
seen (at least I hope that's the case for --topo-order).
Signed-off-by: Eric Wong <normalperson@yhbt.net>
This should make life easier for all those who type:
`git-rev-parse --symbolic --all | xargs -n1 echo -b`
every time they run git-svn fetch.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
If new revisions are fetched, that implies we haven't merged,
acked, or nacked them yet, and attempting to write the tree
we're committing means we'd silently clobber the newly fetched
changes.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
svn forces UTF-8 for commit messages, and with LC_ALL set to 'C'
it is unable to determine encoding of the git commit message.
Now we'll just assume the user has set LC_* correctly for
the commit message they're using.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
If we read the maximum size of our buffer into $buf, and the
last character is '\015', there's a chance that the character is
'\012', which means our regex won't work correctly. At the
worst case, this could introduce an extra newline into the code.
We'll now read an extra character if we see '\015' is the last
character in $buf.
We also forgot to recalculate the length of $buf after doing the
newline substitution, causing some files to appeare truncated.
We'll do that now and force byte semantics in length() for good
measure.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
svn has trouble parsing files with embedded '@' characters. For
example,
svn propget svn:keywords foo@bar.c
svn: Syntax error parsing revision 'bar.c'
I asked about this on #svn and the workaround suggested was to append
an explicit revision specifier:
svn propget svn:keywords foo@bar.c@BASE
This patch appends '@BASE' to the filename in all calls to 'svn
propget'.
Patch originally by Seth Falcon <sethfalcon@gmail.com>
Seth: signoff?
[ew: Made to work with older svn that don't support peg revisions]
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Some changes to the latest git.git made this test croak. So
we'll always just force everything when using a new branch.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
svn < 1.3.x would display changes to keywords lines as modified
if they aren't expanded in the working copy. We already check
for changes against the git tree here, so checking against the
svn one is probably excessive.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The dash installed on my Debian Sarge boxes don't seem to like
<<'' as a heredoc starter. Recent versions of dash do not need
this fix.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Unlike my earlier test patch, this also checks svn:eol-style and
makes sure it's applied to working copy updates. This is
definitely more correct than my original attempt at killing
keyword expansions, but I still haven't tested it enough to
know. Feedback would be much appreciated.
Also changed assert_svn_wc_clean() to only work on the svn
working copy. This requires a separate call to assert_tree() to
check wc integrity against git in preparation for another change
I'm planning.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* Clarify that 'init' requires an argument
* Remove instances of 'SVN_URL' in the manpage, it's not an
environment variable.
* Refer to 'Additional Fetch Arguments' when documenting 'fetch'
* document --authors-file / -A option
Thanks to Pavel Roskin and Seth Falcon for bringing these issues
to my attention.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Bugs like the last one could've been avoided if it weren't for
this...
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
fetching from repos without an authors-file defined was broken.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
contrib/git-svn/git-svn.txt:
added git-repo-config key names for options
fixed quoting of "git-svn-HEAD" in the manpage
use preformatted text for examples
contrib/git-svn/Makefile:
add target to generate HTML:
http://git-svn.yhbt.net/git-svn.html
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
repo-config keys are any of the long option names minus the '-'
characters
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
We chdir internally, so we need a consistent GIT_DIR variable.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
We should be safely able to import histories with thousands
of revisions without hogging up lots of memory.
With this, we lose the ability to autocorrect mistakes when
people specify revisions in reverse, but it's probably no longer
a problem since we only have one method of log parsing nowadays.
I've added an extra check to ensure that revision numbers do
increment.
Also, increment the version number to 0.11.0. I really should
just call it 1.0 soon...
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Because committing back to an SVN repository from different
machines can result in different lineages, two different
repositories running git-svn can result in different commit
SHA1s (but of the same tree). Sometimes trees that are tracked
independently are merged together (usually via children),
resulting in non-unique git-svn-id: lines in rev-list.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
It's only for repositories that were imported with very early
versions of git-svn. Unfortunately, some of those repos are out
in the wild already, so fix this warning.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Output a big warning if somebody actually has a pre-1.0 version
of svn that doesn't support it.
Thanks to Yann Dirson for reminding me it still existed
and attempting to re-enable it :)
I think I subconciously removed support for it earlier...
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
'svn info' doesn't work with URLs in svn <= 1.1. Now we
only run svn info in local directories.
As a side effect, this should also work better for 'init' off
directories that are no longer in the latest revision of the
repository.
svn checkout -r<revision> arguments are fixed.
Newer versions of svn (1.2.x) seem to need URL@REV as well as
-rREV to checkout a particular revision...
Add an example in the manpage of how to track directory that has
been moved since its initial revision.
A huge thanks to Yann Dirson for the bug reporting and testing
my original patch. Thanks also to Junio C Hamano for suggesting
a safer way to use git-rev-parse.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
I thought passing --stop-on-copy to svn would save us from all
the trouble svn-arch-mirror had with directory (project) copies.
I was wrong, there was one thing I overlooked.
If a tree was moved from /foo/trunk to /bar/foo/trunk with no
other changes in r10, but the last change was done in r5, the
Last Changed Rev (from svn info) in /bar/foo/trunk will still be
r5, even though the copy in the repository didn't exist until
r10.
Now, if we ever detect that the Last Changed Rev isn't what
we're expecting, we'll run svn diff and only croak if there are
differences between them.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
I ended up using GIT_SVN_ID far more than I ever thought I
would. Typing less is good.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
If it does change, we're screwed anyways as SVN will refuse to
commit or update. We also never access more than one SVN
repository per-invocation, so we can store it as a global, too.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
In a new repository, the initial fetch creates a master branch
if one does not exist so HEAD has something to point to.
It now creates a master at the end of the initial fetch run,
pointing to the latest revision. Previously it pointed to the
first revision imported, which is generally less useful.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Syntax is compatible with git-svnimport and git-cvsimport:
normalperson = Eric Wong <normalperson@yhbt.net>
If this option is specified and git-svn encounters an SVN
committer name that it cannot parse, it git-svn will abort.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
We regenerate and use git-svn-id: whenever we fetch or otherwise
commit to remotes/git-svn. We don't actually know what revision
number we'll commit to SVN at commit time, so this is useless.
It won't throw off things like 'rebuild', though, which knows to
only use the last instance of git-svn-id: in a log message
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* Fixed manually-edited commit messages not going to
remotes/git-svn on sequential commits after the sequential
commit optimization.
* format help correctly after adding 'show-ignore'
* sha1_short regexp matches down to 4 hex characters
(from git-rev-parse --short documentation)
* Print the first line of the commit message when we commit to
SVN next to the sha1.
* Document 'T' (type change) in the comments
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
I've said I don't like branches in Subversion, and I still don't.
This is a bit more flexible, though, as the argument for -b is any
arbitrary git head/tag reference.
This makes some things easier:
* Importing git history into a brand new SVN branch.
* Tracking multiple SVN branches via GIT_SVN_ID, even from multiple
repositories.
* Adding tags from SVN (still need to use GIT_SVN_ID, though).
* Even merge tracking is supported, if and only the heads end up with
100% equivalent tree objects. This is more stricter but more robust
and foolproof than parsing commit messages, imho.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
After reading a lengthy discussion on the list, I've come to the
conclusion that creating a 'remotes' directory in refs isn't
such a bad idea.
You can still branch from it by specifying remotes/git-svn (not
needing the leading 'refs/'), and the documentation has been
updated to reflect that.
The 'git-svn' part of the ref can of course be set to whatever
you want by using the GIT_SVN_ID environment variable, as
before.
I'm using refs/remotes/git-svn, and not going with something
like refs/remotes/git-svn/HEAD as it's redundant for Subversion
where there's zero distinction between branches and directories.
Run git-svn rebuild --upgrade to upgrade your repository to use
the new head. git-svn-HEAD must be manually deleted for safety
reasons.
Side note: if you ever (and I hope you never) want to run
git-update-refs on a 'remotes/' ref, make sure you have the
'refs/' prefix as you don't want to be clobbering your
'remotes/' in $GIT_DIR (where remote URLs are stored).
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Thanks to Nicolas Vilz <niv@iaglans.de> for noticing this.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
