It turns out "git clone" over rsync transport has been broken when
the source repository has packed references for a long time, and
nobody noticed nor complained about it.
* jk/drop-rsync-transport:
transport: drop support for git-over-rsync
The git-over-rsync protocol is inefficient and broken, and
has been for a long time. It transfers way more objects than
it needs (grabbing all of the remote's "objects/",
regardless of which objects we need). It does its own ad-hoc
parsing of loose and packed refs from the remote, but
doesn't properly override packed refs with loose ones,
leading to garbage results (e.g., expecting the other side
to have an object pointed to by a stale packed-refs entry,
or complaining that the other side has two copies of the
refs[1]).
This latter breakage means that nobody could have
successfully pulled from a moderately active repository
since cd547b4 (fetch/push: readd rsync support, 2007-10-01).
We never made an official deprecation notice in the release
notes for git's rsync protocol, but the tutorial has marked
it as such since 914328a (Update tutorial., 2005-08-30).
And on the mailing list as far back as Oct 2005, we can find
Junio mentioning it as having "been deprecated for quite
some time."[2,3,4]. So it was old news then; cogito had
deprecated the transport in July of 2005[5] (though it did
come back briefly when Linus broke git-http-pull!).
Of course some people professed their love of rsync through
2006, but Linus clarified in his usual gentle manner[6]:
> Thanks! This is why I still use rsync, even though
> everybody and their mother tells me "Linus says rsync is
> deprecated."
No. You're using rsync because you're actively doing
something _wrong_.
The deprecation sentiment was reinforced in 2008, with a
mention that cloning via rsync is broken (with no fix)[7].
Even the commit porting rsync over to C from shell (cd547b4)
lists it as deprecated! So between the 10 years of informal
warnings, and the fact that it has been severely broken
since 2007, it's probably safe to simply remove it without
further deprecation warnings.
[1] http://article.gmane.org/gmane.comp.version-control.git/285101
[2] http://article.gmane.org/gmane.comp.version-control.git/10093
[3] http://article.gmane.org/gmane.comp.version-control.git/17734
[4] http://article.gmane.org/gmane.comp.version-control.git/18911
[5] http://article.gmane.org/gmane.comp.version-control.git/5617
[6] http://article.gmane.org/gmane.comp.version-control.git/19354
[7] http://article.gmane.org/gmane.comp.version-control.git/103635
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Documentation/git.txt is not consistent in the way it
stylizes mentions of Environment Variables. Most of them are
enclosed in single quotes, some are enclosed in backticks,
some are not enclosed.
Signed-off-by: Andrey Rybak <rybak.a.v@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Previously, libcurl would follow redirection to any protocol
it was compiled for support with. This is desirable to allow
redirection from HTTP to HTTPS. However, it would even
successfully allow redirection from HTTP to SFTP, a protocol
that git does not otherwise support at all. Furthermore
git's new protocol-whitelisting could be bypassed by
following a redirect within the remote helper, as it was
only enforced at transport selection time.
This patch limits redirects within libcurl to HTTP, HTTPS,
FTP and FTPS. If there is a protocol-whitelist present, this
list is limited to those also allowed by the whitelist. As
redirection happens from within libcurl, it is impossible
for an HTTP redirect to a protocol implemented within
another remote helper.
When the curl version git was compiled with is too old to
support restrictions on protocol redirection, we warn the
user if GIT_ALLOW_PROTOCOL restrictions were requested. This
is a little inaccurate, as even without that variable in the
environment, we would still restrict SFTP, etc, and we do
not warn in that case. But anything else means we would
literally warn every time git accesses an http remote.
This commit includes a test, but it is not as robust as we
would hope. It redirects an http request to ftp, and checks
that curl complained about the protocol, which means that we
are relying on curl's specific error message to know what
happened. Ideally we would redirect to a working ftp server
and confirm that we can clone without protocol restrictions,
and not with them. But we do not have a portable way of
providing an ftp server, nor any other protocol that curl
supports (https is the closest, but we would have to deal
with certificates).
[jk: added test and version warning]
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If we are cloning an untrusted remote repository into a
sandbox, we may also want to fetch remote submodules in
order to get the complete view as intended by the other
side. However, that opens us up to attacks where a malicious
user gets us to clone something they would not otherwise
have access to (this is not necessarily a problem by itself,
but we may then act on the cloned contents in a way that
exposes them to the attacker).
Ideally such a setup would sandbox git entirely away from
high-value items, but this is not always practical or easy
to set up (e.g., OS network controls may block multiple
protocols, and we would want to enable some but not others).
We can help this case by providing a way to restrict
particular protocols. We use a whitelist in the environment.
This is more annoying to set up than a blacklist, but
defaults to safety if the set of protocols git supports
grows). If no whitelist is specified, we continue to default
to allowing all protocols (this is an "unsafe" default, but
since the minority of users will want this sandboxing
effect, it is the only sensible one).
A note on the tests: ideally these would all be in a single
test file, but the git-daemon and httpd test infrastructure
is an all-or-nothing proposition rather than a test-by-test
prerequisite. By putting them all together, we would be
unable to test the file-local code on machines without
apache.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* es/doc-clean-outdated-tools:
Documentation/git-tools: retire manually-maintained list
Documentation/git-tools: drop references to defunct tools
Documentation/git-tools: fix item text formatting
Documentation/git-tools: improve discoverability of Git wiki
Documentation/git: drop outdated Cogito reference
* es/doc-clean-outdated-tools:
Documentation/git-tools: retire manually-maintained list
Documentation/git-tools: drop references to defunct tools
Documentation/git-tools: fix item text formatting
Documentation/git-tools: improve discoverability of Git wiki
Documentation/git: drop outdated Cogito reference
Enhance packet tracing machinery to allow capturing an incoming
pack data to a file for debugging.
* jk/pkt-log-pack:
pkt-line: support tracing verbatim pack contents
pkt-line: tighten sideband PACK check when tracing
pkt-line: simplify starts_with checks in packet tracing
Cogito hasn't been maintained since late 2006, so drop the reference
to it. The warning that SCMS front-ends might override listed
environment variables, however, may still be valuable, so keep it but
generalize the wording.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Update to the "linked checkout" in 2.5.0-rc1.
Instead of "checkout --to" that does not do what "checkout"
normally does, move the functionality to "git worktree add".
As this makes the end-user experience of the "worktree add" more or
less complete, I am tempted to say we should cook the other topic
that removes the internal "new-worktree-mode" hack from "checkout"
a bit longer in 'next', and release 2.5 final without that one.
* es/worktree-add:
Documentation/git: fix stale "MULTIPLE CHECKOUT MODE" reference
worktree: caution that this is still experimental
Documentation/git-worktree: fix stale "git checkout --to" references
This should have been changed by 93a3649 (Documentation: move linked
worktree description from checkout to worktree, 2015-07-06).
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When debugging the pack protocol, it is sometimes useful to
store the verbatim pack that we sent or received on the
wire. Looking at the on-disk result is often not helpful for
a few reasons:
1. If the operation is a clone, we destroy the repo on
failure, leaving nothing on disk.
2. If the pack is small, we unpack it immediately, and the
full pack never hits the disk.
3. If we feed the pack to "index-pack --fix-thin", the
resulting pack has the extra delta bases added to it.
We already have a GIT_TRACE_PACKET mechanism for tracing
packets. Let's extend it with GIT_TRACE_PACKFILE to dump the
verbatim packfile.
There are a few other positive fallouts that come from
rearranging this code:
- We currently disable the packet trace after seeing the
PACK header, even though we may get human-readable lines
on other sidebands; now we include them in the trace.
- We currently try to print "PACK ..." in the trace to
indicate that the packfile has started. But because we
disable packet tracing, we never printed this line. We
will now do so.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
