1
0
Fork 0
mirror of https://github.com/git/git.git synced 2024-11-15 05:33:04 +01:00
Commit graph

132 commits

Author SHA1 Message Date
Jeff King
030e9dd64f fetch-pack: fix out-of-bounds buffer offset in get_ack
When we read acks from the remote, we expect either:

  ACK <sha1>

or

  ACK <sha1> <multi-ack-flag>

We parse the "ACK <sha1>" bit from the line, and then start
looking for the flag strings at "line+45"; if we don't have
them, we assume it's of the first type.  But if we do have
the first type, then line+45 is not necessarily inside our
string at all!

It turns out that this works most of the time due to the way
we parse the packets. They should come in with a newline,
and packet_read puts an extra NUL into the buffer, so we end
up with:

  ACK <sha1>\n\0

with the newline at offset 44 and the NUL at offset 45. We
then strip the newline, putting a NUL at offset 44. So
when we look at "line+45", we are looking past the end of
our string; but it's OK, because we hit the terminator from
the original string.

This breaks down, however, if the other side does not
terminate their packets with a newline. In that case, our
packet is one character shorter, and we start looking
through uninitialized memory for the flag. No known
implementation sends such a packet, so it has never come up
in practice.

This patch tightens the check by looking for a short,
flagless ACK before trying to parse the flag.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-02-20 13:42:21 -08:00
Junio C Hamano
6e7b66eebd fetch: fetch objects by their exact SHA-1 object names
Teach "git fetch" to accept an exact SHA-1 object name the user may
obtain out of band on the LHS of a pathspec, and send it on a "want"
message when the server side advertises the allow-tip-sha1-in-want
capability.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-02-07 14:07:53 -08:00
Junio C Hamano
f2db854d24 fetch: use struct ref to represent refs to be fetched
Even though "git fetch" has full infrastructure to parse refspecs to
be fetched and match them against the list of refs to come up with
the final list of refs to be fetched, the list of refs that are
requested to be fetched were internally converted to a plain list of
strings at the transport layer and then passed to the underlying
fetch-pack driver.

Stop this conversion and instead pass around an array of refs.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-02-07 13:53:59 -08:00
Junio C Hamano
4acfff9dda Merge branch 'jk/gc-auto-after-fetch'
Help "fetch only" repositories that do not trigger "gc --auto"
often enough.

* jk/gc-auto-after-fetch:
  fetch-pack: avoid repeatedly re-scanning pack directory
  fetch: run gc --auto after fetching
2013-02-01 12:40:16 -08:00
Junio C Hamano
012a1bb524 Merge branch 'jk/maint-gc-auto-after-fetch' into jk/gc-auto-after-fetch
* jk/maint-gc-auto-after-fetch:
  fetch-pack: avoid repeatedly re-scanning pack directory
  fetch: run gc --auto after fetching
2013-01-26 19:42:09 -08:00
Junio C Hamano
8cabd200d2 Merge branch 'mk/qnx'
Port to QNX.

* mk/qnx:
  Port to QNX
  Make lock local to fetch_pack
2013-01-03 10:28:33 -08:00
Nguyễn Thái Ngọc Duy
745f7a8cac fetch-pack: move core code to libgit.a
fetch_pack() is used by transport.c, part of libgit.a while it stays
in builtin/fetch-pack.c. Move it to fetch-pack.c so that we won't get
undefined reference if a program that uses libgit.a happens to pull it
in.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
2012-10-29 03:40:29 -04:00
Daniel Barkalow
2d4177c01c Make fetch-pack a builtin with an internal API
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-09-19 03:22:30 -07:00
Junio C Hamano
a6080a0a44 War on whitespace
This uses "git-apply --whitespace=strip" to fix whitespace errors that have
crept in to our source files over time.  There are a few files that need
to have trailing whitespaces (most notably, test vectors).  The results
still passes the test, and build result in Documentation/ area is unchanged.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-06-07 00:04:01 -07:00
Michael S. Tsirkin
7841ce7985 connect: display connection progress
Make git notify the user about host resolution/connection attempts.
This is useful both as a progress indicator on slow links, and helps
reassure the user there are no firewall problems.

Signed-off-by: Michael S. Tsirkin <mst@dev.mellanox.co.il>
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-05-16 12:48:18 -07:00
Junio C Hamano
3ddad98b74 Merge branch 'js/fetch-progress' (early part)
* 'js/fetch-progress' (early part):
  Fixup no-progress for fetch & clone
  fetch & clone: do not output progress when not on a tty

Conflicts:

	git-fetch.sh
2007-03-04 17:31:21 -08:00
Johannes Schindelin
b0e908977e Fixup no-progress for fetch & clone
The intent of the commit 'fetch & clone: do not output progress when
not on a tty' was to make fetching and cloning less chatty when
output was not redirected (such as in a cron job).

However, there was a serious thinko in that commit. It assumed that
the client _and_ the server got this update at the same time. But
this is obviously not the case, and therefore upload-pack died on
seeing the option "--no-progress".

This patch fixes that issue by making it a protocol option. So, until
your server is updated, you still see the progress, but once the
server has this patch, it will be quiet.

A minor issue was also fixed: when cloning, the checkout did not
heed no_progress.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-02-24 00:26:18 -08:00
Junio C Hamano
599065a3bb prefixcmp(): fix-up mechanical conversion.
Previous step converted use of strncmp() with literal string
mechanically even when the result is only used as a boolean:

    if (!strncmp("foo", arg, 3)) ==> if (!(-prefixcmp(arg, "foo")))

This step manually cleans them up to read:

    if (!prefixcmp(arg, "foo"))

Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-02-20 22:03:15 -08:00
Junio C Hamano
cc44c7655f Mechanical conversion to use prefixcmp()
This mechanically converts strncmp() to use prefixcmp(), but only when
the parameters match specific patterns, so that they can be verified
easily.  Leftover from this will be fixed in a separate step, including
idiotic conversions like

    if (!strncmp("foo", arg, 3))

  =>

    if (!(-prefixcmp(arg, "foo")))

This was done by using this script in px.perl

   #!/usr/bin/perl -i.bak -p
   if (/strncmp\(([^,]+), "([^\\"]*)", (\d+)\)/ && (length($2) == $3)) {
           s|strncmp\(([^,]+), "([^\\"]*)", (\d+)\)|prefixcmp($1, "$2")|;
   }
   if (/strncmp\("([^\\"]*)", ([^,]+), (\d+)\)/ && (length($1) == $3)) {
           s|strncmp\("([^\\"]*)", ([^,]+), (\d+)\)|(-prefixcmp($2, "$1"))|;
   }

and running:

   $ git grep -l strncmp -- '*.c' | xargs perl px.perl

Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-02-20 22:03:15 -08:00
Johannes Schindelin
83a5ad6126 fetch & clone: do not output progress when not on a tty
This adds the option "--no-progress" to fetch-pack and upload-pack,
and makes fetch and clone pass this option when stdout is not a tty.

While at documenting that option, also document --strict and --timeout
options for upload-pack.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-02-19 19:20:05 -08:00
Junio C Hamano
01754769ab Don't force everybody to call setup_ident().
Back when only handful commands that created commit and tag were
the only users of committer identity information, it made sense
to explicitly call setup_ident() to pre-fill the default value
from the gecos information.  But it is much simpler for programs
to make the call automatic when get_ident() is called these days,
since many more programs want to use the information when updating
the reflog.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-01-28 01:58:50 -08:00
Junio C Hamano
e28714c527 Consolidate {receive,fetch}.unpackLimit
This allows transfer.unpackLimit to specify what these two
configuration variables want to set.

We would probably want to deprecate the two separate variables,
as I do not see much point in specifying them independently.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-01-24 18:08:02 -08:00
Junio C Hamano
af7cf268f0 fetch-pack: remove --keep-auto and make it the default.
This makes git-fetch over git native protocol to automatically
decide to keep the downloaded pack if the fetch results in more
than 100 objects, just like receive-pack invoked by git-push
does.  This logic is disabled when --keep is explicitly given
from the command line, so that a very small clone still keeps
the downloaded pack as before.

The 100 threshold can be adjusted with fetch.unpacklimit
configuration.  We might want to introduce transfer.unpacklimit
to consolidate the two unpacklimit variables, which will be a
topic for the next patch.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-01-24 18:08:02 -08:00
Junio C Hamano
9e10fd1ac0 Allow fetch-pack to decide keeping the fetched pack without exploding
With --keep-auto option, fetch-pack decides to keep the pack
without exploding it just like receive-pack does.

We may want to later make this the default.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-01-24 18:08:02 -08:00
Uwe Kleine-König
27dca07fb7 rename --exec to --upload-pack for fetch-pack and peek-remote
Just some option name disambiguation.  This is the counter part to
commit d23842fd which made a similar change for push and send-pack.

--exec continues to work.

Signed-off-by: Uwe Kleine-König <ukleinek@informatik.uni-freiburg.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-01-24 16:12:15 -08:00
Uwe Kleine-König
18bd8821ca Update documentation of fetch-pack, push and send-pack
add all supported options to Documentation/git-....txt and the usage strings.

Signed-off-by: Uwe Kleine-König <zeisberg@informatik.uni-freiburg.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-01-19 17:54:32 -08:00
Junio C Hamano
54b9e0225a fetch-pack: do not use lockfile structure on stack.
They are used in atexit() for clean-up, and you will be
accessing unallocated memory at that point.

See 31f584c2 for the fix for a similar problem.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2007-01-02 11:22:08 -08:00
Junio C Hamano
37818d7db0 Merge branch 'master' into js/shallow
This is to adjust to:

  count-objects -v: show number of packs as well.

which will break a test in this series.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-12-27 02:43:46 -08:00
Junio C Hamano
85023577a8 simplify inclusion of system header files.
This is a mechanical clean-up of the way *.c files include
system header files.

 (1) sources under compat/, platform sha-1 implementations, and
     xdelta code are exempt from the following rules;

 (2) the first #include must be "git-compat-util.h" or one of
     our own header file that includes it first (e.g. config.h,
     builtin.h, pkt-line.h);

 (3) system headers that are included in "git-compat-util.h"
     need not be included in individual C source files.

 (4) "git-compat-util.h" does not have to include subsystem
     specific header files (e.g. expat.h).

Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-12-20 09:51:35 -08:00
Junio C Hamano
310b86d480 fetch-pack: do not barf when duplicate re patterns are given
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-11-25 01:33:06 -08:00
Alexandre Julliard
4bcb310c25 fetch-pack: Do not fetch tags for shallow clones.
A better fix may be to only fetch tags that point to commits that we
are downloading, but git-clone doesn't have support for following
tags. This will happen automatically on the next git-fetch though.

Signed-off-by: Alexandre Julliard <julliard@winehq.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-11-24 15:42:50 -08:00
Alexandre Julliard
d6491e3a21 fetch-pack: Properly remove the shallow file when it becomes empty.
The code was unlinking the lock file instead.

Signed-off-by: Alexandre Julliard <julliard@winehq.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-11-24 15:42:49 -08:00
Junio C Hamano
fcd1e31906 Why does it mean we do not have to register shallow if we have one? 2006-11-24 15:42:49 -08:00
Junio C Hamano
cf01bd52ef We should make sure that the protocol is still extensible.
This just reformats if .. else if .. else chain to make it clear we
are handling extended response from the other end.
2006-11-24 15:42:49 -08:00
Johannes Schindelin
f53514bc2d allow deepening of a shallow repository
Now, by saying "git fetch -depth <n> <repo>" you can deepen
a shallow repository.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-11-24 15:42:49 -08:00
Johannes Schindelin
016e6ccbe0 allow cloning a repository "shallowly"
By specifying a depth, you can now clone a repository such that
all fetched ancestor-chains' length is at most "depth". For example,
if the upstream repository has only 2 branches ("A" and "B"), which
are linear, and you specify depth 3, you will get A, A~1, A~2, A~3,
B, B~1, B~2, and B~3. The ends are automatically made shallow
commits.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-11-24 15:42:49 -08:00
Johannes Schindelin
ed09aef06f support fetching into a shallow repository
A shallow commit is a commit which has parents, which in turn are
"grafted away", i.e. the commit appears as if it were a root.

Since these shallow commits should not be edited by the user, but
only by core git, they are recorded in the file $GIT_DIR/shallow.

A repository containing shallow commits is called shallow.

The advantage of a shallow repository is that even if the upstream
contains lots of history, your local (shallow) repository needs not
occupy much disk space.

The disadvantage is that you might miss a merge base when pulling
some remote branch.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-11-24 15:42:49 -08:00
Nicolas Pitre
da093d3750 improve fetch-pack's handling of kept packs
Since functions in fetch-clone.c were only used from fetch-pack.c,
its content has been merged with fetch-pack.c.  This allows for better
coupling of features with much simpler implementations.

One new thing is that the (abscence of) --thin also enforce it on
index-pack now, such that index-pack will abort if a thin pack was
_not_ asked for.

The -k or --keep, when provided twice, now causes the fetched pack
to be left as a kept pack just like receive-pack currently does.
Eventually this will be used to close a race against concurrent
repacking.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-11-03 00:24:07 -08:00
Junio C Hamano
407e1d6e12 Merge branch 'master' into np/index-pack
* master: (90 commits)
  gitweb: Better support for non-CSS aware web browsers
  gitweb: Output also empty patches in "commitdiff" view
  gitweb: Use git-for-each-ref to generate list of heads and/or tags
  for-each-ref: "creator" and "creatordate" fields
  Add --global option to git-repo-config.
  pack-refs: Store the full name of the ref even when packing only tags.
  git-clone documentation didn't mention --origin as equivalent of -o
  Minor grammar fixes for git-diff-index.txt
  link_temp_to_file: call adjust_shared_perm() only when we created the directory
  Remove uneccessarily similar printf() from print_ref_list() in builtin-branch
  pack-objects doesn't create random pack names
  branch: work in subdirectories.
  gitweb: Use 's' regexp modifier to secure against filenames with LF
  gitweb: Secure against commit-ish/tree-ish with the same name as path
  gitweb: esc_html() author in blame
  git-svnimport: support for partial imports
  link_temp_to_file: don't leave the path truncated on adjust_shared_perm failure
  Move deny_non_fast_forwards handling completely into receive-pack.
  revision traversal: --unpacked does not limit commit list anymore.
  Continue traversal when rev-list --unpacked finds a packed commit.
  ...
2006-11-03 00:23:52 -08:00
Junio C Hamano
58a1e0e83b Merge branch 'lj/refs'
* lj/refs: (63 commits)
  Fix show-ref usagestring
  t3200: git-branch testsuite update
  sha1_name.c: avoid compilation warnings.
  Make git-branch a builtin
  ref-log: fix D/F conflict coming from deleted refs.
  git-revert with conflicts to behave as git-merge with conflicts
  core.logallrefupdates thinko-fix
  git-pack-refs --all
  core.logallrefupdates create new log file only for branch heads.
  Remove bashism from t3210-pack-refs.sh
  ref-log: allow ref@{count} syntax.
  pack-refs: call fflush before fsync.
  pack-refs: use lockfile as everybody else does.
  git-fetch: do not look into $GIT_DIR/refs to see if a tag exists.
  lock_ref_sha1_basic does not remove empty directories on BSD
  Do not create tag leading directories since git update-ref does it.
  Check that a tag exists using show-ref instead of looking for the ref file.
  Use git-update-ref to delete a tag instead of rm()ing the ref file.
  Fix refs.c;:repack_without_ref() clean-up path
  Clean up "git-branch.sh" and add remove recursive dir test cases.
  ...
2006-11-01 08:48:50 -08:00
Nicolas Pitre
d9c20ba13d enhance clone and fetch -k experience
Now that index-pack can be streamed with a pack, it is probably a good
idea to use it directly instead of creating a temporary file and running
index-pack afterwards.  This way index-pack can abort early whenever a
corruption is encountered even if the pack has not been fully
downloaded, it can display a progress percentage as it knows how much to
expects, and it is a bit faster since the pack indexing is partially
done as data is received. Using fetch -k doesn't need to disable thin
pack generation on the remote end either.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-10-27 14:58:31 -07:00
Nicolas Pitre
e4fe4b8ef7 let the GIT native protocol use offsets to delta base when possible
There is no reason not to always do this when both ends agree.
Therefore a client that can accept offsets to delta base always sends
the "ofs-delta" flag.  The server will stream a pack with or without
offset to delta base depending on whether that flag is provided or not
with no additional cost.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-09-27 00:12:00 -07:00
Junio C Hamano
8da1977554 Tell between packed, unpacked and symbolic refs.
This adds a "int *flag" parameter to resolve_ref() and makes
for_each_ref() family to call callback function with an extra
"int flag" parameter.  They are used to give two bits of
information (REF_ISSYMREF and REF_ISPACKED) about the ref.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-09-20 22:02:01 -07:00
Junio C Hamano
cb5d709ff8 Add callback data to for_each_ref() family.
This is a long overdue fix to the API for for_each_ref() family
of functions.  It allows the callers to specify a callback data
pointer, so that the caller does not have to use static
variables to communicate with the callback funciton.

The updated for_each_ref() family takes a function of type

	int (*fn)(const char *, const unsigned char *, void *)

and a void pointer as parameters, and calls the function with
the name of the ref and its SHA-1 with the caller-supplied void
pointer as parameters.

The commit updates two callers, builtin-name-rev.c and
builtin-pack-refs.c as an example.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-09-20 21:47:42 -07:00
Junio C Hamano
4d69065d3a Merge branch 'jc/archive'
* jc/archive:
  git-tar-tree: devolve git-tar-tree into a wrapper for git-archive
  git-archive: inline default_parse_extra()
  builtin-archive.c: rename remote_request() to extract_remote_arg()
  upload-archive: monitor child communication more carefully.
  Add sideband status report to git-archive protocol
  Prepare larger packet buffer for upload-pack protocol.
  Teach --exec to git-archive --remote
  Add --verbose to git-archive
  archive: force line buffered output to stderr
  Use xstrdup instead of strdup in builtin-{tar,zip}-tree.c
  Move sideband server side support into reusable form.
  Move sideband client side support into reusable form.
  archive: allow remote to have more formats than we understand.
  git-archive: make compression level of ZIP archives configurable
  Add git-upload-archive
  git-archive: wire up ZIP format.
  git-archive: wire up TAR format.
  Add git-archive
2006-09-17 02:46:00 -07:00
Franck Bui-Huu
8a5dbef8ac Test return value of finish_connect()
Signed-off-by: Franck Bui-Huu <vagabon.xyz@gmail.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-09-13 12:20:15 -07:00
Junio C Hamano
d47f3db75c Prepare larger packet buffer for upload-pack protocol.
The original side-band support added to the upload-pack protocol used the
default 1000-byte packet length.  The pkt-line format allows up to 64k, so
prepare the receiver for the maximum size, and have the uploader and
downloader negotiate if larger packet length is allowed.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-09-10 16:27:08 -07:00
Shawn Pearce
e702496e43 Convert memcpy(a,b,20) to hashcpy(a,b).
This abstracts away the size of the hash values when copying them
from memory location to memory location, much as the introduction
of hashcmp abstracted away hash value comparsion.

A few call sites were using char* rather than unsigned char* so
I added the cast rather than open hashcpy to be void*.  This is a
reasonable tradeoff as most call sites already use unsigned char*
and the existing hashcmp is also declared to be unsigned char*.

[jc: Splitted the patch to "master" part, to be followed by a
 patch for merge-recursive.c which is not in "master" yet.

 Fixed the cast in the latter hunk to combine-diff.c which was
 wrong in the original.

 Also converted ones left-over in combine-diff.c, diff-lib.c and
 upload-pack.c ]

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-08-23 13:53:10 -07:00
David Rientjes
96f1e58f52 remove unnecessary initializations
[jc: I needed to hand merge the changes to the updated codebase,
 so the result needs to be checked.]

Signed-off-by: David Rientjes <rientjes@google.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-08-15 21:22:20 -07:00
Linus Torvalds
1974632c66 Remove TYPE_* constant macros and use object_type enums consistently.
This updates the type-enumeration constants introduced to reduce
the memory footprint of "struct object" to match the type bits
already used in the packfile format, by removing the former
(i.e. TYPE_* constant macros) and using the latter (i.e. enum
object_type) throughout the code for consistency.

Eventually we can stop passing around the "type strings"
entirely, and this will help - no confusion about two different
integer enumeration.

Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-07-12 23:18:03 -07:00
Junio C Hamano
583b7ea31b upload-pack/fetch-pack: support side-band communication
This implements a protocol extension between fetch-pack and
upload-pack to allow stderr stream from upload-pack (primarily
used for the progress bar display) to be passed back.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-06-21 02:50:32 -07:00
Linus Torvalds
885a86abe2 Shrink "struct object" a bit
This shrinks "struct object" by a small amount, by getting rid of the
"struct type *" pointer and replacing it with a 3-bit bitfield instead.

In addition, we merge the bitfields and the "flags" field, which
incidentally should also remove a useless 4-byte padding from the object
when in 64-bit mode.

Now, our "struct object" is still too damn large, but it's now less
obviously bloated, and of the remaining fields, only the "util" (which is
not used by most things) is clearly something that should be eventually
discarded.

This shrinks the "git-rev-list --all" memory use by about 2.5% on the
kernel archive (and, perhaps more importantly, on the larger mozilla
archive). That may not sound like much, but I suspect it's more on a
64-bit platform.

There are other remaining inefficiencies (the parent lists, for example,
probably have horrible malloc overhead), but this was pretty obvious.

Most of the patch is just changing the comparison of the "type" pointer
from one of the constant string pointers to the appropriate new TYPE_xxx
small integer constant.

Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-06-17 18:49:18 -07:00
Junio C Hamano
f061e5fdd6 fetch-pack: give up after getting too many "ack continue"
If your repository have more roots than the remote repository
you ask an object for, the remote upload-pack keeps responding
"ack continue" until it fills up its received-have buffer
(currently 256 entries).  Usually this is not a problem because
the requester stops traversing the ancestry chain from the commit
it gets "ack continue" for, but this mechanism does not work as
a roadblock when it traverses down the path to the root the
other side does not have.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-05-24 22:30:56 -07:00
Junio C Hamano
9546010b7b fetch-pack: output refs in the order they were given on the command line.
Currently, fetched refs are output in the order the remote side
happened to send them.  This changes the order to match the
order of refs that were given on the command line.  To the
existing core callers (git-fetch and git-clone) this does not
make any difference, but for other Porcelain use, it would be
more intuitive.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-05-22 05:32:38 -07:00
Junio C Hamano
dfeff66ed9 revamp git-clone.
This does two things.

 * A new flag --reference can be used to name a local repository
   that is to be used as an alternate.  This is in response to
   an inquiry by James Cloos in the message on the list
   <m3r74ykue7.fsf@lugabout.cloos.reno.nv.us>.

 * A new flag --use-separate-remote stops contaminating local
   branch namespace by upstream branch names.  The upstream
   branch heads are copied in .git/refs/remotes/ instead of
   .git/refs/heads/ and .git/remotes/origin file is set up to
   reflect this as well.  It requires to have fetch/pull update
   to understand .git/refs/remotes by Eric Wong to further
   update the repository cloned this way.

For the former change, git-fetch-pack is taught a new flag --all
to fetch from all the remote heads.  Nobody uses the git-clone-pack
with this change, so we could deprecate the command, but removal
of the command will be left to a separate round.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-03-20 00:21:10 -08:00
Timo Hirvonen
962554c616 Use setenv(), fix warnings
- Fix -Wundef -Wold-style-definition warnings
  - Make pll_free() static

[jc: original patch by Timo had another unrelated bits:

  - Use setenv() instead of putenv()

 I'm postponing that part for now.]

Signed-off-by: Timo Hirvonen <tihirvon@gmail.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-02-26 15:06:45 -08:00
Junio C Hamano
b19696c2e7 Use thin pack transfer in "git fetch".
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-02-20 00:38:39 -08:00
Linus Torvalds
5ee2ad654b Make "git clone" less of a deathly quiet experience
It used to be that "git-unpack-objects" would give nice percentages, but
now that we don't unpack the initial clone pack any more, it doesn't. And
I'd love to do that nice percentage view in the pack objects downloader
too, but the thing doesn't even read the pack header, much less know how
much it's going to get, so I was lazy and didn't.

Instead, it at least prints out how much data it's gotten, and what the
packing speed is. Which makes the user realize that it's actually doing
something useful instead of sitting there silently (and if the recipient
knows how large the final result is, he can at least make a guess about
when it migt be done).

So with this patch, I get something like this on my DSL line:

	[torvalds@g5 ~]$ time git clone master.kernel.org:/pub/scm/linux/kernel/git/torvalds/linux-2.6 clone-test
	Packing 188543 objects
	  48.398MB  (154 kB/s)

where even the speed approximation seems to be roughtly correct (even
though my algorithm is a truly stupid one, and only really gives "speed in
the last half second or so").

Anyway, _something_ like this is definitely needed. It could certainly be
better (if it showed the same kind of thing that git-unpack-objects did,
that would be much nicer, but would require parsing the object stream as
it comes in). But this is  big step forward, I think.

Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-02-10 22:28:30 -08:00
Johannes Schindelin
e921fb82cf git-fetch-pack: really do not ask for funny refs
If git-fetch-pack was called with out any refspec, it would ask the server
for funny refs. That cannot work, since the funny refs are not marked
as OUR_REF by upload-pack, which just exits with an error.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-01-19 18:29:43 -08:00
Junio C Hamano
2247efb40b clone-pack: remove unused and undocumented --keep flag
While we are at it, give fully spelled --keep to fetch-pack.
Also give --quiet in addition to -q to fetch-pack as well.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-12-18 01:55:29 -08:00
Junio C Hamano
ad89721508 fetch-pack: -k option to keep downloaded pack.
Split out the functions that deal with the socketpair after
finishing git protocol handshake to receive the packed data into
a separate file, and use it in fetch-pack to keep/explode the
received pack data.  We earlier had something like that on
clone-pack side once, but the list discussion resulted in the
decision that it makes sense to always keep the pack for
clone-pack, so unpacking option is not enabled on the clone-pack
side, but we later still could do so easily if we wanted to with
this change.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-12-17 23:11:29 -08:00
Junio C Hamano
5a3277133d Make networking commands to work from a subdirectory.
These are whole-tree operations and there is not much point
making them operable from within a subdirectory, but it is easy
to do so, and using setup_git_directory() upfront helps git://
proxy specification picked up from the correct place.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-11-28 23:13:02 -08:00
Junio C Hamano
9e5d2b4096 git-fetch: fail if specified refspec does not match remote.
'git-fetch remote no-such-ref' succeeded without fetching any
ref from the remote.  Detect such case and report an error.

Note that this makes 'git-fetch remote master master' to fail,
because the remote branch 'master' matches the first refspec,
and the second refspec is left unmatched, which is detected by
the error checking logic.  This is somewhat unintuitive, but
giving the same refspec more than once to git-fetch is useless
in any case so it should not be much of a problem.  I'd accept a
patch to change this if somebody cares enough, though.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-11-06 00:26:21 -08:00
Junio C Hamano
9534f40bc4 Be careful when dereferencing tags.
One caller of deref_tag() was not careful enough to make sure
what deref_tag() returned was not NULL (i.e. we found a tag
object that points at an object we do not have).  Fix it, and
warn about refs that point at such an incomplete tag where
needed.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-11-02 16:50:58 -08:00
Johannes Schindelin
c4c86f07d0 git-fetch-pack: Support multi_ack extension
The client side support for multi_ack.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-28 22:57:01 -07:00
Johannes Schindelin
1baaae5e1f Make maximal use of the remote refs
When git-fetch-pack gets the remote refs, it does not need to filter them
right away, but it can see which refs are common (taking advantage of the
patch which makes git-fetch-pack not use git-rev-list).

This means that we ask get_remote_heads() to return all remote refs,
including the funny refs, and filtering them with a separate function later.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-28 22:56:58 -07:00
Johannes Schindelin
23d61f8343 Subject: [PATCH] git-fetch-pack: Do not use git-rev-list
The code used to call git-rev-list to enumerate the local revisions.
A disadvantage of that method was that git-rev-list, lacking a
control apart from the command line, would happily enumerate
ancestors of acknowledged common commits, which was just taking
unnecessary bandwidth.

Therefore, do not use git-rev-list on the fetching side, but rather
construct the list on the go. Send the revisions starting from the
local heads, ignoring the revisions known to be common.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-28 22:56:58 -07:00
Junio C Hamano
af2d3aa4d8 Revert recent fetch-pack/upload-pack updates.
Let's have it simmer a bit longer in the proposed updates branch
and shake the problems out.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-25 14:55:24 -07:00
Johannes Schindelin
40a1046249 git-fetch-pack: Implement client part of the multi_ack extension
This patch concludes the series, which makes
git-fetch-pack/git-upload-pack negotiate a potentially better set of
common revs. It should make a difference when fetching from a repository
with a few branches.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-24 15:13:37 -07:00
Johannes Schindelin
69779a562a git-fetch-pack: Do not use git-rev-list
The code used to call git-rev-list to enumerate the local revisions. A
disadvantage of that method was that git-rev-list, lacking a control apart
from the command line, would happily enumerate ancestors of acknowledged
common commits, which was just taking unnecessary bandwidth.

Therefore, do not use git-rev-list on the fetching side, but rather
construct the list on the go. Send the revisions starting from the local
heads, ignoring the revisions known to be common.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-24 15:13:37 -07:00
Junio C Hamano
f1f0a2be9f Be more careful tangling object chains while marking commits.
Also Johannes noticed we use parse_object to look up if we know that
object already -- we should just ask the in-core object registry with
lookup_object() for that.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-19 21:55:49 -07:00
Junio C Hamano
4dab94d52e Do not feed rev-list an invalid SHA1 expression.
The previous round to optimize fetch-pack has a small bug that
feeds SHA1^ ("parent commit") before making sure SHA1 is
actually a commit (or a tag that eventually dereferences to a
commit).  Also it did not help culling the known-to-be-common
parents if the common one was a merge.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-19 21:04:53 -07:00
Johannes Schindelin
0a8944dd48 [PATCH] Do not send "want" lines for complete objects
It was all good and well to check if all remote refs are complete (local
refs or descendants thereof), but we can just as easily use the same
information to avoid sending "want" lines just for the complete objects in
the case that not all remote refs are complete (or their names differ).

Also, git-fetch-pack does not have to ask for descendants of remote refs
which are complete (for now, git-rev-list is told to ignore only the first
parent). That change also eliminates a code path where a popen()ed handle
was not pclose()ed.

Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-19 16:14:34 -07:00
Junio C Hamano
49bb805e69 Do not ask for objects known to be complete.
On top of optimization by Linus not to ask refs that already match, we
can walk our refs and not issue "want" for things that are known to be
reachable from them.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-19 14:27:02 -07:00
Linus Torvalds
2759cbc774 git-fetch-pack: avoid unnecessary zero packing
If everything is up-to-date locally, we don't need to even ask for a
pack-file from the remote, or try to unpack it.

This is especially important for tags - since the pack-file common commit
logic is based purely on the commit history, it will never be able to find
a common tag, and will thus always end up re-fetching them.

Especially notably, if the tag points to a non-commit (eg a tagged tree),
the pack-file would be unnecessarily big, just because it cannot any most
recent common point between commits for pruning.

Short-circuiting the case where we already have that reference means that
we avoid a lot of these in the common case.

NOTE! This only matches remote ref names against the same local name,
which works well for tags, but is not as generic as it could be. If we
ever need to, we could match against _any_ local ref (if we have it, we
have it), but this "match against same name" is simpler and more
efficient, and covers the common case.

Renaming of refs is common for branch heads, but since those are always
commits, the pack-file generation can optimize that case.

In some cases we might still end up fetching pack-files unnecessarily, but
this at least avoids the re-fetching of tags over and over if you use a
regular

	git fetch --tags ...

which was the main reason behind the change.

Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-18 11:35:17 -07:00
Junio C Hamano
1a7141ff28 Ignore funny refname sent from remote
This allows the remote side (most notably, upload-pack) to show
additional information without affecting the downloader.  Peek-remote
does not ignore them -- this is to make it useful for Pasky's
automatic tag following.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2005-10-15 11:23:40 -07:00
Junio C Hamano
33b8303466 fetch-pack: start multi-head pulling.
This is a beginning of resurrecting the multi-head pulling support
for git-fetch-pack command.  The git-fetch-script wrapper still
only knows about fetching a single head, without renaming, so it is
not very useful unless you directly call git-fetch-pack itself yet.

It also fixes a longstanding obsolete description of how the command
discovers the list of local commits.
2005-08-12 10:38:21 -07:00
Linus Torvalds
d1c133f5d4 Merge three separate "fetch refs" functions
It really just boils down to one "get_remote_heads()" function, and a
common "struct ref" structure definition.
2005-07-16 13:55:50 -07:00
Junio C Hamano
8b3d9dc0e2 [PATCH] Documentation: clone/fetch/upload.
This adds documentation for 'smarter pull' family of commands.

Signed-off-by: Junio C Hamano <junkio@cox.net>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-07-14 08:54:31 -07:00
Linus Torvalds
516236ce3f Get rid of nasty utf-8 characters in printout
Oh, well..  FC4 has UTF-8 as the default environment, and I applaud
that, but then it sometimes results in these characters that aren't
actually visible as a problem.
2005-07-13 20:55:56 -07:00
Linus Torvalds
85c414b5e0 git-fetch-pack: close output fd after dup'ing the input
With the socket case, the input and output fd's might end up being the same,
so we want to dup the other before we close either of them.
2005-07-13 19:40:06 -07:00
Linus Torvalds
41cb7488b9 Move "get_ack()" to common git_connect functions
git-clone-pack will want it too. Soon.
2005-07-05 15:44:09 -07:00
Linus Torvalds
4f7770c87c Remove multi-head support from fetch-pack
It was a misguided attempt to mix fetching and cloning. I'll make
a separate clone thing.
2005-07-05 14:39:30 -07:00
Linus Torvalds
723c31fea2 Add "git_path()" and "head_ref()" helper functions.
"git_path()" returns a static pathname pointer into the git directory
using a printf-like format specifier.

"head_ref()" works like "for_each_ref()", except for just the HEAD.
2005-07-05 11:31:32 -07:00
Linus Torvalds
75bfc6c232 Make git-fetch-pack actually do all the unpacking etc.
It returns the result SHA1 on stdout, so you can do

	remote=$(git-fetch-pack host:dir branchname)

and it will unpack the objects and "remote" will be the SHA1 name of the
branch on the other side.  You can then save that off, or merge it, or
whatever.
2005-07-04 16:35:13 -07:00
Linus Torvalds
fb9040cc83 Make git-fetch-pack and git-upload-pack negotiate needs/haves fully
Now the only piece missing is actually generating the pack-file.
2005-07-04 15:29:17 -07:00
Linus Torvalds
def88e9afb Commit first cut at "git-fetch-pack"
It's meant to be used by "git fetch" for the local and ssh case.

It doesn't actually do the fetching now, but it does discover the common
commit point.
2005-07-04 13:26:53 -07:00