Currently the helper must somehow guess how many import statements to
read before it starts outputting its fast-export stream. This is
because the remote helper infrastructure runs fast-import only once,
so the helper is forced to output one stream for all import commands
it will receive. The only reason this worked in the past was because
only one ref was imported at a time.
Change the semantics of the import statement such that it matches
that of the push statement. That is, the import statement is followed
by a series of import statements that are terminated by a '\n'.
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Also add check_output from python 2.7.
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In other words, use fast-export --use-done-feature to add a 'done'
command at the end of streams passed to remote helpers' "import"
commands, and teach the remote helpers implementing "export" to use
the 'done' command in turn when producing their streams.
The trailing \n in the protocol signals the helper that the
connection is about to close, allowing it to do whatever cleanup
neccesary.
Previously, the connection would already be closed by the
time the trailing \n was to be written. Now that the remote-helper
protocol uses the new done command in its fast-import streams, this
is no longer the case and we can safely write the trailing \n.
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Previously the status code of all helpers were ignored, allowing
errors that occur to go unnoticed if the error text output by the
helper is not noticed.
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The update ref status part of push is useful for the export command
as well, factor it out into it's own function.
Also factor out push_update_ref_status to avoid a long loop without
an explicit condition with a non-trivial body.
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The gitdir capability is recognized by git and can be used to tell
the helper where the .git directory is. But it is not mentioned in
the documentation and considered worse than if gitdir was passed
via GIT_DIR environment variable.
Remove support for the gitdir capability and export GIT_DIR instead.
Teach testgit to use env instead of the now-removed gitdir command.
[sr: fixed up documentation]
Signed-off-by: Dmitry Ivankov <divanorama@gmail.com>
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When we want to push to a remote helper that has the
"export" capability, we collect all of the refs we want to
push and then feed them to fast-export.
However, the list of refs is actually a list of remote refs,
not local refs. The mapped local refs are included via the
peer_ref pointer. So when we add an argument to our
fast-export command line, we must be sure to use the local
peer_ref name (and if there is no local name, it is because
we are not actually sending that ref, or we may not even
have the ref at all).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Fix warnings from 'make check'.
- These files don't include 'builtin.h' causing sparse to complain that
cmd_* isn't declared:
builtin/clone.c:364, builtin/fetch-pack.c:797,
builtin/fmt-merge-msg.c:34, builtin/hash-object.c:78,
builtin/merge-index.c:69, builtin/merge-recursive.c:22
builtin/merge-tree.c:341, builtin/mktag.c:156, builtin/notes.c:426
builtin/notes.c:822, builtin/pack-redundant.c:596,
builtin/pack-refs.c:10, builtin/patch-id.c:60, builtin/patch-id.c:149,
builtin/remote.c:1512, builtin/remote-ext.c:240,
builtin/remote-fd.c:53, builtin/reset.c:236, builtin/send-pack.c:384,
builtin/unpack-file.c:25, builtin/var.c:75
- These files have symbols which should be marked static since they're
only file scope:
submodule.c:12, diff.c:631, replace_object.c:92, submodule.c:13,
submodule.c:14, trace.c:78, transport.c:195, transport-helper.c:79,
unpack-trees.c:19, url.c:3, url.c:18, url.c:104, url.c:117, url.c:123,
url.c:129, url.c:136, thread-utils.c:21, thread-utils.c:48
- These files redeclare symbols to be different types:
builtin/index-pack.c:210, parse-options.c:564, parse-options.c:571,
usage.c:49, usage.c:58, usage.c:63, usage.c:72
- These files use a literal integer 0 when they really should use a NULL
pointer:
daemon.c:663, fast-import.c:2942, imap-send.c:1072, notes-merge.c:362
While we're in the area, clean up some unused #includes in builtin files
(mostly exec_cmd.h).
Signed-off-by: Stephen Boyd <bebarino@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint:
Prepare draft release notes to 1.7.4.2
gitweb: highlight: replace tabs with spaces
make_absolute_path: return the input path if it points to our buffer
valgrind: ignore SSE-based strlen invalid reads
diff --submodule: split into bite-sized pieces
cherry: split off function to print output lines
branch: split off function that writes tracking info and commit subject
standardize brace placement in struct definitions
compat: make gcc bswap an inline function
enums: omit trailing comma for portability
Conflicts:
RelNotes
In a struct definitions, unlike functions, the prevailing style is for
the opening brace to go on the same line as the struct name, like so:
struct foo {
int bar;
char *baz;
};
Indeed, grepping for 'struct [a-z_]* {$' yields about 5 times as many
matches as 'struct [a-z_]*$'.
Linus sayeth:
Heretic people all over the world have claimed that this inconsistency
is ... well ... inconsistent, but all right-thinking people know that
(a) K&R are _right_ and (b) K&R are right.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
'bytes' is of type size_t which is unsigned thus can't be negative. But
the assigned write() returns ssize_t, and -1 on error.
For testing < 0, 'bytes' needs to be of a signed type.
Signed-off-by: Nicolas Kaiser <nikai@nikai.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
All files that include this header file use the same four line
incantation:
#ifndef NO_PTHREADS
#include <pthread.h>
#include "thread-utils.h"
#endif
Move the responsibility for that gymnastics to the header file from the
files that include it. This approach makes it easier to later declare new
services that are related to threading in thread-utils.h and have them
available to all the threading code.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When compiling with pthread support, transport-helper.c needs to include
necessary header files. Also fix a few error messages in remote-ext and
remote-fd programs, and a potential buffer underrun in remote-fd.
In the documentation, clarify how %G and %V are used; the old description
looked as if they take repository/vhost parameters, which was wrong.
Also fix AsciiDoc markup for the page title of remote-fd/remote-ext manpages,
and tweak the way how section headers are shown.
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This helper function copies bidirectional stream of data between
stdin/stdout and specified file descriptors.
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Update the definition and callers of string_list_append to use the
string_list as the first argument. This helps make the string_list
API easier to use by being more consistent.
Signed-off-by: Julian Phillips <julian@quantumfyre.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The 'gitdir' capability is reported by the remote helper if it
requires the location of the .git directory. The location of the .git
directory can then be used by the helper to store status files even
when the current directory is not a git repository (such as is the
case when cloning).
The location of the .git dir is specified as an absolute path.
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* tc/transport-verbosity:
transport: update flags to be in running order
fetch and pull: learn --progress
push: learn --progress
transport->progress: use flag authoritatively
clone: support multiple levels of verbosity
push: support multiple levels of verbosity
fetch: refactor verbosity option handling into transport.[ch]
Documentation/git-push: put --quiet before --verbose
Documentation/git-pull: put verbosity options before merge/fetch ones
Documentation/git-clone: mention progress in -v
Conflicts:
transport.h
Set transport->progress in transport.c::transport_set_verbosity() after
checking for the appropriate conditions (eg. --progress, isatty(2)),
and thereafter use it without having to check again.
The rules used are as follows (processing aborts when a rule is
satisfied):
1. Report progress, if force_progress is 1 (ie. --progress).
2. Don't report progress, if verbosity < 0 (ie. -q/--quiet).
3. Report progress if isatty(2) is 1.
This changes progress reporting behaviour such that if both --progress
and --quiet are specified, progress is reported.
In two areas, the logic to determine whether to *not* show progress is
changed to simply use the negation of transport->progress. This changes
behaviour in some ways (see previous paragraph for details).
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Remove the flags TRANSPORT_PUSH_QUIET and TRANSPORT_PUSH_VERBOSE; use
transport->verbose instead to determine verbosity for pushing.
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* js/exec-error-report:
Improve error message when a transport helper was not found
start_command: detect execvp failures early
run-command: move wait_or_whine earlier
start_command: report child process setup errors to the parent's stderr
Conflicts:
Makefile
* tr/http-push-ref-status:
transport-helper.c::push_refs(): emit "no refs" error message
transport-helper.c::push_refs(): ignore helper-reported status if ref is not to be pushed
transport.c::transport_push(): make ref status affect return value
refactor ref status logic for pushing
t5541-http-push.sh: add test for unmatched, non-fast-forwarded refs
t5541-http-push.sh: add tests for non-fast-forward pushes
Conflicts:
transport-helper.c
* tc/clone-v-progress:
clone: use --progress to force progress reporting
clone: set transport->verbose when -v/--verbose is used
git-clone.txt: reword description of progress behaviour
check stderr with isatty() instead of stdout when deciding to show progress
Conflicts:
transport.c
Perviously, the error message was:
git: 'remote-foo' is not a git-command. See 'git --help'.
By not treating the transport helper as a git command, a more suitable
error is reported:
fatal: Unable to find remote helper for 'foo'
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Emit an error message when remote_refs is not set.
This behaviour is consistent with that of builtin-send-pack.c and
http-push.c.
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If the status of a ref is REF_STATUS_NONE, the remote helper will not
be told to push the ref (via a 'push' command).
However, the remote helper may still act on these refs.
If the helper does act on the ref, and prints a status for it, ignore
the report (ie. don't overwrite the status of the ref with it, nor the
message in the remote_status member) if the reported status is 'no
match'.
This allows the user to be alerted to more "interesting" ref statuses,
like REF_STATUS_NONFASTFORWARD.
Cc: Jeff King <peff@peff.net>
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move the logic that detects up-to-date and non-fast-forward refs to a
new function in remote.[ch], set_ref_status_for_push().
Make transport_push() invoke set_ref_status_for_push() before invoking
the push_refs() implementation. (As a side-effect, the push_refs()
implementation in transport-helper.c now knows of non-fast-forward
pushes.)
Removed logic for detecting up-to-date refs from the push_refs()
implementation in transport-helper.c, as transport_push() has already
done so for it.
Make cmd_send_pack() invoke set_ref_status_for_push() before invoking
send_pack(), as transport_push() can't do it for send_pack() here.
Mark the test on the return status of non-fast-forward push to fail.
Git now exits with success, as transport.c::transport_push() does not
check for refs with status REF_STATUS_REJECT_NONFASTFORWARD nor does it
indicate rejected pushes with its return value.
Mark the test for ref status to succeed. As mentioned earlier, refs
might be marked as non-fast-forwards, triggering the push status
printing mechanism in transport.c.
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make transport code (viz. transport.c::fetch_refs_via_pack() and
transport-helper.c::standard_options()) that decides to show progress
check if stderr is a terminal, instead of stdout. After all, progress
reports (via the API in progress.[ch]) are sent to stderr.
Update the documentation for git-clone to say "standard error" as well.
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Previously, remote archive required internal (non remote-helper)
smart transport. Extend the remote archive to also support smart
transports implemented by remote helpers.
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* sr/vcs-helper:
tests: handle NO_PYTHON setting
builtin-push: don't access freed transport->url
Add Python support library for remote helpers
Basic build infrastructure for Python scripts
Allow helpers to report in "list" command that the ref is unchanged
Fix various memory leaks in transport-helper.c
Allow helper to map private ref names into normal names
Add support for "import" helper command
Allow specifying the remote helper in the url
Add a config option for remotes to specify a foreign vcs
Allow fetch to modify refs
Use a function to determine whether a remote is valid
Allow programs to not depend on remotes having urls
Fix memory leak in helper method for disconnect
Conflicts:
Documentation/git-remote-helpers.txt
Makefile
builtin-ls-remote.c
builtin-push.c
transport-helper.c
Add support for taking over transports that turn out to be smart.
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change URL handling to allow external protocol handlers to implement
new protocols without the '::' syntax if helper name does not conflict
with any built-in protocol.
foo:// now invokes git-remote-foo with foo:// as the URL.
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add support for marking capability as mandatory for hosting git version
to understand. This is useful for helpers which require various types
of assistance from main git binary.
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Remote helpers deadlock easily, so support debug mode which shows the
interaction steps.
Signed-off-by: Ilari Liusvaara <ilari.liusvaara@elisanet.fi>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* master: (334 commits)
bash: update 'git commit' completion
Git 1.6.5.5
Fix diff -B/--dirstat miscounting of newly added contents
reset: improve worktree safety valves
Documentation: Avoid use of xmlto --stringparam
archive: clarify description of path parameter
rerere: don't segfault on failure to open rr-cache
Prepare for 1.6.5.5
gitweb: Describe (possible) gitweb.js minification in gitweb/README
Documentation: xmlto 0.0.18 does not know --stringparam
Fix crasher on encountering SHA1-like non-note in notes tree
t9001: use older Getopt::Long boolean prefix '--no' rather than '--no-'
t4201: use ISO8859-1 rather than ISO-8859-1
Git 1.6.5.4
Unconditionally set man.base.url.for.relative.links
Documentation/Makefile: allow man.base.url.for.relative.link to be set from Make
Git 1.6.6-rc1
git-pull.sh: Fix call to git-merge for new command format
Prepare for 1.6.5.4
merge: do not add standard message when message is given with -m option
...
Conflicts:
Documentation/git-remote-helpers.txt
Makefile
builtin-ls-remote.c
builtin-push.c
transport-helper.c
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Helpers may use a line like "? name unchanged" to specify that there
is nothing new at that name, without any git-specific code to
determine the correct response.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Found with:
valgrind --tool=memcheck --leak-check=full --show-reachable=yes
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This allows a helper to say that, when it handles "import
refs/heads/topic", the script it outputs will actually write to
refs/svn/origin/branches/topic; therefore, transport-helper should
read it from the latter location after git-fast-import completes.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This command, supported if the "import" capability is advertized,
allows a helper to support fetching by outputting a git-fast-import
stream.
If both "fetch" and "import" are advertized, git itself will use
"fetch" (although other users may use "import" in this case).
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This allows the transport to use the null sha1 for a ref reported to
be present in the remote repository to indicate that a ref exists but
its actual value is presently unknown and will be set if the objects
are fetched.
Also adds documentation to the API to specify exactly what the methods
should do and how they should interpret arguments.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since some cases may need to disconnect from the helper and reconnect,
wrap the function that just disconnects in a function that also frees
transport->data.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Clemens Buchacher <drizzd@aon.at>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The remote helper interface now supports the push capability,
which can be used to ask the implementation to push one or more
specs to the remote repository. For remote-curl we implement this
by calling the existing WebDAV based git-http-push executable.
Internally the helper interface uses the push_refs transport hook
so that the complexity of the refspec parsing and matching can be
reused between remote implementations. When possible however the
helper protocol uses source ref name rather than the source SHA-1,
thereby allowing the helper to access this name if it is useful.
>From Clemens Buchacher <drizzd@aon.at>:
update http tests according to remote-curl capabilities
o Pushing packed refs is now fixed.
o The transport helper fails if refs are already up-to-date. Add
a test for that.
o The transport helper will notice if refs are already
up-to-date. We therefore need to update server info in the
unpacked-refs test.
o The transport helper will purge deleted branches automatically.
o Use a variable ($ORIG_HEAD) instead of full SHA-1 name.
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Clemens Buchacher <drizzd@aon.at>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CC: Daniel Barkalow <barkalow@iabervon.org>
CC: Mike Hommey <mh@glandium.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some transports, like the native pack transport implemented by
fetch-pack, support useful features like depth or include tags.
These should be exposed if the underlying helper knows how to
use them.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CC: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some network protocols (e.g. native git://) are able to fetch more
than one ref at a time and reduce the overall transfer cost by
combining the requests into a single exchange. Instead of feeding
each fetch request one at a time to the helper, feed all of them
at once so the helper can decide whether or not it should batch them.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CC: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There were several unchecked use of fdopen(); replace them with xfdopen()
that checks and dies.
Signed-off-by: Jim Meyering <meyering@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is the same fix to use write_str_in_full() helper to write a constant
string out without counting the length of it ourselves.
Signed-off-by: Jim Meyering <meyering@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This style is overkill for some commands, but it's worthwhile to use
the same style to issue all commands, and it's useful to avoid
open-coding string lengths.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of trying to make http://, https://, and ftp:// URLs
indicative of some sort of pattern of transport helper usage, make
them a special case which runs the "curl" helper, and leave the
mechanism by which arbitrary helpers will be chosen entirely to future
work.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This splits up git-http-fetch so that it isn't built-in.
It also removes the general dependency on curl, because it is no
longer used by any built-in code. Because they are no longer LIB_OBJS,
add LIB_H to the dependencies of http-related object files, and remove
http.h from the dependencies of transport.o
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
transport_get() can call transport_native_helper_init() to have list and
fetch-ref operations handled by running a separate program as:
git remote-<something> <remote> [<url>]
This program then accepts, on its stdin, "list" and "fetch <hex>
<name>" commands; the former prints out a list of available refs and
either their hashes or what they are symrefs to, while the latter
fetches them into the local object database and prints a newline when done.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
