This is in the great tradition of POSIX. Original fix by Olivier Refalo.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As of "Win32: Thread-safe windows console output", child processes may
print to the console even if stdout has been redirected to a file. E.g.:
git config tar.cat.command "cat"
git archive -o test.cat HEAD
Detecting whether stdout / stderr point to our console pipe is currently
based on the assumption that OS HANDLE values are never reused. This is
apparently not true if stdout / stderr is replaced via dup2() (as in
builtin/archive.c:17).
Instead of comparing handle values, check if the file descriptor isatty()
backed by a pipe OS handle. This is only possible by swapping the handles
in MSVCRT's internal data structures, as we do in winansi_init().
Reported-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As of "Win32: Thread-safe windows console output", git-log no longer
terminates when the pager process dies. This is due to disabling buffering
for the replaced stdout / stderr streams. Git-log will periodically fflush
stdout (see write_or_die.c/mayble_flush_or_die()), but with no buffering,
this is a NOP that always succeeds (so we never detect the EPIPE error).
Exchange the original console handles with our console thread pipe handles
by accessing the internal MSVCRT data structures directly (which are
exposed via __pioinfo for some reason).
Implement this with minimal assumptions about the actual data structure to
make it work with different (hopefully even future) MSVCRT versions.
While messing with internal data structures is ugly, this patch solves the
problem at the source instead of adding more workarounds. We no longer need
the special winansi_isatty override, and the limitations documented in
"Win32: Thread-safe windows console output" are gone (i.e. fdopen(1/2)
returns unbuffered streams now, and isatty() for duped console file
descriptors works as expected).
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Winansi.c has many static variables that are accessed and modified from
the [v][f]printf / fputs functions overridden in the file. This may cause
multi threaded git commands that print to the console to produce corrupted
output or even crash.
Additionally, winansi.c doesn't override all functions that can be used to
print to the console (e.g. fwrite, write, fputc are missing), so that ANSI
escapes don't work properly for some git commands (e.g. git-grep).
Instead of doing ANSI emulation in just a few wrapped functions on top of
the IO API, let's plug into the IO system and take advantage of the thread
safety inherent to the IO system.
Redirect stdout and stderr to a pipe if they point to the console. A
background thread reads from the pipe, handles ANSI escape sequences and
UTF-8 to UTF-16 conversion, then writes to the console.
The pipe-based stdout and stderr replacements must be set to unbuffered, as
MSVCRT doesn't support line buffering and fully buffered streams are
inappropriate for console output.
Due to the byte-oriented pipe, ANSI escape sequences and multi-byte UTF-8
sequences can no longer be expected to arrive in one piece. Replace the
string-based ansi_emulate() with a simple stateful parser (this also fixes
colored diff hunk headers, which were broken as of commit 2efcc977).
Override isatty to return true for the pipes redirecting to the console.
Exec/spawn obtain the original console handle to pass to the next process
via winansi_get_osfhandle().
All other overrides are gone, the default stdio implementations work as
expected with the piped stdout/stderr descriptors.
Global variables are either initialized on startup (single threaded) or
exclusively modified by the background thread. Threads communicate through
the pipe, no further synchronization is necessary.
The background thread is terminated by disonnecting the pipe after flushing
the stdio and pipe buffers. This doesn't work for anonymous pipes (created
via CreatePipe), as DisconnectNamedPipe only works on the read end, which
discards remaining data. Thus we have to setup the pipe manually, with the
write end beeing the server (opened with CreateNamedPipe) and the read end
the client (opened with CreateFile).
Limitations: doesn't track reopened or duped file descriptors, i.e.:
- fdopen(1/2) returns fully buffered streams
- dup(1/2), dup2(1/2) returns normal pipe descriptors (i.e. isatty() =
false, winansi_get_osfhandle won't return the original console handle)
Currently, only the git-format-patch command uses xfdopen(xdup(1)) (see
"realstdout" in builtin/log.c), but works well with these limitations.
Many thanks to Atsushi Nakagawa <atnak@chejz.com> for suggesting and
reviewing the thread-exit-mechanism.
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add Unicode conversion functions to convert between Windows native UTF-16LE
encoding to UTF-8 and back.
To support repositories with legacy-encoded file names, the UTF-8 to UTF-16
conversion function tries to create valid, unique file names even for
invalid UTF-8 byte sequences, so that these repositories can be checked out
without error.
The current implementation leaves invalid UTF-8 bytes in range 0xa0 - 0xff
as is (producing printable Unicode chars \u00a0 - \u00ff, equivalent to
ISO-8859-1), and converts 0x80 - 0x9f to hex-code (\u0080 - \u009f are
control chars).
The Windows MultiByteToWideChar API was not used as it either drops invalid
UTF-8 sequences (on Win2k/XP; producing non-unique or even empty file
names) or converts them to the replacement char \ufffd (Vista/7; causing
ERROR_INVALID_NAME in subsequent calls to file system APIs).
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Unicode console output won't display correctly with default settings
because the default console font ("Terminal") only supports the system's
OEM charset. Unfortunately, this is a user specific setting, so it cannot
be easily fixed by e.g. some registry tricks in the setup program.
This change prints a warning on exit if console output contained non-ascii
characters and the console font is supposedly not a TrueType font (which
usually have decent Unicode support).
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
GetStdHandle(STD_OUTPUT_HANDLE) doesn't work for stderr if stdout is
redirected. Use _get_osfhandle of the FILE* instead.
_isatty() is true for all character devices (including parallel and serial
ports). Check return value of GetConsoleScreenBufferInfo instead to
reliably detect console handles (also don't initialize internal state from
an uninitialized CONSOLE_SCREEN_BUFFER_INFO structure if the function
fails).
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
WriteConsoleW seems to be the only way to reliably print unicode to the
console (without weird code page conversions).
Also redirects vfprintf to the winansi.c version.
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Fix const warnings in http-fetch.c and remote-curl.c main() where is
argv declared as const.
The fix should work for all future declarations of main, no matter
whether the second parameter's type is "char**", "const char**", or
"char *[]".
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The code in the MinGW main macro is getting more and more complex, move to
a separate initialization function for readabiliy and extensibility.
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
[efl: moved MinGW-specific part to compat/]
[jes: fixed compilation on non-Windows]
Eric Sunshine fixed mingw_offset_1st_component() to return
consistently "foo" for UNC "//machine/share/foo", cf
http://groups.google.com/group/msysgit/browse_thread/thread/c0af578549b5dda0
Author: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Cezary Zawadka <czawadka@gmail.com>
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The third maintenance release for Git 1.9; contains all the fixes
that are scheduled to appear in Git 2.0 since 1.9.2.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The git-shell(1) manpage says
EXAMPLE
To disable interactive logins, displaying a greeting
instead:
+
$ chsh -s /usr/bin/git-shell
$ mkdir $HOME/git-shell-commands
[...]
The stray "+" has been there ever since the example was added in
v1.8.3-rc0~210^2 (shell: new no-interactive-login command to print a
custom message, 2013-03-09). The "+" sign between paragraphs is
needed in asciidoc to attach extra paragraphs to a list item but here
it is not needed and ends up rendered as a literal "+". Remove it.
A quick search with "grep -e '<p>+' /usr/share/doc/git/html/*.html"
doesn't find any other instances of this problem.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git p4" dealing with changes in binary files were broken by a
change in 1.9 release.
* cl/p4-use-diff-tree:
git-p4: format-patch to diff-tree change breaks binary patches
The shell prompt script (in contrib/), when using the PROMPT_COMMAND
interface, used an unsafe construct when showing the branch name in
$PS1.
* rh/prompt-pcmode-avoid-eval-on-refname:
git-prompt.sh: don't put unsanitized branch names in $PS1
"git rebase" used a POSIX shell construct FreeBSD /bin/sh does not
work well with.
* km/avoid-non-function-return-in-rebase:
Revert "rebase: fix run_specific_rebase's use of "return" on FreeBSD"
rebase: avoid non-function use of "return" on FreeBSD
Some more Unicode codepoints defined in Unicode 6.3 as having zero
width have been taught to our display column counting logic.
* tb/unicode-6.3-zero-width:
utf8.c: partially update to version 6.3
When applying binary patches a full index is required. format-patch
already handles this, but diff-tree needs '--full-index' argument
to always output full index. When git-p4 runs git-apply to test
the patch, git-apply rejects the patch due to abbreviated blob
object names. This is the error message git-apply emits in this
case:
error: cannot apply binary patch to '<filename>' without full index line
error: <filename>: patch does not apply
Signed-off-by: Tolga Ceylan <tolga.ceylan@gmail.com>
Acked-by: Pete Wyckoff <pw@padd.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Both bash and zsh subject the value of PS1 to parameter expansion,
command substitution, and arithmetic expansion. Rather than include
the raw, unescaped branch name in PS1 when running in two- or
three-argument mode, construct PS1 to reference a variable that holds
the branch name. Because the shells do not recursively expand, this
avoids arbitrary code execution by specially-crafted branch names such
as '$(IFS=_;cmd=sudo_rm_-rf_/;$cmd)'.
Signed-off-by: Richard Hansen <rhansen@bbn.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This reverts commit 99855ddf4b.
The workaround 99855ddf introduced to deal with problematic
"return" statements in scripts run by "dot" commands located
inside functions only handles one part of the problem. The
issue has now been addressed by not using "return" statements
in this way in the git-rebase--*.sh scripts.
This workaround is therefore no longer necessary, so clean
up the code by reverting it.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Acked-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since a1549e10, 15d4bf2e and 01a1e646 (first appearing in v1.8.4)
the git-rebase--*.sh scripts have used a "return" to stop execution
of the dot-sourced file and return to the "dot" command that
dot-sourced it. The /bin/sh utility on FreeBSD however behaves
poorly under some circumstances when such a "return" is executed.
In particular, if the "dot" command is contained within a function,
then when a "return" is executed by the script it runs (that is not
itself inside a function), control will return from the function
that contains the "dot" command skipping any statements that might
follow the dot command inside that function. Commit 99855ddf (first
appearing in v1.8.4.1) addresses this by making the "dot" command
the last line in the function.
Unfortunately the FreeBSD /bin/sh may also execute some statements
in the script run by the "dot" command that appear after the
troublesome "return". The fix in 99855ddf does not address this
problem.
For example, if you have script1.sh with these contents:
run_script2() {
. "$(dirname -- "$0")/script2.sh"
_e=$?
echo only this line should show
[ $_e -eq 5 ] || echo expected status 5 got $_e
return 3
}
run_script2
e=$?
[ $e -eq 3 ] || { echo expected status 3 got $e; exit 1; }
And script2.sh with these contents:
if [ 5 -gt 3 ]; then
return 5
fi
case bad in *)
echo always shows
esac
echo should not get here
! :
When running script1.sh (e.g. '/bin/sh script1.sh' or './script1.sh'
after making it executable), the expected output from a POSIX shell
is simply the single line:
only this line should show
However, when run using FreeBSD's /bin/sh, the following output
appears instead:
should not get here
expected status 3 got 1
Not only did the lines following the "dot" command in the run_script2
function in script1.sh get skipped, but additional lines in script2.sh
following the "return" got executed -- but not all of them (e.g. the
"echo always shows" line did not run).
These issues can be avoided by not using a top-level "return" in
script2.sh. If script2.sh is changed to this:
main() {
if [ 5 -gt 3 ]; then
return 5
fi
case bad in *)
echo always shows
esac
echo should not get here
! :
}
main
Then it behaves the same when using FreeBSD's /bin/sh as when using
other more POSIX compliant /bin/sh implementations.
We fix the git-rebase--*.sh scripts in a similar fashion by moving
the top-level code that contains "return" statements into its own
function and then calling that as the last line in the script.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Acked-by: Matthieu Moy <Matthieu.Moy@imag.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since fd0a8c2e (first appearing in v1.7.0), the
t/t5560-http-backend-noserver.sh test has used a backslash escape
inside a ${} expansion in order to specify a literal '?' character.
Unfortunately the FreeBSD /bin/sh does not interpret this correctly.
In a POSIX compliant shell, the following:
x='one?two?three'
echo "${x#*\?}"
Would be expected to produce this:
two?three
When using the FreeBSD /bin/sh instead you get this:
one?two?three
In fact the FreeBSD /bin/sh treats the backslash as a literal
character to match so that this:
y='one\two\three'
echo "${y#*\?}"
Produces this unexpected value:
wo\three
In this case the backslash is not only treated literally, it also
fails to defeat the special meaning of the '?' character.
Instead, we can use the [...] construct to defeat the special meaning
of the '?' character and match it exactly in a way that works for the
FreeBSD /bin/sh as well as other POSIX /bin/sh implementations.
Changing the example like so:
x='one?two?three'
echo "${x#*[?]}"
Produces the expected output using the FreeBSD /bin/sh.
Therefore, change the use of \? to [?] in order to be compatible with
the FreeBSD /bin/sh which allows t/t5560-http-backend-noserver.sh to
pass on FreeBSD again.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since 11502468 and 04c1ee57 (both first appearing in v1.8.5), the
t7001-mv test has used "cp -a" to perform a copy in several of the
tests.
However, the "-a" option is not required for a POSIX cp utility and
some platforms' cp utilities do not support it.
The POSIX equivalent of -a is -R -P -p.
Change "cp -a" to "cp -R -P -p" so that the t7001-mv test works
on systems with a cp utility that only implements the POSIX
required set of options and not the "-a" option.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The second maintenance release for Git 1.9; contains all the fixes
that are scheduled to appear in Git 2.0.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jl/nor-or-nand-and:
code and test: fix misuses of "nor"
comments: fix misuses of "nor"
contrib: fix misuses of "nor"
Documentation: fix misuses of "nor"
* cn/fetch-prune-overlapping-destination:
fetch: handle overlaping refspecs on --prune
fetch: add a failing test for prunning with overlapping refspecs
Unicode 6.3 defines more code points as combining or accents. For
example, the character "ö" could be expressed as an "o" followed by
U+0308 COMBINING DIARESIS (aka umlaut, double-dot-above). We should
consider that such a sequence of two codepoints occupies one display
column for the alignment purposes, and for that, git_wcwidth()
should return 0 for them. Affected codepoints are:
U+0358..U+035C
U+0487
U+05A2, U+05BA, U+05C5, U+05C7
U+0604, U+0616..U+061A, U+0659..U+065F
Earlier unicode standards had defined these as "reserved".
Only the range 0..U+07FF has been checked to see which codepoints
need to be marked as 0-width while preparing for this commit; more
updates may be needed.
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* bp/commit-p-editor:
run-command: mark run_hook_with_custom_index as deprecated
merge hook tests: fix and update tests
merge: fix GIT_EDITOR override for commit hook
commit: fix patch hunk editing with "commit -p -m"
test patch hunk editing with "commit -p -m"
merge hook tests: use 'test_must_fail' instead of '!'
merge hook tests: fix missing '&&' in test
* mh/remove-subtree-long-pathname-fix:
entry.c: fix possible buffer overflow in remove_subtree()
checkout_entry(): use the strbuf throughout the function
* jk/shallow-update-fix:
shallow: verify shallow file after taking lock
shallow: automatically clean up shallow tempfiles
shallow: use stat_validity to check for up-to-date file
* jn/wt-status:
wt-status: lift the artificual "at least 20 columns" floor
wt-status: i18n of section labels
wt-status: extract the code to compute width for labels
wt-status: make full label string to be subject to l10n
Signed-off-by: Aman Gupta <aman@tmm1.net>
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Acked-by: Brad King <brad.king@kitware.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
One of the tests in t4212 checks our behavior when we feed
gmtime a date so far in the future that it gives up and
returns NULL. Some implementations, like AIX, may actually
just provide us a bogus result instead.
It's not worth it for us to come up with heuristics that
guess whether the return value is sensible or not. On good
platforms where gmtime reports the problem to us with NULL,
we will print the epoch value. On bad platforms, we will
print garbage. But our test should be written for the
lowest common denominator so that it passes everywhere.
Reported-by: Charles Bailey <cbailey32@bloomberg.net>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
