Handle memory pressure and file descriptor pressure separately when
deciding to release pack windows to honor resource limits.
* bc/unuse-packfile:
Don't close pack fd when free'ing pack windows
sha1_file: introduce close_one_pack() to close packs on fd pressure
* da/darwin:
OS X: Fix redeclaration of die warning
Makefile: Fix APPLE_COMMON_CRYPTO with BLK_SHA1
imap-send: use Apple's Security framework for base64 encoding
compat/apple-common-crypto.h uses die() in one of its macros, but was
included in git-compat-util.h before the definition of die.
Fix by simply moving the relevant block after the die/error/warning
declarations.
Signed-off-by: Brian Gernhardt <brian@gernhardtsoftware.com>
Reviewed-by: Jeremy Huddleston Sequoia <jeremyhu@apple.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Cygwin port added a "not quite correct but a lot faster and good
enough for many lstat() calls that are only used to see if the
working tree entity matches the index entry" lstat() emulation some
time ago, and it started biting us in places. This removes it and
uses the standard lstat() that comes with Cygwin.
Recent topic that uses lstat on packed-refs file is broken when
this cheating lstat is used, and this is a simplest fix that is
also the cleanest direction to go in the long run.
* rj/cygwin-clarify-use-of-cheating-lstat:
cygwin: Remove the Win32 l/stat() implementation
Now that close_one_pack() has been introduced to handle file
descriptor pressure, it is not strictly necessary to close the
pack file descriptor in unuse_one_window() when we're under memory
pressure.
Jeff King provided a justification for leaving the pack file open:
If you close packfile descriptors, you can run into racy situations
where somebody else is repacking and deleting packs, and they go away
while you are trying to access them. If you keep a descriptor open,
you're fine; they last to the end of the process. If you don't, then
they disappear from under you.
For normal object access, this isn't that big a deal; we just rescan
the packs and retry. But if you are packing yourself (e.g., because
you are a pack-objects started by upload-pack for a clone or fetch),
it's much harder to recover (and we print some warnings).
Let's do so (or uh, not do so).
Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use Apple's supported functions for base64 encoding instead
of the deprecated OpenSSL functions.
Signed-off-by: Jeremy Huddleston <jeremyhu@apple.com>
Signed-off-by: David Aguilar <davvid@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use the function attributes extension to catch mistakes in use of
our own variadic functions that use NULL sentinel at the end
(i.e. like execl(3)) and format strings (i.e. like printf(3)).
* jk/gcc-function-attributes:
Add the LAST_ARG_MUST_BE_NULL macro
wt-status: use "format" function attribute for status_printf
use "sentinel" function attribute for variadic lists
add missing "format" function attributes
The sentinel function attribute is not understood by versions of
the gcc compiler prior to v4.0. At present, for earlier versions
of gcc, the build issues 108 warnings related to the unknown
attribute. In order to suppress the warnings, we conditionally
define the LAST_ARG_MUST_BE_NULL macro to provide the sentinel attribute
for gcc v4.0 and newer.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit adbc0b6b ("cygwin: Use native Win32 API for stat", 30-09-2008)
added a Win32 specific implementation of the stat functions. In order
to handle absolute paths, cygwin mount points and symbolic links, this
implementation may fall back on the standard cygwin l/stat() functions.
Also, the choice of cygwin or Win32 functions is made lazily (by the
first call(s) to l/stat) based on the state of some config variables.
Unfortunately, this "schizophrenic stat" implementation has been the
source of many problems ever since. For example, see commits 7faee6b8,
79748439, 452993c2, 085479e7, b8a97333, 924aaf3e, 05bab3ea and 0117c2f0.
In order to avoid further problems, such as the issue raised by the new
reference handling API, remove the Win32 l/stat() implementation.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Update build for Cygwin 1.[57]. Torsten Bögershausen reports that
this is fine with Cygwin 1.7 ($gmane/225824) so let's try moving it
ahead.
* rj/mingw-cygwin:
cygwin: Remove the CYGWIN_V15_WIN32API build variable
mingw: rename WIN32 cpp macro to GIT_WINDOWS_NATIVE
Mac OS X does not like to write(2) more than INT_MAX number of
bytes.
* fc/macos-x-clipped-write:
compate/clipped-write.c: large write(2) fails on Mac OS X/XNU
On MinGW, sparse issues an "'get_st_mode_bits' not declared. Should
it be static?" warning. The MinGW and MSVC builds do not see the
declaration of this function, within git-compat-util.h, due to its
placement within an preprocessor conditional.
In order to suppress the warning, we simply move the declaration to
the top level of the header.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When $HOME is misconfigured to point at an unreadable directory, we
used to complain and die. This loosens the check.
* jn/config-ignore-inaccessible:
config: allow inaccessible configuration under $HOME
Due to a bug in the Darwin kernel, write(2) calls have a maximum size
of INT_MAX bytes.
Introduce a new compat function, clipped_write(), that only writes
at most INT_MAX bytes and returns the number of bytes written, as
a substitute for write(2), and allow platforms that need this to
enable it from the build mechanism with NEEDS_CLIPPED_WRITE.
Set it for Mac OS X by default. It may be necessary to include this
function on Windows, too.
Signed-off-by: Filipe Cabecinhas <filcab+git@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Throughout git, it is assumed that the WIN32 preprocessor symbol is
defined on native Windows setups (mingw and msvc) and not on Cygwin.
On Cygwin, most of the time git can pretend this is just another Unix
machine, and Windows-specific magic is generally counterproductive.
Unfortunately Cygwin *does* define the WIN32 symbol in some headers.
Best to rely on a new git-specific symbol GIT_WINDOWS_NATIVE instead,
defined as follows:
#if defined(WIN32) && !defined(__CYGWIN__)
# define GIT_WINDOWS_NATIVE
#endif
After this change, it should be possible to drop the
CYGWIN_V15_WIN32API setting without any negative effect.
[rj: %s/WINDOWS_NATIVE/GIT_WINDOWS_NATIVE/g ]
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A regression fix for the logic to detect die() handler triggering
itself recursively.
* jk/a-thread-only-dies-once:
run-command: use thread-aware die_is_recursing routine
usage: allow pluggable die-recursion checks
When any git code calls die or die_errno, we use a counter
to detect recursion into the die functions from any of the
helper functions. However, such a simple counter is not good
enough for threaded programs, which may call die from a
sub-thread, killing only the sub-thread (but incrementing
the counter for everyone).
Rather than try to deal with threads ourselves here, let's
just allow callers to plug in their own recursion-detection
function. This is similar to how we handle the die routine
(the caller plugs in a die routine which may kill only the
sub-thread).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The changes v1.7.12.1~2^2~4 (config: warn on inaccessible files,
2012-08-21) and v1.8.1.1~22^2~2 (config: treat user and xdg config
permission problems as errors, 2012-10-13) were intended to prevent
important configuration (think "[transfer] fsckobjects") from being
ignored when the configuration is unintentionally unreadable (for
example with EIO on a flaky filesystem, or with ENOMEM due to a DoS
attack). Usually ~/.gitconfig and ~/.config/git are readable by the
current user, and if they aren't then it would be easy to fix those
permissions, so the damage from adding this check should have been
minimal.
Unfortunately the access() check often trips when git is being run as
a server. A daemon (such as inetd or git-daemon) starts as "root",
creates a listening socket, and then drops privileges, meaning that
when git commands are invoked they cannot access $HOME and die with
fatal: unable to access '/root/.config/git/config': Permission denied
Any patch to fix this would have one of three problems:
1. We annoy sysadmins who need to take an extra step to handle HOME
when dropping privileges (the current behavior, or any other
proposal that they have to opt into).
2. We annoy sysadmins who want to set HOME when dropping privileges,
either by making what they want to do impossible, or making them
set an extra variable or option to accomplish what used to work
(e.g., a patch to git-daemon to set HOME when --user is passed).
3. We loosen the check, so some cases which might be noteworthy are
not caught.
This patch is of type (3).
Treat user and xdg configuration that are inaccessible due to
permissions (EACCES) as though no user configuration was provided at
all.
An alternative method would be to check if $HOME is readable, but that
would not help in cases where the user who dropped privileges had a
globally readable HOME with only .config or .gitconfig being private.
This does not change the behavior when /etc/gitconfig or .git/config
is unreadable (since those are more serious configuration errors),
nor when ~/.gitconfig or ~/.config/git is unreadable due to problems
other than permissions.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Improved-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When core.sharedRepository is used, set_shared_perm() in path.c
needs lstat() to return the correct POSIX permissions.
The default for cygwin is core.ignoreCygwinFSTricks = false, which
means that the fast implementation in do_stat() is used instead of
lstat().
lstat() under cygwin uses the Windows security model to implement
POSIX-like permissions. The user, group or everyone bits can be set
individually.
do_stat() simplifes the file permission bits, and may return a wrong
value. The read-only attribute of a file is used to calculate the
permissions, resulting in either rw-r--r-- or r--r--r--
One effect of the simplified do_stat() is that t1301 fails.
Add a function cygwin_get_st_mode_bits() which returns the POSIX
permissions. When not compiling for cygwin, true_mode_bits() in
path.c is used.
Side note:
t1301 passes under cygwin 1.5.
The "user write" bit is synchronized with the "read only" attribute
of a file:
$ chmod 444 x
$ attrib x
A R C:\temp\pt\x
cygwin 1.7 would show
A C:\temp\pt\x
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint-1.8.1:
bundle: Add colons to list headings in "verify"
bundle: Fix "verify" output if history is complete
Documentation: filter-branch env-filter example
git-filter-branch.txt: clarify ident variables usage
git-compat-util.h: Provide missing netdb.h definitions
describe: Document --match pattern format
Documentation/githooks: Explain pre-rebase parameters
update-index: list supported idx versions and their features
diff-options: unconfuse description of --color
read-cache.c: use INDEX_FORMAT_{LB,UB} in verify_hdr()
index-format.txt: mention of v4 is missing in some places
Some sources failed to compile on systems that lack NI_MAXHOST in
their system header.
* dm/ni-maxhost-may-be-missing:
git-compat-util.h: Provide missing netdb.h definitions
On systems without NI_MAXHOST in their system header files,
connect.c (hence most of the transport) did not compile.
* dm/ni-maxhost-may-be-missing:
git-compat-util.h: Provide missing netdb.h definitions
This reverts commit 78457bc0cc.
commit 28c5d9e ("vcs-svn: drop string_pool") previously removed
the only call-site for strtok_r. So let's get rid of the compat
implementation as well.
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 0f77dea9 ("mingw: move poll out of sys-folder", 24-10-2011), along
with other commits in the 'ef/mingw-upload-archive' branch (see commit
7406aa20), effectively reintroduced the same problem addressed by commit
56fb3ddc ("msvc: Fix compilation errors in compat/win32/sys/poll.c",
04-12-2010).
In order to fix the compilation errors, we use the same solution adopted
in that earlier commit. In particular, we set _WIN32_WINNT to 0x0502
(which would target Windows Server 2003) prior to including the winsock2.h
header file.
Also, we delete the compat/vcbuild/include/sys/poll.h header file, since
it is now redundant and it's presence may cause some confusion.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Tested-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some platforms may lack the NI_MAXHOST and NI_MAXSERV values in their
system headers, so ensure they are available.
Signed-off-by: David Michael <fedora.dm0@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
QNX 6.3.2 uses GCC 2.95.3 by default, and GCC 2.95.3 doesn't remove the
comma if the error macro's variable argument is left out.
Instead of testing for a sufficiently recent version of GCC, make
__VA_ARGS__ match all of the arguments.
Signed-off-by: Matt Kraai <matt.kraai@amo.abbott.com>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Replace our use of fnmatch(3) with a more feature-rich wildmatch.
A handful patches at the bottom have been moved to nd/wildmatch to
graduate as part of that branch, before this series solidifies.
We may want to mark USE_WILDMATCH as an experimental curiosity a
bit more clearly (i.e. should not be enabled in production
environment, because it will make the behaviour between builds
unpredictable).
* nd/retire-fnmatch:
Makefile: add USE_WILDMATCH to use wildmatch as fnmatch
wildmatch: advance faster in <asterisk> + <literal> patterns
wildmatch: make a special case for "*/" with FNM_PATHNAME
test-wildmatch: add "perf" command to compare wildmatch and fnmatch
wildmatch: support "no FNM_PATHNAME" mode
wildmatch: make dowild() take arbitrary flags
wildmatch: rename constants and update prototype
Commit a469a10 wraps some error calls in macros to give the
compiler a chance to do more static analysis on their
constant -1 return value. We limit the use of these macros
to __GNUC__, since gcc is the primary beneficiary of the new
information, and because we use GNU features for handling
variadic macros.
However, clang also defines __GNUC__, but generates warnings
with -Wunused-value when these macros are used in a void
context, because the constant "-1" ends up being useless.
Gcc does not complain about this case (though it is unclear
if it is because it is smart enough to see what we are
doing, or too dumb to realize that the -1 is unused). We
can squelch the warning by just disabling these macros when
clang is in use.
Signed-off-by: Max Horn <max@quendi.de>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When attempting to read the XDG-style $HOME/.config/git/config and
finding that $HOME/.config/git is a file, we gave a wrong error
message, instead of treating the case as "a custom config file does
not exist there" and moving on.
* jn/warn-on-inaccessible-loosen:
config: exit on error accessing any config file
doc: advertise GIT_CONFIG_NOSYSTEM
config: treat user and xdg config permission problems as errors
config, gitignore: failure to access with ENOTDIR is ok
Allows pathname patterns in .gitignore and .gitattributes files
with double-asterisks "foo/**/bar" to match any number of directory
hierarchies.
* nd/wildmatch:
wildmatch: replace variable 'special' with better named ones
compat/fnmatch: respect NO_FNMATCH* even on glibc
wildmatch: fix "**" special case
t3070: Disable some failing fnmatch tests
test-wildmatch: avoid Windows path mangling
Support "**" wildcard in .gitignore and .gitattributes
wildmatch: make /**/ match zero or more directories
wildmatch: adjust "**" behavior
wildmatch: fix case-insensitive matching
wildmatch: remove static variable force_lower_case
wildmatch: make wildmatch's return value compatible with fnmatch
t3070: disable unreliable fnmatch tests
Integrate wildmatch to git
wildmatch: follow Git's coding convention
wildmatch: remove unnecessary functions
Import wildmatch from rsync
ctype: support iscntrl, ispunct, isxdigit and isprint
ctype: make sane_ctype[] const array
Conflicts:
Makefile
Deal with a situation where .config/git is a file and we notice
.config/git/config is not readable due to ENOTDIR, not ENOENT.
* jn/warn-on-inaccessible-loosen:
config: exit on error accessing any config file
doc: advertise GIT_CONFIG_NOSYSTEM
config: treat user and xdg config permission problems as errors
config, gitignore: failure to access with ENOTDIR is ok
Help compilers' flow analysis by making it more explicit that
error() always returns -1, to reduce false "variable used
uninitialized" warnings. Looks somewhat ugly but not too much.
* jk/error-const-return:
silence some -Wuninitialized false positives
make error()'s constant return value more visible
This is similar to NO_FNMATCH but it uses wildmatch instead of
compat/fnmatch. This is an intermediate step to let wildmatch be used
as fnmatch replacement for wider audience before it replaces fnmatch
completely and compat/fnmatch is removed.
fnmatch in test-wildmatch is not impacted by this and is the only
place that NO_FNMATCH or NO_FNMATCH_CASEFOLD remain active when
USE_WILDMATCH is set.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Earlier we allowed platforms that lack <sys/param.h> not to include
the header file from git-compat-util.h; we have included this header
file since the early days back when we used MAXPATHLEN (which we no
longer use) and also depended on it slurping ULONG_MAX (which we get
by including stdint.h or inttypes.h these days).
It turns out that we can compile our modern codebase just file
without including it on many platforms (so far, Fedora, Debian,
Ubuntu, MinGW, Mac OS X, Cygwin, HP-Nonstop, QNX and z/OS are
reported to be OK).
Let's stop including it by default, and on platforms that need it to
be included, leave "make NEEDS_SYS_PARAM_H=YesPlease" as an escape
hatch and ask them to report to us, so that we can find out about
the real dependency and fix it in a more platform agnostic way.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When git is compiled with "gcc -Wuninitialized -O3", some
inlined calls provide an additional opportunity for the
compiler to do static analysis on variable initialization.
For example, with two functions like this:
int get_foo(int *foo)
{
if (something_that_might_fail() < 0)
return error("unable to get foo");
*foo = 0;
return 0;
}
void some_fun(void)
{
int foo;
if (get_foo(&foo) < 0)
return -1;
printf("foo is %d\n", foo);
}
If get_foo() is not inlined, then when compiling some_fun,
gcc sees only that a pointer to the local variable is
passed, and must assume that it is an out parameter that
is initialized after get_foo returns.
However, when get_foo() is inlined, the compiler may look at
all of the code together and see that some code paths in
get_foo() do not initialize the variable. As a result, it
prints a warning. But what the compiler can't see is that
error() always returns -1, and therefore we know that either
we return early from some_fun, or foo ends up initialized,
and the code is safe. The warning is a false positive.
If we can make the compiler aware that error() will always
return -1, it can do a better job of analysis. The simplest
method would be to inline the error() function. However,
this doesn't work, because gcc will not inline a variadc
function. We can work around this by defining a macro. This
relies on two gcc extensions:
1. Variadic macros (these are present in C99, but we do
not rely on that).
2. Gcc treats the "##" paste operator specially between a
comma and __VA_ARGS__, which lets our variadic macro
work even if no format parameters are passed to
error().
Since we are using these extra features, we hide the macro
behind an #ifdef. This is OK, though, because our goal was
just to help gcc.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The header strings.h was formerly only included for HP NonStop (aka
Tandem) to define strcasecmp, but another platform requiring this
inclusion has been found. The build system will now include the
file based on its presence determined by configure.
Signed-off-by: David Michael <fedora.dm0@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
An option is added to the Makefile to skip the inclusion of sys/param.h.
The only known platform with this condition thus far is the z/OS UNIX System
Services environment.
Signed-off-by: David Michael <fedora.dm0@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Various rfc2047 quoting issues around a non-ASCII name on the From:
line in the output from format-patch have been corrected.
* js/format-2047:
format-patch tests: check quoting/encoding in To: and Cc: headers
format-patch: fix rfc2047 address encoding with respect to rfc822 specials
format-patch: make rfc2047 encoding more strict
format-patch: introduce helper function last_line_length()
format-patch: do not wrap rfc2047 encoded headers too late
format-patch: do not wrap non-rfc2047 headers too early
utf8: fix off-by-one wrapping of text
Fixes many rfc2047 quoting issues in the output from format-patch.
* js/format-2047:
format-patch tests: check quoting/encoding in To: and Cc: headers
format-patch: fix rfc2047 address encoding with respect to rfc822 specials
format-patch: make rfc2047 encoding more strict
format-patch: introduce helper function last_line_length()
format-patch: do not wrap rfc2047 encoded headers too late
format-patch: do not wrap non-rfc2047 headers too early
utf8: fix off-by-one wrapping of text
RFC 2047 requires more characters to be encoded than it is currently done.
Especially, RFC 2047 distinguishes between allowed remaining characters
in encoded words in addresses (From, To, etc.) and other headers, such
as Subject.
Make add_rfc2047() and is_rfc2047_special() location dependent and include
all non-allowed characters to hopefully be RFC 2047 conformant.
This especially fixes a problem, where RFC 822 specials (e. g. ".") were
left unencoded in addresses, which was solved with a non-standard-conforming
workaround in the past (which is going to be removed in a follow-up patch).
Signed-off-by: Jan H. Schönherr <schnhrr@cs.tu-berlin.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git reads multiple configuration files: settings come first from the
system config file (typically /etc/gitconfig), then the xdg config
file (typically ~/.config/git/config), then the user's dotfile
(~/.gitconfig), then the repository configuration (.git/config).
Git has always used access(2) to decide whether to use each file; as
an unfortunate side effect, that means that if one of these files is
unreadable (e.g., EPERM or EIO), git skips it. So if I use
~/.gitconfig to override some settings but make a mistake and give it
the wrong permissions then I am subject to the settings the sysadmin
chose for /etc/gitconfig.
Better to error out and ask the user to correct the problem.
This only affects the user and xdg config files, since the user
presumably has enough access to fix their permissions. If the system
config file is unreadable, the best we can do is to warn about it so
the user knows to notify someone and get on with work in the meantime.
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The access_or_warn() function is used to check for optional
configuration files like .gitconfig and .gitignore and warn when they
are not accessible due to a configuration issue (e.g., bad
permissions). It is not supposed to complain when a file is simply
missing.
Noticed on a system where ~/.config/git was a file --- when the new
XDG_CONFIG_HOME support looks for ~/.config/git/config it should
ignore ~/.config/git instead of printing irritating warnings:
$ git status -s
warning: unable to access '/home/jrn/.config/git/config': Not a directory
warning: unable to access '/home/jrn/.config/git/config': Not a directory
warning: unable to access '/home/jrn/.config/git/config': Not a directory
warning: unable to access '/home/jrn/.config/git/config': Not a directory
Compare v1.7.12.1~2^2 (attr:failure to open a .gitattributes file
is OK with ENOTDIR, 2012-09-13).
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
