1
0
Fork 0
mirror of https://github.com/git/git.git synced 2024-11-16 06:03:44 +01:00
Find a file
Steffen Prohaska 9035d75a2b convert: stream from fd to required clean filter to reduce used address space
The data is streamed to the filter process anyway.  Better avoid mapping
the file if possible.  This is especially useful if a clean filter
reduces the size, for example if it computes a sha1 for binary data,
like git media.  The file size that the previous implementation could
handle was limited by the available address space; large files for
example could not be handled with (32-bit) msysgit.  The new
implementation can filter files of any size as long as the filter output
is small enough.

The new code path is only taken if the filter is required.  The filter
consumes data directly from the fd.  If it fails, the original data is
not immediately available.  The condition can easily be handled as
a fatal error, which is expected for a required filter anyway.

If the filter was not required, the condition would need to be handled
in a different way, like seeking to 0 and reading the data.  But this
would require more restructuring of the code and is probably not worth
it.  The obvious approach of falling back to reading all data would not
help achieving the main purpose of this patch, which is to handle large
files with limited address space.  If reading all data is an option, we
can simply take the old code path right away and mmap the entire file.

The environment variable GIT_MMAP_LIMIT, which has been introduced in
a previous commit is used to test that the expected code path is taken.
A related test that exercises required filters is modified to verify
that the data actually has been modified on its way from the file system
to the object store.

Signed-off-by: Steffen Prohaska <prohaska@zib.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-08-28 10:25:15 -07:00
block-sha1
builtin Merge branch 'jk/misc-fixes-maint' 2014-07-28 11:30:41 -07:00
compat Merge branch 'sk/mingw-uni-fix-more' 2014-07-30 14:21:09 -07:00
contrib various contrib: Fix links in man pages 2014-08-07 09:43:21 -07:00
Documentation Git 2.1 2014-08-15 15:09:12 -07:00
ewah
git-gui Merge git://repo.or.cz/git-gui 2014-06-26 13:44:11 -07:00
gitk-git Merge branch 'master' of git://ozlabs.org/~paulus/gitk 2014-08-10 11:03:03 -07:00
gitweb
mergetools
perl Merge branch 'mm/pager-less-sans-S' 2014-06-06 11:02:59 -07:00
po gitk: Updated Bulgarian translation (302t,0f,0u) 2014-08-08 16:39:30 +10:00
ppc
t convert: stream from fd to required clean filter to reduce used address space 2014-08-28 10:25:15 -07:00
templates
vcs-svn
xdiff git-merge-file: do not add LF at EOF while applying unrelated change 2014-06-30 14:07:58 -07:00
.gitattributes
.gitignore Merge branch 'nd/split-index' 2014-07-16 11:25:40 -07:00
.mailmap Sync with v2.0.3 2014-07-23 11:36:40 -07:00
abspath.c
aclocal.m4
advice.c refactor skip_prefix to return a boolean 2014-06-20 10:44:43 -07:00
advice.h
alias.c use skip_prefix to avoid magic numbers 2014-06-20 10:44:45 -07:00
alloc.c alloc: factor out commit index 2014-07-28 10:14:33 -07:00
archive-tar.c
archive-zip.c
archive.c
archive.h
argv-array.c
argv-array.h
attr.c
attr.h
base85.c
bisect.c
bisect.h
blob.c add object_as_type helper for casting objects 2014-07-28 10:14:33 -07:00
blob.h
branch.c refactor skip_prefix to return a boolean 2014-06-20 10:44:43 -07:00
branch.h
builtin.h verify-commit: scriptable commit signature verification 2014-06-23 15:50:31 -07:00
bulk-checkin.c
bulk-checkin.h
bundle.c bundle: use internal argv_array of struct child_process in create_bundle() 2014-07-18 12:14:47 -07:00
bundle.h
cache-tree.c cache-tree: mark istate->cache_changed on prime_cache_tree() 2014-06-13 11:49:39 -07:00
cache-tree.h cache-tree: mark istate->cache_changed on prime_cache_tree() 2014-06-13 11:49:39 -07:00
cache.h config.c: add git_env_ulong() to parse environment variable 2014-08-28 10:24:54 -07:00
check-builtins.sh
check-racy.c
check_bindir check_bindir: avoid "test <cond> -a/-o <cond>" 2014-06-09 14:47:06 -07:00
color.c
color.h
column.c refactor skip_prefix to return a boolean 2014-06-20 10:44:43 -07:00
column.h
combine-diff.c Merge branch 'mk/show-s-no-extra-blank-line-for-merges' into maint 2014-06-25 11:49:39 -07:00
command-list.txt verify-commit: scriptable commit signature verification 2014-06-23 15:50:31 -07:00
commit-slab.h commit-slab: provide a static initializer 2014-06-13 12:08:17 -07:00
commit.c add object_as_type helper for casting objects 2014-07-28 10:14:33 -07:00
commit.h Merge branch 'cc/replace-graft' 2014-07-27 15:14:18 -07:00
config.c config.c: add git_env_ulong() to parse environment variable 2014-08-28 10:24:54 -07:00
config.mak.in
config.mak.uname Merge branch 'sk/mingw-uni-fix-more' 2014-07-30 14:21:09 -07:00
configure.ac
connect.c Merge branch 'maint' 2014-07-21 12:35:39 -07:00
connect.h
connected.c use strip_suffix instead of ends_with in simple cases 2014-06-30 13:43:17 -07:00
connected.h
convert.c convert: stream from fd to required clean filter to reduce used address space 2014-08-28 10:25:15 -07:00
convert.h convert: stream from fd to required clean filter to reduce used address space 2014-08-28 10:25:15 -07:00
copy.c copy_fd(): do not close the input file descriptor 2014-08-28 10:25:14 -07:00
COPYING
credential-cache--daemon.c refactor skip_prefix to return a boolean 2014-06-20 10:44:43 -07:00
credential-cache.c
credential-store.c
credential.c refactor skip_prefix to return a boolean 2014-06-20 10:44:43 -07:00
credential.h
csum-file.c
csum-file.h
ctype.c
daemon.c Merge branch 'cc/replace-edit' 2014-07-16 11:25:47 -07:00
date.c
decorate.c hashmap: factor out getting a hash code from a SHA1 2014-07-07 13:56:24 -07:00
decorate.h
delta.h
diff-delta.c
diff-lib.c Merge branch 'jk/diff-files-assume-unchanged' into maint 2014-06-25 11:47:09 -07:00
diff-no-index.c
diff.c Merge branch 'bg/xcalloc-nmemb-then-size' into maint 2014-07-22 10:25:17 -07:00
diff.h
diffcore-break.c
diffcore-delta.c
diffcore-order.c
diffcore-pickaxe.c
diffcore-rename.c hashmap: add simplified hashmap_get_from_hash() API 2014-07-07 13:56:35 -07:00
diffcore.h
dir.c prep_exclude: remove the artificial PATH_MAX limit 2014-07-14 15:24:34 -07:00
dir.h prep_exclude: remove the artificial PATH_MAX limit 2014-07-14 15:24:34 -07:00
editor.c
entry.c read-cache: mark updated entries for split index 2014-06-13 11:49:40 -07:00
environment.c setup_git_env(): introduce git_path_from_env() helper 2014-06-25 10:33:27 -07:00
exec_cmd.c
exec_cmd.h
fast-import.c fast-import: use hashcmp() for SHA1 hash comparison 2014-07-18 12:14:47 -07:00
fetch-pack.c Merge branch 'jk/skip-prefix' 2014-07-09 11:33:28 -07:00
fetch-pack.h
fmt-merge-msg.h
fsck.c fsck: simplify fsck_commit_buffer() by using commit_list_count() 2014-07-10 14:10:27 -07:00
fsck.h
generate-cmdlist.sh
gettext.c
gettext.h
git-add--interactive.perl
git-am.sh
git-archimport.perl
git-bisect.sh git-bisect.sh: avoid "test <cond> -a/-o <cond>" 2014-06-09 14:47:07 -07:00
git-compat-util.h Revert "Merge branch 'dt/refs-check-refname-component-sse'" 2014-07-28 10:41:53 -07:00
git-cvsexportcommit.perl
git-cvsimport.perl
git-cvsserver.perl
git-difftool--helper.sh
git-difftool.perl
git-filter-branch.sh filter-branch: eliminate duplicate mapped parents 2014-07-01 08:30:41 -07:00
git-instaweb.sh
git-merge-octopus.sh
git-merge-one-file.sh
git-merge-resolve.sh
git-mergetool--lib.sh
git-mergetool.sh git-mergetool.sh: avoid "test <cond> -a/-o <cond>" 2014-06-09 14:47:07 -07:00
git-p4.py git-p4: fix submit in non --prepare-p4-only mode 2014-06-13 11:04:04 -07:00
git-parse-remote.sh
git-pull.sh Sync with maint 2014-06-12 12:22:38 -07:00
git-quiltimport.sh
git-rebase--am.sh rebase: omit patch-identical commits with --fork-point 2014-07-16 13:07:40 -07:00
git-rebase--interactive.sh rebase: omit patch-identical commits with --fork-point 2014-07-16 13:07:40 -07:00
git-rebase--merge.sh Merge branch 'bc/fix-rebase-merge-skip' into maint 2014-07-16 11:16:16 -07:00
git-rebase.sh rebase: omit patch-identical commits with --fork-point 2014-07-16 13:07:40 -07:00
git-relink.perl
git-remote-testgit.sh Merge branch 'ep/shell-assign-and-export-vars' into maint 2014-07-22 10:22:57 -07:00
git-request-pull.sh
git-send-email.perl Merge branch 'mt/send-email-cover-to-cc' 2014-06-20 13:12:20 -07:00
git-sh-i18n.sh
git-sh-setup.sh
git-stash.sh Merge branch 'ep/shell-assign-and-export-vars' into maint 2014-07-22 10:22:57 -07:00
git-submodule.sh git-submodule.sh: avoid "echo" path-like values 2014-06-19 13:30:03 -07:00
git-svn.perl
GIT-VERSION-GEN Git 2.1 2014-08-15 15:09:12 -07:00
git-web--browse.sh
git.c Merge branch 'kb/perf-trace' 2014-07-22 10:59:19 -07:00
git.rc
git.spec.in
gpg-interface.c gpg-interface: provide access to the payload 2014-06-23 15:50:30 -07:00
gpg-interface.h gpg-interface: provide access to the payload 2014-06-23 15:50:30 -07:00
graph.c
graph.h
grep.c
grep.h
hashmap.c hashmap: add string interning API 2014-07-07 13:56:38 -07:00
hashmap.h hashmap: add string interning API 2014-07-07 13:56:38 -07:00
help.c Merge branch 'jk/strip-suffix' 2014-07-16 11:26:00 -07:00
help.h
hex.c
http-backend.c Merge branch 'maint' 2014-07-21 12:35:39 -07:00
http-fetch.c
http-push.c Merge branch 'ah/fix-http-push' into maint 2014-07-22 10:29:07 -07:00
http-walker.c use xstrfmt to replace xmalloc + sprintf 2014-06-19 15:20:54 -07:00
http.c Merge branch 'jk/skip-prefix' 2014-07-09 11:33:28 -07:00
http.h
ident.c
imap-send.c imap-send: use skip_prefix instead of using magic numbers 2014-06-20 10:45:19 -07:00
INSTALL Fix profile feedback with -jN and add profile-fast 2014-07-08 10:56:47 -07:00
khash.h hashmap: factor out getting a hash code from a SHA1 2014-07-07 13:56:24 -07:00
kwset.c
kwset.h
levenshtein.c
levenshtein.h
LGPL-2.1
line-log.c use commit_list_count() to count the members of commit_lists 2014-07-17 13:36:25 -07:00
line-log.h
line-range.c
line-range.h
list-objects.c
list-objects.h
ll-merge.c
ll-merge.h
lockfile.c copy_fd(): do not close the input file descriptor 2014-08-28 10:25:14 -07:00
log-tree.c Merge branch 'zk/log-graph-showsig' into maint 2014-07-22 10:28:51 -07:00
log-tree.h
mailmap.c
mailmap.h
Makefile Merge branch 'kb/perf-trace' 2014-07-22 10:59:19 -07:00
match-trees.c use xstrfmt to replace xmalloc + sprintf 2014-06-19 15:20:54 -07:00
merge-blobs.c
merge-blobs.h
merge-recursive.c Merge branch 'ta/string-list-init' 2014-07-23 11:35:54 -07:00
merge-recursive.h
merge.c Merge branch 'nd/split-index' 2014-07-16 11:25:40 -07:00
mergesort.c
mergesort.h
name-hash.c hashmap: add simplified hashmap_get_from_hash() API 2014-07-07 13:56:35 -07:00
notes-cache.c Merge branch 'jk/commit-buffer-length' 2014-07-02 12:53:02 -07:00
notes-cache.h
notes-merge.c commit: record buffer length in cache 2014-06-13 12:09:38 -07:00
notes-merge.h
notes-utils.c Merge branch 'jk/commit-buffer-length' 2014-07-02 12:53:02 -07:00
notes-utils.h commit_tree: take a pointer/len pair rather than a const strbuf 2014-06-12 10:29:41 -07:00
notes.c
notes.h
object.c object_as_type: set commit index 2014-07-28 10:14:34 -07:00
object.h add object_as_type helper for casting objects 2014-07-28 10:14:33 -07:00
pack-bitmap-write.c
pack-bitmap.c
pack-bitmap.h
pack-check.c
pack-objects.c hashmap: factor out getting a hash code from a SHA1 2014-07-07 13:56:24 -07:00
pack-objects.h
pack-revindex.c
pack-revindex.h
pack-write.c
pack.h
pager.c Merge branch 'je/pager-do-not-recurse' 2014-06-06 11:17:00 -07:00
parse-options-cb.c
parse-options.c refactor skip_prefix to return a boolean 2014-06-20 10:44:43 -07:00
parse-options.h Merge branch 'jk/squelch-compiler-warning-from-funny-error-macro' 2014-06-06 11:21:36 -07:00
patch-delta.c
patch-ids.c
patch-ids.h
path.c Merge branch 'rs/code-cleaning' 2014-07-22 10:59:37 -07:00
pathspec.c Merge branch 'maint' 2014-07-21 12:35:39 -07:00
pathspec.h
pkt-line.c trace: improve trace performance 2014-07-13 21:24:23 -07:00
pkt-line.h
preload-index.c cache.h: rename cache_def_free to cache_def_clear 2014-07-13 10:12:37 -07:00
pretty.c Merge branch 'rs/code-cleaning' 2014-07-22 10:59:37 -07:00
prio-queue.c prio-queue: make output stable with respect to insertion 2014-07-15 11:02:54 -07:00
prio-queue.h prio-queue: make output stable with respect to insertion 2014-07-15 11:02:54 -07:00
progress.c progress: simplify performance measurement by using getnanotime() 2014-07-13 21:25:21 -07:00
progress.h
prompt.c
prompt.h
quote.c
quote.h
reachable.c
reachable.h
read-cache.c Merge branch 'nd/split-index' 2014-07-16 11:25:40 -07:00
README
reflog-walk.c
reflog-walk.h
refs.c Revert "Merge branch 'dt/refs-check-refname-component-sse'" 2014-07-28 10:41:53 -07:00
refs.h Merge branch 'rs/unify-is-branch' 2014-07-21 11:18:57 -07:00
RelNotes Git 2.0.4 2014-07-30 14:19:53 -07:00
remote-curl.c Merge branch 'jk/remote-curl-squelch-extra-errors' 2014-07-21 11:18:41 -07:00
remote-testsvn.c remote-testsvn: use internal argv_array of struct child_process in cmd_import() 2014-07-18 12:50:03 -07:00
remote.c Merge branch 'jk/xstrfmt' 2014-07-09 11:34:05 -07:00
remote.h
replace_object.c
rerere.c Merge branch 'rs/code-cleaning' 2014-07-16 11:33:09 -07:00
rerere.h
resolve-undo.c resolve-undo: be specific what part of the index has changed 2014-06-13 11:49:38 -07:00
resolve-undo.h
revision.c Merge branch 'jk/commit-buffer-length' into maint 2014-07-16 11:16:38 -07:00
revision.h
run-command.c Merge branch 'sk/mingw-uni-fix-more' 2014-07-30 14:21:09 -07:00
run-command.h
send-pack.c
send-pack.h
sequencer.c Merge branch 'nd/split-index' 2014-07-16 11:25:40 -07:00
sequencer.h
server-info.c
setup.c
sh-i18n--envsubst.c use xmemdupz() to allocate copies of strings given by start and length 2014-07-21 10:37:02 -07:00
sha1-array.c
sha1-array.h
sha1-lookup.c
sha1-lookup.h
sha1_file.c convert: stream from fd to required clean filter to reduce used address space 2014-08-28 10:25:15 -07:00
sha1_name.c Merge branch 'jk/misc-fixes-maint' 2014-07-28 11:30:41 -07:00
shallow.c trace: improve trace performance 2014-07-13 21:24:23 -07:00
shell.c use xstrfmt to replace xmalloc + strcpy/strcat 2014-06-19 15:20:54 -07:00
shortlog.h
show-index.c
sideband.c
sideband.h
sigchain.c
sigchain.h
split-index.c update-index: new options to enable/disable split index mode 2014-06-13 11:49:41 -07:00
split-index.h split-index: the reading part 2014-06-13 11:49:40 -07:00
strbuf.c Merge branch 'jk/strip-suffix' 2014-07-16 11:26:00 -07:00
strbuf.h Merge branch 'jk/strip-suffix' 2014-07-16 11:26:00 -07:00
streaming.c
streaming.h
string-list.c string-list: add string_list initializer helper function 2014-07-21 10:23:36 -07:00
string-list.h string-list: add string_list initializer helper function 2014-07-21 10:23:36 -07:00
submodule.c Merge branch 'ta/string-list-init' 2014-07-23 11:35:54 -07:00
submodule.h
symlinks.c symlinks: remove PATH_MAX limitation 2014-07-07 11:22:42 -07:00
tag.c add object_as_type helper for casting objects 2014-07-28 10:14:33 -07:00
tag.h
tar.h
test-chmtime.c
test-ctype.c
test-date.c
test-delta.c
test-dump-cache-tree.c cache-tree: mark istate->cache_changed on cache tree update 2014-06-13 11:49:39 -07:00
test-dump-split-index.c t1700: new tests for split-index mode 2014-06-13 11:49:42 -07:00
test-genrandom.c
test-hashmap.c hashmap: add string interning API 2014-07-07 13:56:38 -07:00
test-index-version.c
test-line-buffer.c
test-match-trees.c
test-mergesort.c
test-mktemp.c
test-parse-options.c
test-path-utils.c
test-prio-queue.c
test-read-cache.c
test-regex.c
test-revision-walking.c
test-run-command.c
test-scrap-cache-tree.c read-cache: new API write_locked_index instead of write_index/write_cache 2014-06-13 11:49:10 -07:00
test-sha1.c
test-sha1.sh
test-sigchain.c
test-string-list.c
test-subprocess.c
test-svn-fe.c
test-urlmatch-normalization.c
test-wildmatch.c
thread-utils.c
thread-utils.h
trace.c git: add performance tracing for git's main() function to debug scripts 2014-07-13 21:25:21 -07:00
trace.h git: add performance tracing for git's main() function to debug scripts 2014-07-13 21:25:21 -07:00
transport-helper.c transport-helper: avoid reading past end-of-string 2014-06-20 10:44:44 -07:00
transport.c Merge branch 'jk/misc-fixes-maint' 2014-07-28 11:30:41 -07:00
transport.h
tree-diff.c
tree-walk.c cleanup duplicate name_compare() functions 2014-06-20 10:12:14 -07:00
tree-walk.h
tree.c add object_as_type helper for casting objects 2014-07-28 10:14:33 -07:00
tree.h
unicode_width.h Update of unicode_width.h to Unicode Version 7.0 2014-06-18 10:53:45 -07:00
unimplemented.sh
unix-socket.c unix-socket: remove stale socket before calling chdir() 2014-07-21 09:38:07 -07:00
unix-socket.h
unpack-trees.c Merge branch 'nd/split-index' 2014-07-16 11:25:40 -07:00
unpack-trees.h
update_unicode.sh
upload-pack.c
url.c use strbuf_addch for adding single characters 2014-07-10 14:06:46 -07:00
url.h
urlmatch.c refactor skip_prefix to return a boolean 2014-06-20 10:44:43 -07:00
urlmatch.h
usage.c
userdiff.c
userdiff.h
utf8.c Merge branch 'tb/unicode-6.3-zero-width' 2014-06-06 11:29:38 -07:00
utf8.h
varint.c
varint.h
version.c
version.h
versioncmp.c
walker.c walker_fetch: fix minor memory leak 2014-06-19 15:20:56 -07:00
walker.h
wildmatch.c
wildmatch.h
wrap-for-bin.sh
wrapper.c memory_limit: use git_env_ulong() to parse GIT_ALLOC_LIMIT 2014-08-28 10:25:04 -07:00
write_or_die.c
ws.c
wt-status.c Merge branch 'kb/perf-trace' 2014-07-22 10:59:19 -07:00
wt-status.h
xdiff-interface.c
xdiff-interface.h
zlib.c

////////////////////////////////////////////////////////////////

	Git - the stupid content tracker

////////////////////////////////////////////////////////////////

"git" can mean anything, depending on your mood.

 - random three-letter combination that is pronounceable, and not
   actually used by any common UNIX command.  The fact that it is a
   mispronunciation of "get" may or may not be relevant.
 - stupid. contemptible and despicable. simple. Take your pick from the
   dictionary of slang.
 - "global information tracker": you're in a good mood, and it actually
   works for you. Angels sing, and a light suddenly fills the room.
 - "goddamn idiotic truckload of sh*t": when it breaks

Git is a fast, scalable, distributed revision control system with an
unusually rich command set that provides both high-level operations
and full access to internals.

Git is an Open Source project covered by the GNU General Public
License version 2 (some parts of it are under different licenses,
compatible with the GPLv2). It was originally written by Linus
Torvalds with help of a group of hackers around the net.

Please read the file INSTALL for installation instructions.

See Documentation/gittutorial.txt to get started, then see
Documentation/everyday.txt for a useful minimum set of commands, and
Documentation/git-commandname.txt for documentation of each command.
If git has been correctly installed, then the tutorial can also be
read with "man gittutorial" or "git help tutorial", and the
documentation of each command with "man git-commandname" or "git help
commandname".

CVS users may also want to read Documentation/gitcvs-migration.txt
("man gitcvs-migration" or "git help cvs-migration" if git is
installed).

Many Git online resources are accessible from http://git-scm.com/
including full documentation and Git related tools.

The user discussion and development of Git take place on the Git
mailing list -- everyone is welcome to post bug reports, feature
requests, comments and patches to git@vger.kernel.org (read
Documentation/SubmittingPatches for instructions on patch submission).
To subscribe to the list, send an email with just "subscribe git" in
the body to majordomo@vger.kernel.org. The mailing list archives are
available at http://news.gmane.org/gmane.comp.version-control.git/,
http://marc.info/?l=git and other archival sites.

The maintainer frequently sends the "What's cooking" reports that
list the current status of various development topics to the mailing
list.  The discussion following them give a good reference for
project status, development direction and remaining tasks.