mirrors/git
1
0
Fork 0
mirror of https://github.com/git/git.git synced 2024-10-31 06:17:56 +01:00
Code Activity
git/Documentation
History
Jeff King 6bc0cb5176 http-backend: spool ref negotiation requests to buffer 
When http-backend spawns "upload-pack" to do ref
negotiation, it streams the http request body to
upload-pack, who then streams the http response back to the
client as it reads. In theory, git can go full-duplex; the
client can consume our response while it is still sending
the request.  In practice, however, HTTP is a half-duplex
protocol. Even if our client is ready to read and write
simultaneously, we may have other HTTP infrastructure in the
way, including the webserver that spawns our CGI, or any
intermediate proxies.

In at least one documented case[1], this leads to deadlock
when trying a fetch over http. What happens is basically:

  1. Apache proxies the request to the CGI, http-backend.

  2. http-backend gzip-inflates the data and sends
     the result to upload-pack.

  3. upload-pack acts on the data and generates output over
     the pipe back to Apache. Apache isn't reading because
     it's busy writing (step 1).

This works fine most of the time, because the upload-pack
output ends up in a system pipe buffer, and Apache reads
it as soon as it finishes writing. But if both the request
and the response exceed the system pipe buffer size, then we
deadlock (Apache blocks writing to http-backend,
http-backend blocks writing to upload-pack, and upload-pack
blocks writing to Apache).

We need to break the deadlock by spooling either the input
or the output. In this case, it's ideal to spool the input,
because Apache does not start reading either stdout _or_
stderr until we have consumed all of the input. So until we
do so, we cannot even get an error message out to the
client.

The solution is fairly straight-forward: we read the request
body into an in-memory buffer in http-backend, freeing up
Apache, and then feed the data ourselves to upload-pack. But
there are a few important things to note:

  1. We limit the in-memory buffer to prevent an obvious
     denial-of-service attack. This is a new hard limit on
     requests, but it's unlikely to come into play. The
     default value is 10MB, which covers even the ridiculous
     100,000-ref negotation in the included test (that
     actually caps out just over 5MB). But it's configurable
     on the off chance that you don't mind spending some
     extra memory to make even ridiculous requests work.

  2. We must take care only to buffer when we have to. For
     pushes, the incoming packfile may be of arbitrary
     size, and we should connect the input directly to
     receive-pack. There's no deadlock problem here, though,
     because we do not produce any output until the whole
     packfile has been read.

     For upload-pack's initial ref advertisement, we
     similarly do not need to buffer. Even though we may
     generate a lot of output, there is no request body at
     all (i.e., it is a GET, not a POST).

[1] http://article.gmane.org/gmane.comp.version-control.git/269020

Test-adapted-from: Dennis Kaarsemaker <dennis@kaarsemaker.net>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-05-25 20:43:18 -07:00
..
howto Documentation: typofixes 2014-11-04 13:14:44 -08:00
RelNotes Git 2.2.2 2015-01-12 14:06:12 -08:00
technical Documentation: typofixes 2014-11-04 13:14:44 -08:00
.gitattributes
.gitignore doc: generate a list of valid merge tools 2013-02-02 21:46:52 -08:00
asciidoc.conf Documentation: avoid poor-man's small caps GIT 2013-02-01 13:53:25 -08:00
blame-options.txt blame: document multiple -L support 2013-08-06 14:34:43 -07:00
build-docdep.perl
cat-texi.perl Documentation: Strip texinfo anchors to avoid duplicates 2013-04-03 16:14:19 -07:00
cmd-list.perl
CodingGuidelines Merge branch 'po/error-message-style' 2014-07-16 11:33:03 -07:00
config.txt Merge branch 'jk/colors-fix' into maint 2014-12-22 12:16:58 -08:00
date-formats.txt Correct word usage of "timezone" in "Documentation" directory 2013-11-12 10:47:17 -08:00
diff-config.txt Documentation: use "command-line" when used as a compound adjective, and fix other minor grammatical issues 2014-05-21 13:57:10 -07:00
diff-format.txt
diff-generate-patch.txt Documentation: fix misuses of "nor" 2014-03-31 15:16:22 -07:00
diff-options.txt Documentation: fix misuses of "nor" 2014-03-31 15:16:22 -07:00
docbook-xsl.css
docbook.xsl
everyday.txto doc: add 'everyday' to 'git help' 2014-10-10 16:02:26 -07:00
fetch-options.txt fetch: allow explicit --refmap to override configuration 2014-06-05 15:13:12 -07:00
fix-texi.perl
git-add.txt Merge branch 'jl/nor-or-nand-and' 2014-04-08 12:00:28 -07:00
git-am.txt Merge branch 'rw/apply-does-not-take-ignore-date' into maint 2015-01-12 14:00:16 -08:00
git-annotate.txt
git-apply.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
git-archimport.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
git-archive.txt docs: clarify remote restrictions for git-upload-archive 2014-02-28 09:55:35 -08:00
git-bisect-lk2009.txt Documentation: typofixes 2014-11-04 13:14:44 -08:00
git-bisect.txt Documentation: use "command-line" when used as a compound adjective, and fix other minor grammatical issues 2014-05-21 13:57:10 -07:00
git-blame.txt docs/git-blame: explain more clearly the example pickaxe use 2014-02-11 11:03:07 -08:00
git-branch.txt Refer to branch.<name>.remote/merge when documenting --track 2013-09-09 11:03:01 -07:00
git-bundle.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
git-cat-file.txt cat-file: provide %(deltabase) batch format 2013-12-26 11:54:26 -08:00
git-check-attr.txt Merge branch 'jc/check-x-z' 2013-09-04 12:23:25 -07:00
git-check-ignore.txt check-ignore: Add option to ignore index contents 2013-09-12 15:40:29 -07:00
git-check-mailmap.txt builtin: add git-check-mailmap command 2013-07-13 10:19:37 -07:00
git-check-ref-format.txt Add new @ shortcut for HEAD 2013-09-12 14:39:34 -07:00
git-checkout-index.txt
git-checkout.txt Documentation: @{-N} can refer to a commit 2014-01-21 13:50:00 -08:00
git-cherry-pick.txt parse-options: multi-word argh should use dash to separate words 2014-03-24 10:43:34 -07:00
git-cherry.txt Documentation: revamp git-cherry(1) 2013-11-27 12:16:49 -08:00
git-citool.txt
git-clean.txt Documentation: typofixes 2014-11-04 13:14:44 -08:00
git-clone.txt docs/git-clone: clarify use of --no-hardlinks option 2014-02-11 11:03:07 -08:00
git-column.txt doc: remote author/documentation sections from more pages 2014-01-27 08:34:34 -08:00
git-commit-tree.txt commit-tree: add and document --no-gpg-sign 2014-02-24 14:51:35 -08:00
git-commit.txt Merge branch 'jc/doc-commit-only' 2014-11-18 10:19:42 -08:00
git-config.txt Documentation: use "command-line" when used as a compound adjective, and fix other minor grammatical issues 2014-05-21 13:57:10 -07:00
git-count-objects.txt Documentation: fix misuses of "nor" 2014-03-31 15:16:22 -07:00
git-credential-cache--daemon.txt credential-cache: close stderr in daemon process 2014-09-16 11:11:58 -07:00
git-credential-cache.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
git-credential-store.txt docs/credential-store: s/--store/--file/ 2014-11-06 09:51:08 -08:00
git-credential.txt Documentation: make AsciiDoc links always point to HTML files 2013-09-06 14:49:06 -07:00
git-cvsexportcommit.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
git-cvsimport.txt Documentation: typofixes 2014-11-04 13:14:44 -08:00
git-cvsserver.txt Documentation: typofixes 2014-11-04 13:14:44 -08:00
git-daemon.txt Documentation: use "command-line" when used as a compound adjective, and fix other minor grammatical issues 2014-05-21 13:57:10 -07:00
git-describe.txt use 'commit-ish' instead of 'committish' 2013-09-04 15:03:03 -07:00
git-diff-files.txt
git-diff-index.txt Documentation/diff-index: mention two modes of operation 2013-05-20 15:50:44 -07:00
git-diff-tree.txt
git-diff.txt Documentation: fix misuses of "nor" 2014-03-31 15:16:22 -07:00
git-difftool.txt difftool: add support for --trust-exit-code 2014-10-28 10:36:57 -07:00
git-fast-export.txt Merge branch 'mh/doc-remote-helper-xref' 2014-11-19 13:47:56 -08:00
git-fast-import.txt doc: add some crossrefs between manual pages 2014-11-11 14:47:04 -08:00
git-fetch-pack.txt Merge branch 'tb/doc-fetch-pack-url' into maint 2013-12-17 11:34:24 -08:00
git-fetch.txt docs: Explain the purpose of fetch's and pull's <refspec> parameter. 2014-06-12 09:59:13 -07:00
git-filter-branch.txt Documentation: typofixes 2014-11-04 13:14:44 -08:00
git-fmt-merge-msg.txt documentation: trivial style cleanups 2013-05-17 12:09:21 -07:00
git-for-each-ref.txt doc: remote author/documentation sections from more pages 2014-01-27 08:34:34 -08:00
git-format-patch.txt format-patch: add "--signature-file=<file>" option 2014-05-27 12:38:32 -07:00
git-fsck-objects.txt
git-fsck.txt documentation: trivial style cleanups 2013-05-17 12:09:21 -07:00
git-gc.txt gc --aggressive: make --depth configurable 2014-03-31 10:26:24 -07:00
git-get-tar-commit-id.txt
git-grep.txt grep: add grep.fullName config variable 2014-03-20 12:38:00 -07:00
git-gui.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
git-hash-object.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
git-help.txt Documentation: use "command-line" when used as a compound adjective, and fix other minor grammatical issues 2014-05-21 13:57:10 -07:00
git-http-backend.txt http-backend: spool ref negotiation requests to buffer 2015-05-25 20:43:18 -07:00
git-http-fetch.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
git-http-push.txt
git-imap-send.txt Documentation: fix mismatched delimiters in git-imap-send 2014-10-13 13:35:37 -07:00
git-index-pack.txt clone: open a shortcut for connectivity check 2013-05-28 08:07:20 -07:00
git-init-db.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
git-init.txt Documentation: git-init: flesh out example 2014-08-08 13:17:42 -07:00
git-instaweb.txt
git-interpret-trailers.txt Documentation: typofixes 2014-11-04 13:14:44 -08:00
git-log.txt Merge branch 'jj/log-doc' into maint 2013-12-17 11:35:41 -08:00
git-ls-files.txt Documentation: use "command-line" when used as a compound adjective, and fix other minor grammatical issues 2014-05-21 13:57:10 -07:00
git-ls-remote.txt ls-remote doc: don't encourage use of branches-file 2013-06-23 00:33:58 -07:00
git-ls-tree.txt
git-mailinfo.txt documentation: trivial style cleanups 2013-05-17 12:09:21 -07:00
git-mailsplit.txt
git-merge-base.txt merge-base: teach "--fork-point" mode 2013-10-29 13:06:08 -07:00
git-merge-file.txt Documentation/git-merge-file: document option "--diff3" 2013-08-09 14:19:59 -07:00
git-merge-index.txt The name of the hash function is "SHA-1", not "SHA1" 2013-04-15 11:08:37 -07:00
git-merge-one-file.txt
git-merge-tree.txt use 'tree-ish' instead of 'treeish' 2013-09-04 15:02:56 -07:00
git-merge.txt merge: enable defaulttoupstream by default 2014-04-22 12:53:59 -07:00
git-mergetool--lib.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
git-mergetool.txt mergetool: document the default for --[no-]prompt 2014-04-24 11:29:05 -07:00
git-mktag.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
git-mktree.txt
git-mv.txt mv: better document side effects when moving a submodule 2014-01-07 14:33:04 -08:00
git-name-rev.txt use 'commit-ish' instead of 'committish' 2013-09-04 15:03:03 -07:00
git-notes.txt parse-options: multi-word argh should use dash to separate words 2014-03-24 10:43:34 -07:00
git-p4.txt git p4 doc: use two-line style for options with multiple spellings 2014-01-22 08:06:20 -08:00
git-pack-objects.txt upload-pack: send shallow info over stdin to pack-objects 2014-03-11 13:32:10 -07:00
git-pack-redundant.txt
git-pack-refs.txt Documentation: remove --prune from pack-refs examples 2013-07-18 16:23:46 -07:00
git-parse-remote.txt
git-patch-id.txt patch-id: make it stable against hunk reordering 2014-06-10 13:09:24 -07:00
git-prune-packed.txt Documentation: adjust document title underlining 2014-10-13 13:35:18 -07:00
git-prune.txt Documentation: fix misuses of "nor" 2014-03-31 15:16:22 -07:00
git-pull.txt Merge branch 'jc/maint-pull-docfix' into maint 2014-02-05 14:03:47 -08:00
git-push.txt Merge branch 'po/everyday-doc' into maint 2014-12-22 12:18:17 -08:00
git-quiltimport.txt Documentation: adjust document title underlining 2014-10-13 13:35:18 -07:00
git-read-tree.txt Documentation: use "command-line" when used as a compound adjective, and fix other minor grammatical issues 2014-05-21 13:57:10 -07:00
git-rebase.txt Merge branch 'so/rebase-doc-fork-point' 2014-10-14 10:49:07 -07:00
git-receive-pack.txt signed push: allow stale nonce in stateless mode 2014-09-17 15:19:54 -07:00
git-reflog.txt Merge branch 'jc/prune-all' 2013-05-29 14:23:04 -07:00
git-relink.txt
git-remote-ext.txt doc: add some crossrefs between manual pages 2014-11-11 14:47:04 -08:00
git-remote-fd.txt doc: add some crossrefs between manual pages 2014-11-11 14:47:04 -08:00
git-remote-helpers.txto Rename {git- => git}remote-helpers.txt 2013-02-01 14:12:34 -08:00
git-remote-testgit.txt Merge branch 'jk/remote-helpers-doc' 2013-02-07 14:41:45 -08:00
git-remote.txt docs/git-remote: capitalize first word of initial blurb 2014-02-11 11:03:07 -08:00
git-repack.txt Merge branch 'jk/repack-pack-keep-objects' 2014-03-18 13:50:29 -07:00
git-replace.txt Merge branch 'cc/replace-graft' 2014-07-27 15:14:18 -07:00
git-request-pull.txt request-pull: documentation updates 2014-03-13 14:22:20 -07:00
git-rerere.txt
git-reset.txt Merge branch 'jl/nor-or-nand-and' 2014-04-08 12:00:28 -07:00
git-rev-list.txt pretty: provide a strict ISO 8601 date format 2014-08-29 12:37:02 -07:00
git-rev-parse.txt Documentation: typofixes 2014-11-04 13:14:44 -08:00
git-revert.txt parse-options: multi-word argh should use dash to separate words 2014-03-24 10:43:34 -07:00
git-rm.txt rm: better document side effects when removing a submodule 2014-01-07 14:34:06 -08:00
git-send-email.txt Merge branch 'mt/send-email-cover-to-cc' 2014-06-20 13:12:20 -07:00
git-send-pack.txt send-pack: take refspecs over stdin 2014-08-26 12:58:02 -07:00
git-sh-i18n--envsubst.txt
git-sh-i18n.txt
git-sh-setup.txt Merge branch 'jc/reflog-doc' 2013-10-18 13:50:12 -07:00
git-shell.txt shell doc: remove stray "+" in example 2014-05-08 10:26:26 -07:00
git-shortlog.txt git-shortlog.txt: make SYNOPSIS match log, update OPTIONS 2013-04-21 23:11:02 -07:00
git-show-branch.txt Documentation: fix misuses of "nor" 2014-03-31 15:16:22 -07:00
git-show-index.txt The name of the hash function is "SHA-1", not "SHA1" 2013-04-15 11:08:37 -07:00
git-show-ref.txt Documentation: fix misuses of "nor" 2014-03-31 15:16:22 -07:00
git-show.txt Documentation/git-show.txt: include common diff options, like git-log.txt 2013-07-17 17:50:56 -07:00
git-stage.txt Documentation: adjust document title underlining 2014-10-13 13:35:18 -07:00
git-stash.txt stash doc: mention short form -k in save description 2014-02-24 09:13:30 -08:00
git-status.txt doc: fix 'git status --help' character quoting 2014-10-19 20:45:16 -07:00
git-stripspace.txt Merge branch 'ta/doc-no-small-caps' 2013-02-05 16:13:32 -08:00
git-submodule.txt Merge branch 'mc/doc-submodule-sync-recurse' into maint 2014-07-10 11:08:31 -07:00
git-svn.txt Documentation: typofixes 2014-11-04 13:14:44 -08:00
git-symbolic-ref.txt git symbolic-ref --delete $symref 2012-10-21 12:17:38 -07:00
git-tag.txt Merge branch 'maint-2.0' into maint 2014-10-07 13:40:51 -07:00
git-tools.txt doc: various spelling fixes 2013-04-12 12:00:52 -07:00
git-unpack-file.txt
git-unpack-objects.txt Merge branch 'vd/doc-unpack-objects' into maint 2013-11-07 14:37:36 -08:00
git-update-index.txt Documentation: typofixes 2014-11-04 13:14:44 -08:00
git-update-ref.txt update-ref --stdin -z: deprecate interpreting the empty string as zeros 2014-04-07 12:09:13 -07:00
git-update-server-info.txt
git-upload-archive.txt add uploadarchive.allowUnreachable option 2014-02-28 09:55:37 -08:00
git-upload-pack.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
git-var.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
git-verify-commit.txt verify-commit: scriptable commit signature verification 2014-06-23 15:50:31 -07:00
git-verify-pack.txt The name of the hash function is "SHA-1", not "SHA1" 2013-04-15 11:08:37 -07:00
git-verify-tag.txt The name of the hash function is "SHA-1", not "SHA1" 2013-04-15 11:08:37 -07:00
git-web--browse.txt Documentation: use "command-line" when used as a compound adjective, and fix other minor grammatical issues 2014-05-21 13:57:10 -07:00
git-whatchanged.txt whatchanged: document its historical nature 2013-08-13 09:01:54 -07:00
git-write-tree.txt
git.txt Git 2.2.2 2015-01-12 14:06:12 -08:00
gitattributes.txt Documentation: typofixes 2014-11-04 13:14:44 -08:00
gitcli.txt Documentation: use "command-line" when used as a compound adjective, and fix other minor grammatical issues 2014-05-21 13:57:10 -07:00
gitcore-tutorial.txt doc: add 'everyday' to 'git help' 2014-10-10 16:02:26 -07:00
gitcredentials.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
gitcvs-migration.txt doc: add 'everyday' to 'git help' 2014-10-10 16:02:26 -07:00
gitdiffcore.txt diffcore-pickaxe doc: document -S and -G properly 2013-06-03 10:53:11 -07:00
giteveryday.txt doc: add 'everyday' to 'git help' 2014-10-10 16:02:26 -07:00
gitglossary.txt doc: add 'everyday' to 'git help' 2014-10-10 16:02:26 -07:00
githooks.txt Documentation: typofixes 2014-11-04 13:14:44 -08:00
gitignore.txt Merge branch 'nd/gitignore-trailing-whitespace' into maint 2014-11-12 12:13:12 -08:00
gitk.txt Documentation: use "command-line" when used as a compound adjective, and fix other minor grammatical issues 2014-05-21 13:57:10 -07:00
gitmodules.txt status/commit: show staged submodules regardless of ignore config 2014-04-07 10:32:20 -07:00
gitnamespaces.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
gitremote-helpers.txt doc: add some crossrefs between manual pages 2014-11-11 14:47:04 -08:00
gitrepository-layout.txt read-cache: split-index mode 2014-06-13 11:49:39 -07:00
gitrevisions.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
gittutorial-2.txt Merge branch 'sn/tutorial-status-output-example' 2014-11-19 13:47:59 -08:00
gittutorial.txt Merge branch 'sn/tutorial-status-output-example' 2014-11-19 13:47:59 -08:00
gitweb.conf.txt Documentation: use "command-line" when used as a compound adjective, and fix other minor grammatical issues 2014-05-21 13:57:10 -07:00
gitweb.txt Documentation: fix documentation AsciiDoc links for external urls 2014-02-20 14:14:58 -08:00
gitworkflows.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
glossary-content.txt Documentation: typofixes 2014-11-04 13:14:44 -08:00
howto-index.sh howto-index.sh: use the $( ... ) construct for command substitution 2014-04-17 11:14:57 -07:00
i18n.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
install-doc-quick.sh
install-webdoc.sh install-webdoc.sh: use the $( ... ) construct for command substitution 2014-04-17 11:14:58 -07:00
line-range-format.txt line-range: teach -L^:RE to search from start of file 2013-08-06 14:48:02 -07:00
mailmap.txt Merge branch 'jk/mailmap-from-blob' 2013-01-05 23:41:42 -08:00
Makefile Merge branch 'bc/asciidoctor' 2014-10-29 10:07:40 -07:00
manpage-1.72.xsl
manpage-base-url.xsl.in
manpage-base.xsl
manpage-bold-literal.xsl
manpage-normal.xsl
manpage-quote-apos.xsl
manpage-suppress-sp.xsl
merge-config.txt Merge branch 'da/mergetool-docs' 2013-02-07 14:42:16 -08:00
merge-options.txt Documentation: fix misuses of "nor" 2014-03-31 15:16:22 -07:00
merge-strategies.txt Merge branch 'rr/doc-merge-strategies' into maint 2014-04-03 13:39:03 -07:00
pretty-formats.txt Merge branch 'bc/asciidoc-pretty-formats-fix' into maint 2014-10-29 10:35:10 -07:00
pretty-options.txt Documentation: fix misuses of "nor" 2014-03-31 15:16:22 -07:00
pull-fetch-param.txt docs: Explain the purpose of fetch's and pull's <refspec> parameter. 2014-06-12 09:59:13 -07:00
rev-list-options.txt Documentation: typofixes 2014-11-04 13:14:44 -08:00
revisions.txt Documentation: mention config sources for @{upstream} 2014-05-13 12:35:00 -07:00
sequencer.txt
SubmittingPatches SubmittingPatches: final submission is To: maintainer and CC: list 2014-11-13 10:39:24 -08:00
urls-remotes.txt Documentation: the name of the system is 'Git', not 'git' 2013-02-01 13:53:33 -08:00
urls.txt Merge branch 'ft/doc-git-transport' into maint 2013-07-21 22:51:24 -07:00
user-manual.conf docs: monospace listings in docbook output 2012-08-07 14:30:52 -07:00
user-manual.txt Merge branch 'jm/doc-wording-tweaks' 2014-06-16 12:18:39 -07:00