This updates the user interface and generated diff data format.
* "diff --binary" is used to signal that we want an e-mailable
binary patch. It implies --full-index and -p.
* "apply --allow-binary-replacement" acquired a short synonym
"apply --binary".
* After the "GIT binary patch\n" header line there is a token
to record which binary patch mechanism was used, so that we
can extend it later. Currently there are two mechanisms
defined: "literal" and "delta". The former records the
deflated postimage and the latter records the deflated delta
from the preimage to postimage.
For purely implementation convenience, I added the deflated
length after these "literal/delta" tokens (otherwise the
decoding side needs to guess and reallocate the buffer while
inflating). Improvement patches are very welcomed.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This adds "binary patch" to the diff output and teaches apply
what to do with them.
On the diff generation side, traditionally, we said "Binary
files differ\n" without giving anything other than the preimage
and postimage object name on the index line. This was good
enough for applying a patch generated from your own repository
(very useful while rebasing), because the postimage would be
available in such a case. However, this was not useful when the
recipient of such a patch via e-mail were to apply it, even if
the preimage was available.
This patch allows the diff to generate "binary" patch when
operating under --full-index option. The binary patch follows
the usual extended git diff headers, and looks like this:
"GIT binary patch\n"
<length byte><data>"\n"
...
"\n"
Each line is prefixed with a "length-byte", whose value is upper
or lowercase alphabet that encodes number of bytes that the data
on the line decodes to (1..52 -- 'A' means 1, 'B' means 2, ...,
'Z' means 26, 'a' means 27, ...). <data> is 1 or more groups of
5-byte sequence, each of which encodes up to 4 bytes in base85
encoding. Because 52 / 4 * 5 = 65 and we have the length byte,
an output line is capped to 66 characters. The payload is the
same diff-delta as we use in the packfiles.
On the consumption side, git-apply now can decode and apply the
binary patch when --allow-binary-replacement is given, the diff
was generated with --full-index, and the receiving repository
has the preimage blob, which is the same condition as it always
required when accepting an "Binary files differ\n" patch.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Ok this really should be the good version. The option
handling has been reworked to be automation safe.
Currently to import the -mm tree I have to work around
git-apply by using patch. Because some of Andrews
patches in quilt will only apply with fuzz.
I started out implementing a --fuzz option and then I realized
fuzz is not a very safe concept for an automated system. What
you really want is a minimum number of context lines that must
match. This allows policy to be set without knowing how many
lines of context a patch actually provides. By default
the policy remains to match all provided lines of context.
Allowng git-apply to match a restricted set of context makes
it much easier to import the -mm tree into git. I am still only
processing 1.5 to 1.6 patches a second for the 692 patches in
2.6.17-rc1-mm2 is still painful but it does help.
If I just loop through all of Andrews patches in order
and run git-apply --index -C1 I process the entire patchset
in 1m53s or about 6 patches per second. So running
git-mailinfo, git-write-tree, git-commit-tree, and
git-update-ref everytime has a measurable impact,
and shows things can be speeded up even more.
All of these timings were taking on my poor 700Mhz Athlon
with 512MB of ram. So people with fast machiens should
see much better performance.
When a match is found after the number of context are reduced a
warning is generated. Since this is a rare event and possibly
dangerous this seems to make sense. Unless you are patching
a single file the error message is a little bit terse at
the moment, but it should be easy to go back and fix.
I have also updated the documentation for git-apply to reflect
the new -C option that sets the minimum number of context
lines that must match.
Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This replaces occurences of "blob", "commit", "tag", and "tree",
where they're really used as type specifiers, which we already
have defined global constants for.
Signed-off-by: Peter Eriksen <s022018@student.dtu.dk>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This was triggered by me testing the "@@" numbering shorthand by GNU
patch, which not only showed that git-apply thought it meant the number
was duplicated (when it means that the second number is 1), but my tests
showed than when git-apply mis-understood the number, it would then not
raise an alarm about it if the patch ended early.
Now, this doesn't actually _matter_, since with a three-line context, the
only case that "x,1" will be shorthanded to "x" is when x itself is 1 (in
which case git-apply got it right), but the fact that git-apply would also
silently accept truncated patches was a missed opportunity for additional
sanity-checking.
So make git-apply refuse to look at a patch fragment that ends early.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Sometimes it is convient for a Porcelain to be able to checkout all
unmerged files in all stages so that an external merge tool can be
executed by the Porcelain or the end-user. Using git-unpack-file
on each stage individually incurs a rather high penalty due to the
need to fork for each file version obtained. git-checkout-index -a
--stage=all will now do the same thing, but faster.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
We were missing the --whitespace option in the usage string for
git-apply and git-am, so this commit adds them.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This changes the default --whitespace policy to nowarn when we
are only getting --stat, --summary etc. IOW when not applying
the patch. When applying the patch, the default is warn (spit
out warning message but apply the patch).
Signed-off-by: Junio C Hamano <junkio@cox.net>
Andrew insists --whitespace=warn should be the default, and I
tend to agree. This introduces --whitespace=warn, so if your
project policy is more lenient, you can squelch them by having
apply.whitespace=nowarn in your configuration file.
Signed-off-by: Junio C Hamano <junkio@cox.net>
The new configuration option apply.whitespace can take one of
"warn", "error", "error-all", or "strip". When git-apply is run
to apply the patch to the index, they are used as the default
value if there is no command line --whitespace option.
Andrew can now tell people who feed him git trees to update to
this version and say:
git repo-config apply.whitespace error
Signed-off-by: Junio C Hamano <junkio@cox.net>
This by default makes --whitespace=warn, error, and strip to
warn only the first 5 additions of trailing whitespaces. A new
option --whitespace=error-all can be used to view all of them
before applying.
Signed-off-by: Junio C Hamano <junkio@cox.net>
In addition to fixing obvious command line parsing bugs in the
previous round, this changes the following:
* Adds "--whitespace=strip". This applies after stripping the
new trailing whitespaces introduced to the patch.
* The output error message format is changed to say
"patch-filename:linenumber:contents of the line". This makes
it similar to typical compiler error message format, and
helps C-x ` (next-error) in Emacs compilation buffer.
* --whitespace=error and --whitespace=warn do not stop at the
first error. We might want to limit the output to say first
20 such lines to prevent cluttering, but on the other hand if
you are willing to hand-fix after inspecting them, getting
everything with a single run might be easier to work with.
After all, somebody has to do the clean-up work somewhere.
Signed-off-by: Junio C Hamano <junkio@cox.net>
On Sat, 25 Feb 2006, Andrew Morton wrote:
>
> I'd suggest a) git will simply refuse to apply such a patch unless given a
> special `forcing' flag, b) even when thus forced, it will still warn and c)
> with a different flag, it will strip-then-apply, without generating a
> warning.
This doesn't do the "strip-then-apply" thing, but it allows you to make
git-apply generate a warning or error on extraneous whitespace.
Use --whitespace=warn to warn, and (surprise, surprise) --whitespace=error
to make it a fatal error to have whitespace at the end.
Totally untested, of course. But it compiles, so it must be fine.
HOWEVER! Note that this literally will check every single patch-line with
"+" at the beginning. Which means that if you fix a simple typo, and the
line had a space at the end before, and you didn't remove it, that's still
considered a "new line with whitespace at the end", even though obviously
the line wasn't really new.
I assume this is what you wanted, and there isn't really any sane
alternatives (you could make the warning activate only for _pure_
additions with no deletions at all in that hunk, but that sounds a bit
insane).
Linus
* jc/nostat:
cache_name_compare() compares name and stage, nothing else.
"assume unchanged" git: documentation.
ls-files: split "show-valid-bit" into a different option.
"Assume unchanged" git: --really-refresh fix.
ls-files: debugging aid for CE_VALID changes.
"Assume unchanged" git: do not set CE_VALID with --refresh
"Assume unchanged" git
Some versions of diff do not correctly detect a missing new-line at the end
of the file under certain circumstances.
When defining NO_ACCURATE_DIFF, work around this bug.
Signed-off-by: Johannes E. Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This adds "assume unchanged" logic, started by this message in the list
discussion recently:
<Pine.LNX.4.64.0601311807470.7301@g5.osdl.org>
This is a workaround for filesystems that do not have lstat()
that is quick enough for the index mechanism to take advantage
of. On the paths marked as "assumed to be unchanged", the user
needs to explicitly use update-index to register the object name
to be in the next commit.
You can use two new options to update-index to set and reset the
CE_VALID bit:
git-update-index --assume-unchanged path...
git-update-index --no-assume-unchanged path...
These forms manipulate only the CE_VALID bit; it does not change
the object name recorded in the index file. Nor they add a new
entry to the index.
When the configuration variable "core.ignorestat = true" is set,
the index entries are marked with CE_VALID bit automatically
after:
- update-index to explicitly register the current object name to the
index file.
- when update-index --refresh finds the path to be up-to-date.
- when tools like read-tree -u and apply --index update the working
tree file and register the current object name to the index file.
The flag is dropped upon read-tree that does not check out the index
entry. This happens regardless of the core.ignorestat settings.
Index entries marked with CE_VALID bit are assumed to be
unchanged most of the time. However, there are cases that
CE_VALID bit is ignored for the sake of safety and usability:
- while "git-read-tree -m" or git-apply need to make sure
that the paths involved in the merge do not have local
modifications. This sacrifices performance for safety.
- when git-checkout-index -f -q -u -a tries to see if it needs
to checkout the paths. Otherwise you can never check
anything out ;-).
- when git-update-index --really-refresh (a new flag) tries to
see if the index entry is up to date. You can start with
everything marked as CE_VALID and run this once to drop
CE_VALID bit for paths that are modified.
Most notably, "update-index --refresh" honours CE_VALID and does
not actively stat, so after you modified a file in the working
tree, update-index --refresh would not notice until you tell the
index about it with "git-update-index path" or "git-update-index
--no-assume-unchanged path".
This version is not expected to be perfect. I think diff
between index and/or tree and working files may need some
adjustment, and there probably needs other cases we should
automatically unmark paths that are marked to be CE_VALID.
But the basics seem to work, and ready to be tested by people
who asked for this feature.
Signed-off-by: Junio C Hamano <junkio@cox.net>
As far as I can see, create_subdirectories() in apply.c just
duplicates the functionality of safe_create_leading_directories() from
sha1_file.c. The former has a warm, fuzzy const parameter, but that's
not important.
The potential problem with EEXIST and creating directories should
never occur here, but will be removed by future
safe_create_leading_directories() changes. Other uses of EEXIST in
apply.c should be fine barring intentionally malicious behavior.
Signed-off-by: Jason Riedy <ejr@cs.berkeley.edu>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This only applies to traditional diffs, not to git diffs.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
It can happen if the temporary file already exists (i.e. after a panic
and reboot).
Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
We had errno==EINTR check after read(2)/write(2) sprinkled all
over the places, always doing continue. Consolidate them into
xread()/xwrite() wrapper routines.
Credits for suggestion goes to HPA -- bugs are mine.
Signed-off-by: Junio C Hamano <junkio@cox.net>
When applying a patch to index file, we need to know where GIT_DIR is;
use setup_git_directory() to find it out. This also allows us to work
from a subdirectory if we wanted to.
When git-apply is run from a subdirectory, it applies the given patch
only to the files under the current directory and below.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Some vintage of diff says just "Files X and Y differ\n", instead
of "Binary files X and Y differ\n", so catch both patterns.
Signed-off-by: Junio C Hamano <junkio@cox.net>
A new option, --allow-binary-replacement, is introduced.
When you feed a diff that records full SHA1 name of pre- and
post-image blob on its index line to git-apply with this option,
the post-image blob replaces the path if what you have in the
working tree matches the pre-image _and_ post-image blob is
already available in the object directory.
Later we _might_ want to enhance the diff output to also include
the full binary data of the post-image, to make this more
useful, but this is good enough for local rebasing application.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Recently we fixed 'git-apply --stat' not to barf on a binary
differences. But it accidentally broke the error detection when
we actually attempt to apply them.
This commit fixes the problem and adds test cases.
Signed-off-by: Junio C Hamano <junkio@cox.net>
The comparison to find "Binary files " string was looking at a
wrong place when offset != 0.
Also, we may have the full 40-byte textual sha1 on the index
line; two off-by-one errors prevented it.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This is a specialized hack to help no-base merges, but other
people might find it useful, so let's document it.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Unlike the previous round that merged the path added differently
in each branches using emptiness as the base, compute a common
version and use it as input to 'merge' program.
This would show the resulting (still conflicting) file left in
the working tree as:
common file contents...
<<<<<< FILENAME
version from our branch...
======
version from their branch...
>>>>>> .merge_file_XXXXXX
more common file contents...
when both sides added similar contents.
Signed-off-by: Junio C Hamano <junkio@cox.net>
We run git-apply with --stat and --summary at the end of the pull
by default, which causes it to barf when the pull brought in changes
to binary files. Just mark them as binary patch and proceed when
not applying nor checking.
[jc: I almost missed --check until I saw Linus did something similar.]
Signed-off-by: Junio C Hamano <junkio@cox.net>
The new option, --numstat, shows number of inserted and deleted
lines for each path. It is similar to --stat output but is
meant to be more machine friendly by giving number of added and
deleted lines and unabbreviated paths.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Linus says he does not use it (and the thinking behind its initial
introduction), and neither Cogito nor StGIT uses it.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Do our own ctype.h, just to get the sane semantics: we want
locale-independence, _and_ we want the right signed behaviour. Plus we
only use a very small subset of ctype.h anyway (isspace, isalpha,
isdigit and isalnum).
Signed-off-by: Junio C Hamano <junkio@cox.net>
Add an new option --show-index-info to git-apply command to
summarize the index information new git-diff outputs. The
command shows something similar to git-ls-files --stage output
for the pre-change image:
100644 7be5041... apply.c
100644 ec2a161... cache.h
...
Signed-off-by: Junio C Hamano <junkio@cox.net>
The original plan was to do 3-way merge between local working tree,
index and the patch being applied, but that was never implemented.
Retire the flag to control its behaviour.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This patch teaches 'git-apply --index' to automatically check
out a file being patched. This happens only when the working
tree does not have it checked out.
Signed-off-by: Junio C Hamano <junkio@cox.net>
A patch that contains no actual diff, and that doesn't change any
meta-data is bad. It shouldn't be a patch at all, and git-apply shouldn't
just accept it.
This caused a corrupted patch to be silently applied as an empty change in
the kernel, because the corruption ended up making the patch look empty.
An example of such a patch is one that contains the patch header, but
where the initial fragment header (the "@@ -nr,.." line) is missing,
causing us to not parse any fragments.
The real "patch" program will also flag such patches as bad, with the
message
patch: **** Only garbage was found in the patch input.
and we should do likewise.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The message "\ No newline at end of file" used by diff(1) to mark
an incomplete line is locale dependent. We can't assume more than
that it begins with "\ ".
For example, given two files, "foo" and "bar", with appropriate
contents, 'diff -u foo bar' will produce the following output on
my system:
--- foo 2005-09-04 18:59:38.000000000 +0200
+++ bar 2005-09-04 18:59:16.000000000 +0200
@@ -1 +1 @@
-foobar
+foo
\ Ingen nyrad vid filslut
[jc: the check for the marker still uses the line length being no less
than 12 bytes for a sanity check, but I think it is safe to assume
that in other locales. I haven't checked the .po files from diff, tho'.]
Signed-off-by: Fredrik Kuivinen <freku045@student.liu.se>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Stop processing and return NULL if we encounter a '\n' character
before we have two matching names in the git header.
Signed-off-by: Robert Fitzsimons <robfitz@273k.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Omitting the first branch in ?: is a GNU extension. Cute,
but not supported by other compilers. Replaced mostly
by explicit tests. Calls to getenv() simply are repeated
on non-GNU compilers.
Signed-off-by: Jason Riedy <ejr@cs.berkeley.edu>
Without this patch, git-apply does not retain the mode when renaming or
copying files.
[jc: Good catch, Johannes. I added a test case to demonstrate the
breackage in the original.]
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
GCC's format __attribute__ is good for checking errors, especially
with -Wformat=2 parameter. This fixes most of the reported problems
against 2005-08-09 snapshot.
When git-apply was printing out long filenames, it used to just truncate
them to show the last "max_len" characters of the filename. Which can be
really quite ugly (note the two filenames that have just been silently
truncated from the beginning - it looks even worse when there are lots
of them, like there were in the current v2.6.13-rc4 cris arch update):
Documentation/video4linux/README.saa7134 | 9
Documentation/video4linux/bttv/Cards | 74
umentation/video4linux/hauppauge-wintv-cx88-ir.txt | 54
Documentation/video4linux/lifeview.txt | 42
mentation/video4linux/not-in-cx2388x-datasheet.txt | 41
Documentation/w1/w1.generic | 107
With this patch it now looks like so:
Documentation/video4linux/README.saa7134 | 9
Documentation/video4linux/bttv/Cards | 74
.../video4linux/hauppauge-wintv-cx88-ir.txt | 54
Documentation/video4linux/lifeview.txt | 42
.../video4linux/not-in-cx2388x-datasheet.txt | 41
Documentation/w1/w1.generic | 107
ie we've made it clear with an ellipsis that we've cut off something from
the beginning, and it also tries to do it cleanly at a subdirectory level.
Signed-off-by: Linus "good taste" Torvalds <torvalds@osdl.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Adds --exclude=pattern option to the "git-apply" command. This
was useful while reimporting the BKCVS patchset dump of the
Linux kernel, starting at 2.4.0 and ending at 2.6.12-rc2 Ingo
announced some time ago to exclude BitKeeper directory.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The parsing code had a bug that failed to recognize an
incomplete line at the end of a fragment, and the fragment
application code had a comparison bug to recognize such. Fix
them to handle incomplete lines correctly.
Add a test script for patches with various combinations of
complete and incomplete lines to make sure the fix works.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
We write them under another name and rename them to their destination,
so that if something bad happens in the middle, we won't have caused any
bigger harm.
Also, this makes the writing be NFS "intr" safe, and as a side effects
makes sure that if the target is hardlinked (or symlinked) we will have
broken the link.
apply.c: In function `show_rename_copy':
apply.c:1147: warning: field precision is not type int (arg 3)
Signed-off-by: Tony Luck <tony.luck@intel.com>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
IIRC our strategy was to let the users' umask take care of the
final mode bits. This patch fixes places that deviate from it.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Typical expected usage is "git-apply --stat --summary" to show
diffstat plus dense description of information available in git
extended headers, such as creations, renames, and mode changes.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
When a patch is a git extended rename/copy patch, "git-apply
--stat" showed the old filename. Change it to show the new
filename, because most of the time we are interested in looking
at the resulting tree.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Applying Andrew's latest patch-bomb showed us failing miserably if a new
subdirectory needed to be created.. That said, it's uncommon enough
that it's worth optimistically assuming it won't be needed, and then
creating the subdirectories only on failure.
Diffs with only mode changes didn't pass through git-apply --stat.
[ Linus' note: they did for me, on my ppc64, where division by zero just
silently returns zero. Duh. ]
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
A meaningful (ie non-empty) git patch always has more information in the
header than just the "diff --git" line itself: it needs to have either a
patch associated with it (which implies "---" and "+++" lines in the
header) or it needs to have rename/copy/delete/create information in it.
Just ignore git patches which have no change information. Otherwise we'll
end up with a patch that doesn't have filenames etc filled in, and we'll
be unhappy.
Clearly even Junio felt git "rename" header lines should say "from/to"
instead of "old/new", since he wrote the documentation that way.
This way it also matches "copy".
git-apply will accept both versions, at least for a while.
It's not "rename from" and "rename to", it's "rename old" and "rename new".
Which is illogical and doesn't match the "copy from/to" case, but that's
life. Maybe Junio will fix it up one of these days.
We update the index only if the "--index" flag is given,
so you can actually use this as a strange kind of "patch"
program even for non-git usage. Not that you'd likely
want to, but it comes in handy for testing.
This _should_ more or less get everythign right, but as
usual I leave the testing to the usrs..
This applies the fragments in memory, but doesn't actually
write the results out to the files yet. But we now do all the
difficult parts, the rest is just basically writing the
results out and updating the index.
Right now it requires that the fragment offsets be exact,
and it doesn't actually apply the fragment yet, but it
does find where it goes and verify the data.
Next step: actually applying the fragment changes.
It had already tried to do that, but with the independent
rounding of the number of '+' and '-' characters, it would
sometimes do 80-char lines after all.
The way broken deletes and creates are shown in the -p
(diff-patch) output format has become consistent with how
rename/copy edits are shown. They will show "dissimilarity
index" value, immediately following the "deleted file mode" and
"new file mode" lines.
The git-apply is taught to grok such an extended header.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Make it a clear two-phase thing: first a read-only parse of
the patch itself (which is independent of any current index
information), and then the second phase actually uses the patch.
The second phase might not be a real apply, it could be just a
diffstat, for example. Which is trivial to do once the patch is
parsed.
Not important but I am a bit annoyed by gcc complaining about the
control falling out of the function without returning value.
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
There's some duplication of filenames when doing filename operations
(creates, deletes, renames and copies), and this makes us verify that
the pathnames match when they should.
.. and print out the information. This sets up all the pathname
information, and whether it's a new file, deleted file, rename,
copy or whatever.
It's slowly getting to the point where it all comes together,
and we can actually apply all the information that we've gathered.
In particular, give line numbers when detecting corrupt patches.
This makes the tool a lot more friendly (indeed, much more so
than regular "patch", I think).
This applies git patches (and old-style unified diffs)
in the index, rather than doing it in the working directory.
That allows for a lot more flexibility, and means that if a
patch fails, we aren't going to mess up the working directory.
NOTE! This is just the first cut at it, and right now it only
parses the incoming patch, it doesn't actually apply it yet.
