mirrors/git
1
0
Fork 0
mirror of https://github.com/git/git.git synced 2024-11-18 15:04:49 +01:00
Code Activity
git/vcs-svn/sliding_window.c

80 lines
2.1 KiB
C
Raw Normal View History

vcs-svn: learn to maintain a sliding view of a file Each section of a Subversion-format delta only requires examining (and keeping in random-access memory) a small portion of the preimage. At any moment, this portion starts at a certain file offset and has a well-defined length, and as the delta is applied, the portion advances from the beginning to the end of the preimage. Add a move_window function to keep track of this view into the preimage. You can use it like this: buffer_init(f, NULL); struct sliding_view window = SLIDING_VIEW_INIT(f); move_window(&window, 3, 7); /* (1) */ move_window(&window, 5, 5); /* (2) */ move_window(&window, 12, 2); /* (3) */ strbuf_release(&window.buf); buffer_deinit(f); The data structure is called sliding_view instead of _window to prevent confusion with svndiff0 Windows. In this example, (1) reads 10 bytes and discards the first 3; (2) discards the first 2, which are not needed any more; and (3) skips 2 bytes and reads 2 new bytes to work with. When move_window returns, the file position indicator is at position window->off + window->width and the data from positions window->off to the current file position are stored in window->buf. This function performs only sequential access from the input file and never seeks, so it can be safely used on pipes and sockets. On end-of-file, move_window silently reads less than the caller requested. On other errors, it prints a message and returns -1. Helped-by: David Barr <david.barr@cordelta.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2011-01-03 04:54:58 +01:00
/*
* Licensed under a two-clause BSD-style license.
* See LICENSE for details.
*/
#include "git-compat-util.h"
#include "sliding_window.h"
#include "line_buffer.h"
#include "strbuf.h"
static int input_error(struct line_buffer *file)
{
if (!buffer_ferror(file))
return error("delta preimage ends early");
return error("cannot read delta preimage: %s", strerror(errno));
}
static int skip_or_whine(struct line_buffer *file, off_t gap)
{
if (buffer_skip_bytes(file, gap) != gap)
return input_error(file);
return 0;
}
static int read_to_fill_or_whine(struct line_buffer *file,
struct strbuf *buf, size_t width)
{
buffer_read_binary(file, buf, width - buf->len);
if (buf->len != width)
return input_error(file);
return 0;
}
vcs-svn: suppress -Wtype-limits warning On 32-bit architectures with 64-bit file offsets, gcc 4.3 and earlier produce the following warning: CC vcs-svn/sliding_window.o vcs-svn/sliding_window.c: In function `check_overflow': vcs-svn/sliding_window.c:36: warning: comparison is always false \ due to limited range of data type The warning appears even when gcc is run without any warning flags (PR12963). In later versions it can be reproduced with -Wtype-limits, which is implied by -Wextra. On 64-bit architectures it really is possible for a size_t not to be representable as an off_t so the check being warned about is not actually redundant. But even false positives are distracting. Avoid the warning by making the "len" argument to check_overflow a uintmax_t; no functional change intended. Reported-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2012-02-02 11:28:02 +01:00
static int check_offset_overflow(off_t offset, uintmax_t len)
vcs-svn: learn to maintain a sliding view of a file Each section of a Subversion-format delta only requires examining (and keeping in random-access memory) a small portion of the preimage. At any moment, this portion starts at a certain file offset and has a well-defined length, and as the delta is applied, the portion advances from the beginning to the end of the preimage. Add a move_window function to keep track of this view into the preimage. You can use it like this: buffer_init(f, NULL); struct sliding_view window = SLIDING_VIEW_INIT(f); move_window(&window, 3, 7); /* (1) */ move_window(&window, 5, 5); /* (2) */ move_window(&window, 12, 2); /* (3) */ strbuf_release(&window.buf); buffer_deinit(f); The data structure is called sliding_view instead of _window to prevent confusion with svndiff0 Windows. In this example, (1) reads 10 bytes and discards the first 3; (2) discards the first 2, which are not needed any more; and (3) skips 2 bytes and reads 2 new bytes to work with. When move_window returns, the file position indicator is at position window->off + window->width and the data from positions window->off to the current file position are stored in window->buf. This function performs only sequential access from the input file and never seeks, so it can be safely used on pipes and sockets. On end-of-file, move_window silently reads less than the caller requested. On other errors, it prints a message and returns -1. Helped-by: David Barr <david.barr@cordelta.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2011-01-03 04:54:58 +01:00
{
vcs-svn: rename check_overflow and its arguments for clarity The canonical interpretation of a range a,b is as an interval [a,b), not [a,a+b), so this function taking argument names a and b feels unnatural. Use more explicit names "offset" and "len" to make the arguments' type and function clearer. While at it, rename the function to convey that we are making sure the sum of this offset and length do not overflow an off_t, not a size_t. [jn: split out from a patch from Ramsay Jones, then improved with advice from Thomas Rast, Dmitry Ivankov, and David Barr] Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Improved-by: Dmitry Ivankov <divanorama@gmail.com>
2012-02-02 10:57:23 +01:00
if (len > maximum_signed_value_of_type(off_t))
vcs-svn: learn to maintain a sliding view of a file Each section of a Subversion-format delta only requires examining (and keeping in random-access memory) a small portion of the preimage. At any moment, this portion starts at a certain file offset and has a well-defined length, and as the delta is applied, the portion advances from the beginning to the end of the preimage. Add a move_window function to keep track of this view into the preimage. You can use it like this: buffer_init(f, NULL); struct sliding_view window = SLIDING_VIEW_INIT(f); move_window(&window, 3, 7); /* (1) */ move_window(&window, 5, 5); /* (2) */ move_window(&window, 12, 2); /* (3) */ strbuf_release(&window.buf); buffer_deinit(f); The data structure is called sliding_view instead of _window to prevent confusion with svndiff0 Windows. In this example, (1) reads 10 bytes and discards the first 3; (2) discards the first 2, which are not needed any more; and (3) skips 2 bytes and reads 2 new bytes to work with. When move_window returns, the file position indicator is at position window->off + window->width and the data from positions window->off to the current file position are stored in window->buf. This function performs only sequential access from the input file and never seeks, so it can be safely used on pipes and sockets. On end-of-file, move_window silently reads less than the caller requested. On other errors, it prints a message and returns -1. Helped-by: David Barr <david.barr@cordelta.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2011-01-03 04:54:58 +01:00
return error("unrepresentable length in delta: "
vcs-svn: suppress -Wtype-limits warning On 32-bit architectures with 64-bit file offsets, gcc 4.3 and earlier produce the following warning: CC vcs-svn/sliding_window.o vcs-svn/sliding_window.c: In function `check_overflow': vcs-svn/sliding_window.c:36: warning: comparison is always false \ due to limited range of data type The warning appears even when gcc is run without any warning flags (PR12963). In later versions it can be reproduced with -Wtype-limits, which is implied by -Wextra. On 64-bit architectures it really is possible for a size_t not to be representable as an off_t so the check being warned about is not actually redundant. But even false positives are distracting. Avoid the warning by making the "len" argument to check_overflow a uintmax_t; no functional change intended. Reported-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2012-02-02 11:28:02 +01:00
"%"PRIuMAX" > OFF_MAX", len);
vcs-svn: rename check_overflow and its arguments for clarity The canonical interpretation of a range a,b is as an interval [a,b), not [a,a+b), so this function taking argument names a and b feels unnatural. Use more explicit names "offset" and "len" to make the arguments' type and function clearer. While at it, rename the function to convey that we are making sure the sum of this offset and length do not overflow an off_t, not a size_t. [jn: split out from a patch from Ramsay Jones, then improved with advice from Thomas Rast, Dmitry Ivankov, and David Barr] Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Improved-by: Dmitry Ivankov <divanorama@gmail.com>
2012-02-02 10:57:23 +01:00
if (signed_add_overflows(offset, (off_t) len))
vcs-svn: learn to maintain a sliding view of a file Each section of a Subversion-format delta only requires examining (and keeping in random-access memory) a small portion of the preimage. At any moment, this portion starts at a certain file offset and has a well-defined length, and as the delta is applied, the portion advances from the beginning to the end of the preimage. Add a move_window function to keep track of this view into the preimage. You can use it like this: buffer_init(f, NULL); struct sliding_view window = SLIDING_VIEW_INIT(f); move_window(&window, 3, 7); /* (1) */ move_window(&window, 5, 5); /* (2) */ move_window(&window, 12, 2); /* (3) */ strbuf_release(&window.buf); buffer_deinit(f); The data structure is called sliding_view instead of _window to prevent confusion with svndiff0 Windows. In this example, (1) reads 10 bytes and discards the first 3; (2) discards the first 2, which are not needed any more; and (3) skips 2 bytes and reads 2 new bytes to work with. When move_window returns, the file position indicator is at position window->off + window->width and the data from positions window->off to the current file position are stored in window->buf. This function performs only sequential access from the input file and never seeks, so it can be safely used on pipes and sockets. On end-of-file, move_window silently reads less than the caller requested. On other errors, it prints a message and returns -1. Helped-by: David Barr <david.barr@cordelta.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2011-01-03 04:54:58 +01:00
return error("unrepresentable offset in delta: "
"%"PRIuMAX" + %"PRIuMAX" > OFF_MAX",
vcs-svn: suppress -Wtype-limits warning On 32-bit architectures with 64-bit file offsets, gcc 4.3 and earlier produce the following warning: CC vcs-svn/sliding_window.o vcs-svn/sliding_window.c: In function `check_overflow': vcs-svn/sliding_window.c:36: warning: comparison is always false \ due to limited range of data type The warning appears even when gcc is run without any warning flags (PR12963). In later versions it can be reproduced with -Wtype-limits, which is implied by -Wextra. On 64-bit architectures it really is possible for a size_t not to be representable as an off_t so the check being warned about is not actually redundant. But even false positives are distracting. Avoid the warning by making the "len" argument to check_overflow a uintmax_t; no functional change intended. Reported-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2012-02-02 11:28:02 +01:00
(uintmax_t) offset, len);
vcs-svn: learn to maintain a sliding view of a file Each section of a Subversion-format delta only requires examining (and keeping in random-access memory) a small portion of the preimage. At any moment, this portion starts at a certain file offset and has a well-defined length, and as the delta is applied, the portion advances from the beginning to the end of the preimage. Add a move_window function to keep track of this view into the preimage. You can use it like this: buffer_init(f, NULL); struct sliding_view window = SLIDING_VIEW_INIT(f); move_window(&window, 3, 7); /* (1) */ move_window(&window, 5, 5); /* (2) */ move_window(&window, 12, 2); /* (3) */ strbuf_release(&window.buf); buffer_deinit(f); The data structure is called sliding_view instead of _window to prevent confusion with svndiff0 Windows. In this example, (1) reads 10 bytes and discards the first 3; (2) discards the first 2, which are not needed any more; and (3) skips 2 bytes and reads 2 new bytes to work with. When move_window returns, the file position indicator is at position window->off + window->width and the data from positions window->off to the current file position are stored in window->buf. This function performs only sequential access from the input file and never seeks, so it can be safely used on pipes and sockets. On end-of-file, move_window silently reads less than the caller requested. On other errors, it prints a message and returns -1. Helped-by: David Barr <david.barr@cordelta.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2011-01-03 04:54:58 +01:00
return 0;
}
int move_window(struct sliding_view *view, off_t off, size_t width)
{
off_t file_offset;
assert(view);
assert(view->width <= view->buf.len);
vcs-svn: rename check_overflow and its arguments for clarity The canonical interpretation of a range a,b is as an interval [a,b), not [a,a+b), so this function taking argument names a and b feels unnatural. Use more explicit names "offset" and "len" to make the arguments' type and function clearer. While at it, rename the function to convey that we are making sure the sum of this offset and length do not overflow an off_t, not a size_t. [jn: split out from a patch from Ramsay Jones, then improved with advice from Thomas Rast, Dmitry Ivankov, and David Barr] Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Improved-by: Dmitry Ivankov <divanorama@gmail.com>
2012-02-02 10:57:23 +01:00
assert(!check_offset_overflow(view->off, view->buf.len));
vcs-svn: learn to maintain a sliding view of a file Each section of a Subversion-format delta only requires examining (and keeping in random-access memory) a small portion of the preimage. At any moment, this portion starts at a certain file offset and has a well-defined length, and as the delta is applied, the portion advances from the beginning to the end of the preimage. Add a move_window function to keep track of this view into the preimage. You can use it like this: buffer_init(f, NULL); struct sliding_view window = SLIDING_VIEW_INIT(f); move_window(&window, 3, 7); /* (1) */ move_window(&window, 5, 5); /* (2) */ move_window(&window, 12, 2); /* (3) */ strbuf_release(&window.buf); buffer_deinit(f); The data structure is called sliding_view instead of _window to prevent confusion with svndiff0 Windows. In this example, (1) reads 10 bytes and discards the first 3; (2) discards the first 2, which are not needed any more; and (3) skips 2 bytes and reads 2 new bytes to work with. When move_window returns, the file position indicator is at position window->off + window->width and the data from positions window->off to the current file position are stored in window->buf. This function performs only sequential access from the input file and never seeks, so it can be safely used on pipes and sockets. On end-of-file, move_window silently reads less than the caller requested. On other errors, it prints a message and returns -1. Helped-by: David Barr <david.barr@cordelta.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2011-01-03 04:54:58 +01:00
vcs-svn: rename check_overflow and its arguments for clarity The canonical interpretation of a range a,b is as an interval [a,b), not [a,a+b), so this function taking argument names a and b feels unnatural. Use more explicit names "offset" and "len" to make the arguments' type and function clearer. While at it, rename the function to convey that we are making sure the sum of this offset and length do not overflow an off_t, not a size_t. [jn: split out from a patch from Ramsay Jones, then improved with advice from Thomas Rast, Dmitry Ivankov, and David Barr] Signed-off-by: Jonathan Nieder <jrnieder@gmail.com> Improved-by: Dmitry Ivankov <divanorama@gmail.com>
2012-02-02 10:57:23 +01:00
if (check_offset_overflow(off, width))
vcs-svn: learn to maintain a sliding view of a file Each section of a Subversion-format delta only requires examining (and keeping in random-access memory) a small portion of the preimage. At any moment, this portion starts at a certain file offset and has a well-defined length, and as the delta is applied, the portion advances from the beginning to the end of the preimage. Add a move_window function to keep track of this view into the preimage. You can use it like this: buffer_init(f, NULL); struct sliding_view window = SLIDING_VIEW_INIT(f); move_window(&window, 3, 7); /* (1) */ move_window(&window, 5, 5); /* (2) */ move_window(&window, 12, 2); /* (3) */ strbuf_release(&window.buf); buffer_deinit(f); The data structure is called sliding_view instead of _window to prevent confusion with svndiff0 Windows. In this example, (1) reads 10 bytes and discards the first 3; (2) discards the first 2, which are not needed any more; and (3) skips 2 bytes and reads 2 new bytes to work with. When move_window returns, the file position indicator is at position window->off + window->width and the data from positions window->off to the current file position are stored in window->buf. This function performs only sequential access from the input file and never seeks, so it can be safely used on pipes and sockets. On end-of-file, move_window silently reads less than the caller requested. On other errors, it prints a message and returns -1. Helped-by: David Barr <david.barr@cordelta.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2011-01-03 04:54:58 +01:00
return -1;
if (off < view->off || off + width < view->off + view->width)
return error("invalid delta: window slides left");
vcs-svn: suppress a signed/unsigned comparison warning The preceding code checks that view->max_off is nonnegative and (off + width) fits in an off_t, so this code is already safe. Signed-off-by: David Barr <davidbarr@google.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2012-05-31 16:41:29 +02:00
if (view->max_off >= 0 && view->max_off < off + (off_t) width)
vcs-svn: cap number of bytes read from sliding view Introduce a "max_off" field in struct sliding_view, roughly representing a maximum number of bytes that can be read from "file". If it is set to a nonnegative integer, a call to move_window() attempting to put the right endpoint beyond that offset will return an error instead. The idea is to use this when applying Subversion-format deltas to prevent reads past the end of the preimage (which has known length). Without such a check, corrupt deltas would cause svn-fe to block indefinitely when data in the input pipe is exhausted. Inspired-by: Ramkumar Ramachandra <artagnon@gmail.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2011-05-27 11:07:44 +02:00
return error("delta preimage ends early");
vcs-svn: learn to maintain a sliding view of a file Each section of a Subversion-format delta only requires examining (and keeping in random-access memory) a small portion of the preimage. At any moment, this portion starts at a certain file offset and has a well-defined length, and as the delta is applied, the portion advances from the beginning to the end of the preimage. Add a move_window function to keep track of this view into the preimage. You can use it like this: buffer_init(f, NULL); struct sliding_view window = SLIDING_VIEW_INIT(f); move_window(&window, 3, 7); /* (1) */ move_window(&window, 5, 5); /* (2) */ move_window(&window, 12, 2); /* (3) */ strbuf_release(&window.buf); buffer_deinit(f); The data structure is called sliding_view instead of _window to prevent confusion with svndiff0 Windows. In this example, (1) reads 10 bytes and discards the first 3; (2) discards the first 2, which are not needed any more; and (3) skips 2 bytes and reads 2 new bytes to work with. When move_window returns, the file position indicator is at position window->off + window->width and the data from positions window->off to the current file position are stored in window->buf. This function performs only sequential access from the input file and never seeks, so it can be safely used on pipes and sockets. On end-of-file, move_window silently reads less than the caller requested. On other errors, it prints a message and returns -1. Helped-by: David Barr <david.barr@cordelta.com> Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2011-01-03 04:54:58 +01:00
file_offset = view->off + view->buf.len;
if (off < file_offset) {
/* Move the overlapping region into place. */
strbuf_remove(&view->buf, 0, off - view->off);
} else {
/* Seek ahead to skip the gap. */
if (skip_or_whine(view->file, off - file_offset))
return -1;
strbuf_setlen(&view->buf, 0);
}
if (view->buf.len > width)
; /* Already read. */
else if (read_to_fill_or_whine(view->file, &view->buf, width))
return -1;
view->off = off;
view->width = width;
return 0;
}
Copy permalink