mirror of
https://github.com/git/git.git
synced 2024-11-18 06:54:55 +01:00
76759c7dff
Mac OS X mangles file names containing unicode on file systems HFS+, VFAT or SAMBA. When a file using unicode code points outside ASCII is created on a HFS+ drive, the file name is converted into decomposed unicode and written to disk. No conversion is done if the file name is already decomposed unicode. Calling open("\xc3\x84", ...) with a precomposed "Ä" yields the same result as open("\x41\xcc\x88",...) with a decomposed "Ä". As a consequence, readdir() returns the file names in decomposed unicode, even if the user expects precomposed unicode. Unlike on HFS+, Mac OS X stores files on a VFAT drive (e.g. an USB drive) in precomposed unicode, but readdir() still returns file names in decomposed unicode. When a git repository is stored on a network share using SAMBA, file names are send over the wire and written to disk on the remote system in precomposed unicode, but Mac OS X readdir() returns decomposed unicode to be compatible with its behaviour on HFS+ and VFAT. The unicode decomposition causes many problems: - The names "git add" and other commands get from the end user may often be precomposed form (the decomposed form is not easily input from the keyboard), but when the commands read from the filesystem to see what it is going to update the index with already is on the filesystem, readdir() will give decomposed form, which is different. - Similarly "git log", "git mv" and all other commands that need to compare pathnames found on the command line (often but not always precomposed form; a command line input resulting from globbing may be in decomposed) with pathnames found in the tree objects (should be precomposed form to be compatible with other systems and for consistency in general). - The same for names stored in the index, which should be precomposed, that may need to be compared with the names read from readdir(). NFS mounted from Linux is fully transparent and does not suffer from the above. As Mac OS X treats precomposed and decomposed file names as equal, we can - wrap readdir() on Mac OS X to return the precomposed form, and - normalize decomposed form given from the command line also to the precomposed form, to ensure that all pathnames used in Git are always in the precomposed form. This behaviour can be requested by setting "core.precomposedunicode" configuration variable to true. The code in compat/precomposed_utf8.c implements basically 4 new functions: precomposed_utf8_opendir(), precomposed_utf8_readdir(), precomposed_utf8_closedir() and precompose_argv(). The first three are to wrap opendir(3), readdir(3), and closedir(3) functions. The argv[] conversion allows to use the TAB filename completion done by the shell on command line. It tolerates other tools which use readdir() to feed decomposed file names into git. When creating a new git repository with "git init" or "git clone", "core.precomposedunicode" will be set "false". The user needs to activate this feature manually. She typically sets core.precomposedunicode to "true" on HFS and VFAT, or file systems mounted via SAMBA. Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
164 lines
4.7 KiB
Bash
Executable file
164 lines
4.7 KiB
Bash
Executable file
#!/bin/sh
|
|
#
|
|
# Copyright (c) 2012 Torsten Bögershausen
|
|
#
|
|
|
|
test_description='utf-8 decomposed (nfd) converted to precomposed (nfc)'
|
|
|
|
. ./test-lib.sh
|
|
|
|
Adiarnfc=`printf '\303\204'`
|
|
Adiarnfd=`printf 'A\314\210'`
|
|
|
|
# check if the feature is compiled in
|
|
mkdir junk &&
|
|
>junk/"$Adiarnfc" &&
|
|
case "$(cd junk && echo *)" in
|
|
"$Adiarnfd")
|
|
test_nfd=1
|
|
;;
|
|
*) ;;
|
|
esac
|
|
rm -rf junk
|
|
|
|
|
|
if test "$test_nfd"
|
|
then
|
|
# create more utf-8 variables
|
|
Odiarnfc=`printf '\303\226'`
|
|
Odiarnfd=`printf 'O\314\210'`
|
|
AEligatu=`printf '\303\206'`
|
|
Invalidu=`printf '\303\377'`
|
|
|
|
|
|
#Create a string with 255 bytes (decomposed)
|
|
Alongd=$Adiarnfd$Adiarnfd$Adiarnfd$Adiarnfd$Adiarnfd$Adiarnfd$Adiarnfd #21 Byte
|
|
Alongd=$Alongd$Alongd$Alongd #63 Byte
|
|
Alongd=$Alongd$Alongd$Alongd$Alongd$Adiarnfd #255 Byte
|
|
|
|
#Create a string with 254 bytes (precomposed)
|
|
Alongc=$AEligatu$AEligatu$AEligatu$AEligatu$AEligatu #10 Byte
|
|
Alongc=$Alongc$Alongc$Alongc$Alongc$Alongc #50 Byte
|
|
Alongc=$Alongc$Alongc$Alongc$Alongc$Alongc #250 Byte
|
|
Alongc=$Alongc$AEligatu$AEligatu #254 Byte
|
|
|
|
test_expect_success "detect if nfd needed" '
|
|
precomposeunicode=`git config core.precomposeunicode` &&
|
|
test "$precomposeunicode" = false &&
|
|
git config core.precomposeunicode true
|
|
'
|
|
test_expect_success "setup" '
|
|
>x &&
|
|
git add x &&
|
|
git commit -m "1st commit" &&
|
|
git rm x &&
|
|
git commit -m "rm x"
|
|
'
|
|
test_expect_success "setup case mac" '
|
|
git checkout -b mac_os
|
|
'
|
|
# This will test nfd2nfc in readdir()
|
|
test_expect_success "add file Adiarnfc" '
|
|
echo f.Adiarnfc >f.$Adiarnfc &&
|
|
git add f.$Adiarnfc &&
|
|
git commit -m "add f.$Adiarnfc"
|
|
'
|
|
# This will test nfd2nfc in git stage()
|
|
test_expect_success "stage file d.Adiarnfd/f.Adiarnfd" '
|
|
mkdir d.$Adiarnfd &&
|
|
echo d.$Adiarnfd/f.$Adiarnfd >d.$Adiarnfd/f.$Adiarnfd &&
|
|
git stage d.$Adiarnfd/f.$Adiarnfd &&
|
|
git commit -m "add d.$Adiarnfd/f.$Adiarnfd"
|
|
'
|
|
test_expect_success "add link Adiarnfc" '
|
|
ln -s d.$Adiarnfd/f.$Adiarnfd l.$Adiarnfc &&
|
|
git add l.$Adiarnfc &&
|
|
git commit -m "add l.Adiarnfc"
|
|
'
|
|
# This will test git log
|
|
test_expect_success "git log f.Adiar" '
|
|
git log f.$Adiarnfc > f.Adiarnfc.log &&
|
|
git log f.$Adiarnfd > f.Adiarnfd.log &&
|
|
test -s f.Adiarnfc.log &&
|
|
test -s f.Adiarnfd.log &&
|
|
test_cmp f.Adiarnfc.log f.Adiarnfd.log &&
|
|
rm f.Adiarnfc.log f.Adiarnfd.log
|
|
'
|
|
# This will test git ls-files
|
|
test_expect_success "git lsfiles f.Adiar" '
|
|
git ls-files f.$Adiarnfc > f.Adiarnfc.log &&
|
|
git ls-files f.$Adiarnfd > f.Adiarnfd.log &&
|
|
test -s f.Adiarnfc.log &&
|
|
test -s f.Adiarnfd.log &&
|
|
test_cmp f.Adiarnfc.log f.Adiarnfd.log &&
|
|
rm f.Adiarnfc.log f.Adiarnfd.log
|
|
'
|
|
# This will test git mv
|
|
test_expect_success "git mv" '
|
|
git mv f.$Adiarnfd f.$Odiarnfc &&
|
|
git mv d.$Adiarnfd d.$Odiarnfc &&
|
|
git mv l.$Adiarnfd l.$Odiarnfc &&
|
|
git commit -m "mv Adiarnfd Odiarnfc"
|
|
'
|
|
# Files can be checked out as nfc
|
|
# And the link has been corrected from nfd to nfc
|
|
test_expect_success "git checkout nfc" '
|
|
rm f.$Odiarnfc &&
|
|
git checkout f.$Odiarnfc
|
|
'
|
|
# Make it possible to checkout files with their NFD names
|
|
test_expect_success "git checkout file nfd" '
|
|
rm -f f.* &&
|
|
git checkout f.$Odiarnfd
|
|
'
|
|
# Make it possible to checkout links with their NFD names
|
|
test_expect_success "git checkout link nfd" '
|
|
rm l.* &&
|
|
git checkout l.$Odiarnfd
|
|
'
|
|
test_expect_success "setup case mac2" '
|
|
git checkout master &&
|
|
git reset --hard &&
|
|
git checkout -b mac_os_2
|
|
'
|
|
# This will test nfd2nfc in git commit
|
|
test_expect_success "commit file d2.Adiarnfd/f.Adiarnfd" '
|
|
mkdir d2.$Adiarnfd &&
|
|
echo d2.$Adiarnfd/f.$Adiarnfd >d2.$Adiarnfd/f.$Adiarnfd &&
|
|
git add d2.$Adiarnfd/f.$Adiarnfd &&
|
|
git commit -m "add d2.$Adiarnfd/f.$Adiarnfd" -- d2.$Adiarnfd/f.$Adiarnfd
|
|
'
|
|
test_expect_success "setup for long decomposed filename" '
|
|
git checkout master &&
|
|
git reset --hard &&
|
|
git checkout -b mac_os_long_nfd_fn
|
|
'
|
|
test_expect_success "Add long decomposed filename" '
|
|
echo longd >$Alongd &&
|
|
git add * &&
|
|
git commit -m "Long filename"
|
|
'
|
|
test_expect_success "setup for long precomposed filename" '
|
|
git checkout master &&
|
|
git reset --hard &&
|
|
git checkout -b mac_os_long_nfc_fn
|
|
'
|
|
test_expect_success "Add long precomposed filename" '
|
|
echo longc >$Alongc &&
|
|
git add * &&
|
|
git commit -m "Long filename"
|
|
'
|
|
# Test if the global core.precomposeunicode stops autosensing
|
|
# Must be the last test case
|
|
test_expect_success "respect git config --global core.precomposeunicode" '
|
|
git config --global core.precomposeunicode true &&
|
|
rm -rf .git &&
|
|
git init &&
|
|
precomposeunicode=`git config core.precomposeunicode` &&
|
|
test "$precomposeunicode" = "true"
|
|
'
|
|
else
|
|
say "Skipping nfc/nfd tests"
|
|
fi
|
|
|
|
test_done
|