...making Linux just a little more fun!
[ In reference to "2-Cent Tips" in LG#164 ]
Paul Sephton [paul at inet.co.za]
@$# This was originally entitled: "about 2c ext2 fragmentation" - Kat $#@
On Fri, 2009-07-03 at 22:46 +0200, Carlos Baiget wrote:
> another 0.5c: If any file in the given directory has a space in his > name, the defragmentation script will not work. To replace all spaces > inside the file name, i suggest to do a > > rename 's/\ /_/g' * > > previously or modify IFS environment variable to not consider space as > field separator. > > Carlos
Ah, yes; or simply quote the file names as follows:
fs=`echo "$line" | cut -f 1 -d':'` fn=`echo "$line" | cut -f 2 -d':'` # copy the file up to 10 times, preserving permissions j=0; while [ -f "$fn" -a $j -lt 10 ]; do ....
IFS also makes sense. Thanks for highlighting this rather serious error. I really should have taken the trouble to test this properly first!
Paul
Paul Sephton [paul at inet.co.za]
Aaaahhhh! Top posted! Too late, she cried!
[[[ It's okay, Paul - for you, I manually corrected it. Thanks for noticing and caring. -- Kat ]]]
Thomas Adam [thomas.adam22 at gmail.com]
2009/7/3 Paul Sephton <[email protected]>:
> Ah, yes; or simply quote the file names as follows: > > fs=`echo "$line" | cut -f 1 -d':'` > fn=`echo "$line" | cut -f 2 -d':'` > # copy the file up to 10 times, preserving permissions > j=0; > while [ -f "$fn" -a $j -lt 10 ]; do > .... > > IFS also makes sense. Thanks for highlighting this rather serious error. I > really should have taken the trouble to test this properly first! > > Paul > > On Fri, 2009-07-03 at 22:46 +0200, Carlos Baiget wrote: >> another 0.5c: If any file in the given directory has a space in his >> name, the defragmentation script will not work. To replace all spaces >> inside the file name, i suggest to do a >> >> rename 's/\ /_/g' *
Err, be careful here -- rename as a command is not the same across distributions. On Debian and I assume it's derivatives, it is indeed the classic perl script. On Redhat, it's something completely different. In which case "mmv" and friends can be used.
-- Thomas Adam
Paul Sephton [paul at inet.co.za]
On Fri, 2009-07-03 at 22:24 +0100, Thomas Adam wrote:
> 2009/7/3 Paul Sephton <[email protected]>: > > On Fri, 2009-07-03 at 22:46 +0200, Carlos Baiget wrote: > >> another 0.5c: If any file in the given directory has a space in his > >> name, the defragmentation script will not work. To replace all spaces > >> inside the file name, i suggest to do a > >> > >> rename 's/\ /_/g' * > > Err, be careful here -- rename as a command is not the same across > distributions. On Debian and I assume it's derivatives, it is indeed > the classic perl script. On Redhat, it's something completely > different. In which case "mmv" and friends can be used. > > -- Thomas Adam
Good point. Anyway, the other problem with the script is that it needs root privilege to run '/sbin/filefrag'. The script is not really safe either as pointed out by Carlos. I have tested the following against a directory of files that contain spaces. On the other hand, the fact that 'filefrag' can't be run as a normal user rather limits it's use. chmod +s /sbin/filefrag is an option.
Can you spot any more loopholes?
#!/bin/sh # Retrieve a list for fragmented files, #fragments:filename FILEFRAG="/sbin/filefrag" if [ ! -f $FILEFRAG ]; then echo requires $FILEFRAG to defrag this directory exit 0 fi flist() { for i in *; do if [ -f "$i" ]; then ff=`$FILEFRAG "$i"` fn=`echo "$ff" | cut -f1 -d':'` fs=`echo "$ff" | cut -f2 -d':' | cut -f2 -d' '` if [ -f "$fn" -a $fs -gt 1 ]; then echo -e "$fs:$fn"; fi fi done } IFS=' ' # Sort the list numeric, descending flist | sort -n -r | ( # for each file while read line; do fs=`echo "$line" | cut -f 1 -d':'` fn=`echo "$line" | cut -f 2 -d':'` # copy the file up to 10 times, preserving permissions j=0; while [ -f "$fn" -a $j -lt 10 ]; do j=$[ $j + 1 ] TMP=$$.tmp.$j if ! cp -p "$fn" "$TMP"; then echo copy failed [$fn] j=10 else # test the new temp file's fragmentation, and if less than the # original, move the temp file over the original ns=`$FILEFRAG $TMP | cut -f2 -d':' | cut -f2 -d' '` if [ $ns -lt $fs ]; then mv "$TMP" "$fn" fs=$ns if [ $ns -lt 2 ]; then j=10; fi fi fi done j=0; # clean up temporary files while [ $j -lt 10 ]; do j=$[ $j + 1 ] TMP=$$.tmp.$j if [ -f "$TMP" ]; then rm "$TMP" else j=10 fi done done ) # report fragmentation for i in *; do if [ -f $i ]; then $FILEFRAG $i; fi; done
Thomas Adam [thomas.adam22 at gmail.com]
2009/7/3 Paul Sephton <[email protected]>:
> Can you spot any more loopholes?
I scanned it very quickly, you, like everyone else, needs to "USE" ""MORE"" """""QUOTES""""".
I can't stress enough just how important that is.
-- Thomas Adam
Paul Sephton [paul at inet.co.za]
On Fri, 2009-07-03 at 22:54 +0100, Thomas Adam wrote:
> 2009/7/3 Paul Sephton <[email protected]>: > > Can you spot any more loopholes? > > I scanned it very quickly, you, like everyone else, needs to "USE" > ""MORE"" """""QUOTES""""". > > I can't stress enough just how important that is. > > -- Thomas Adam
... as in the last line,
for i in *; do if [ -f $i ]; then $FILEFRAG $i; fi; done should be for i in *; do if [ -f "$i" ]; then $FILEFRAG "$i"; fi; done
oh well....
Paul Sephton [paul at inet.co.za]
On Fri, 2009-07-03 at 22:54 +0100, Thomas Adam wrote:
> 2009/7/3 Paul Sephton <[email protected]>: > > Can you spot any more loopholes? > > I scanned it very quickly, you, like everyone else, needs to "USE" > ""MORE"" """""QUOTES""""". > I can't stress enough just how important that is. > -- Thomas Adam
Ok, here's the latest. Gives a bit more feedback about what is happening in a more readable way. Also fixes a bug that left temp files lying around. I'm running out of still fragmented directories/files to test it on.
#!/bin/sh # Retrieve a list for fragmented files, #fragments:filename FILEFRAG="/sbin/filefrag" if [ ! -f $FILEFRAG ]; then echo requires $FILEFRAG to defrag this directory exit 0 fi flist() { for i in *; do if [ -f "$i" ]; then ff=`$FILEFRAG "$i"` fn=`echo "$ff" | cut -f1 -d':'` fs=`echo "$ff" | cut -f2 -d':' | cut -f2 -d' '` if [ -f "$fn" -a $fs -gt 1 ]; then echo -e "$fs:$fn"; fi fi done } # Internal Field Separator is now a newline IFS=' ' # Sort the list numeric, descending echo Reading files... flist | sort -n -r | ( # for each file while read line; do fs=`echo "$line" | cut -f 1 -d':'` fn=`echo "$line" | cut -f 2 -d':'` # copy the file up to 10 times, preserving permissions j=0; echo -e -n "$fn\t$fs" while [ -f "$fn" -a $j -lt 10 ]; do j=$[ $j + 1 ] echo -n "." TMP=$$.tmp.$j if ! cp -p "$fn" "$TMP"; then echo copy failed [$fn] j=10 else # test the new temp file's fragmentation, and if less than the # original, move the temp file over the original ns=`$FILEFRAG $TMP | cut -f2 -d':' | cut -f2 -d' '` if [ $ns -lt $fs ]; then mv "$TMP" "$fn" fs=$ns echo -n "$fs" if [ $ns -lt 2 ]; then j=10; fi fi fi done echo echo " - file now has $fs fragments" j=0; # clean up temporary files while [ $j -lt 10 ]; do j=$[ $j + 1 ] TMP=$$.tmp.$j if [ -f "$TMP" ]; then rm "$TMP" fi done done )
Thomas Adam [thomas.adam22 at gmail.com]
2009/7/3 Paul Sephton <[email protected]>:
> if [ ! -f $FILEFRAG ]; then
[ -x "$FILEFRAG" ]
Just on the off-chance the file isn't executable for some odd reason.
> echo requires $FILEFRAG to defrag this directory
echo "requires $FILEFRAG to defrag this directory"
> exit 0 > fi > > flist() { > for i in *; do > if [ -f "$i" ]; then > ff=`$FILEFRAG "$i"`
To be honest, only the most ancient of shells, and I mean ancient, won't understand:
$()
For command substitution, over backticks. Given nesting backticks is problematic, you should just use "$()".
> fn=`echo "$ff" | cut -f1 -d':'` > fs=`echo "$ff" | cut -f2 -d':' | cut -f2 -d' '` > if [ -f "$fn" -a $fs -gt 1 ]; then echo -e "$fs:$fn"; fi
Odd -- you claim "/bin/sh", yet "echo -e" isn't POSIX -- and why is it even invoked as such here, when you're not asking for backslash interpretation?
> echo -e -n "$fn\t$fs"
printf()
> j=$[ $j + 1 ]
((J++)) -- the square-brackets are an arcane syntax.
-- Thomas Adam
Paul Sephton [paul at inet.co.za]
On Fri, 2009-07-03 at 23:50 +0100, Thomas Adam wrote:
> 2009/7/3 Paul Sephton <[email protected]>: > > if [ ! -f $FILEFRAG ]; then > > ``` > [ -x "$FILEFRAG" ] > ''' > > Just on the off-chance the file isn't executable for some odd reason.
Cool. I had changed it to -x elsewhere, but the real problem is if the script is run as an ordinary user, the execution fails because of a kernel access restriction. I was thinking of checking
-x $FILEFRAG -a $USER=root or -x $FILEFRAG -a -s $FILEFRAG. Might be better.
> > echo requires $FILEFRAG to defrag this directory > > ``` > echo "requires $FILEFRAG to defrag this directory" > > > exit 0 > > fi > > > > flist() { > > for i in *; do > > if [ -f "$i" ]; then > > ff=`$FILEFRAG "$i"` > > To be honest, only the most ancient of shells, and I mean ancient, > won't understand: > > ``` > $() > '''
... you can tell I'm rather ancient, can't you? I have never used $() before, other than in makefiles...
> For command substitution, over backticks. Given nesting backticks is > problematic, you should just use "$()". > > > fn=`echo "$ff" | cut -f1 -d':'` > > fs=`echo "$ff" | cut -f2 -d':' | cut -f2 -d' '` > > if [ -f "$fn" -a $fs -gt 1 ]; then echo -e "$fs:$fn"; fi > > Odd -- you claim "/bin/sh", yet "echo -e" isn't POSIX -- and why is it > even invoked as such here, when you're not asking for backslash > interpretation? > > > echo -e -n "$fn\t$fs" > > printf()
printf() is POSIX? I didn't know. Agreed, echo -e is unnecessary in the first instance. I'll just make it #bin/bash rather, or as you suggest, try printf() instead.
> > > j=$[ $j + 1 ] > > ((J++)) -- the square-brackets are an arcane syntax.
I'm not just ancient, but a bit arcane too
> -- Thomas Adam
Thanks, Thomas. Your help is appreciated.
Paul Sephton [paul at inet.co.za]
On Fri, 2009-07-03 at 23:50 +0100, Thomas Adam wrote:
> 2009/7/3 Paul Sephton <[email protected]>: > > if [ ! -f $FILEFRAG ]; then > > ``` > [ -x "$FILEFRAG" ] > ''' > > Just on the off-chance the file isn't executable for some odd reason.
<snip> Ok, this actually reads and works a whole lot better. Thanks for the tips, Thomas:
#!/bin/sh # Retrieve a list for fragmented files, #fragments:filename FILEFRAG="/sbin/filefrag" if [ ! -x $FILEFRAG ]; then echo requires $FILEFRAG to defrag this directory exit 0 fi if [ ! "$USER" = "root" ]; then if [ ! -u $FILEFRAG ]; then echo $FILEFRAG can only be run as root, or should be SUID root exit 0 fi fi flist() { for i in *; do if [ -f "$i" ]; then ff=$( $FILEFRAG "$i" ) fn=$( echo "$ff" | cut -f1 -d':' ) fs=$( echo "$ff" | cut -f2 -d':' | cut -f2 -d' ' ) if [ -f "$fn" -a $fs -gt 1 ]; then printf "$fs:$fn\n"; fi fi done } # Internal Field Separator is now a newline IFS=' ' # Sort the list numeric, descending echo Reading files... flist | sort -n -r | ( # for each file while read line; do fs=$( echo "$line" | cut -f 1 -d':' ) fn=$( echo "$line" | cut -f 2 -d':' ) # copy the file up to 10 times, preserving permissions j=0 printf "$fn\t$fs" while [ -f "$fn" -a $j -lt 10 ]; do ((j++)) printf "." TMP=$$.tmp.$j if [ -f "$TMP" ]; then echo "Error! target temp file already exists!" elif ! cp -p "$fn" "$TMP"; then echo copy failed [$fn] j=10 else # test the new temp file's fragmentation, and if less than the # original, move the temp file over the original ns=$( $FILEFRAG "$TMP" | cut -f2 -d':' | cut -f2 -d' ' ) if [ $ns -lt $fs ]; then mv "$TMP" "$fn" fs=$ns printf "$fs" if [ $ns -lt 2 ]; then j=10; fi fi fi done echo echo " (file now has $fs fragments)" j=0 # clean up temporary files while [ $j -lt 10 ]; do ((j++)) TMP=$$.tmp.$j if [ -f "$TMP" ]; then rm "$TMP" fi done done )