Run search/replace on all files

[David O'Meara]

I need to remove leading and trailing whitespace from files in a directory and sub-directories (simple HTML reduction)

I have the following for a single file, how to run over all files and directories according to a file pattern?

sed 's/^[\t ]*//'

If it helps, I'd also like to run:

sed 's/[\t ]+/ /'
sed '/(\r\n?)+/$1/'

I have about 10 more, but that's likely to be enough.
[ July 01, 2008: Message edited by: David O'Meara ]

[Rodrigo Tomita]

Hi, David,

Using bash, you can use 'find' to get all the files recursively and do a 'for' loop on the result. I think you can try something like:

cd [your root folder]
for i in $(find . -name \*.html); do
    sed [your expression here] $i > ${i}_tmp
    mv ${i}_tmp $i
done

I usually test if I'm getting the correct the file set by doing a 'echo $i' inside the 'for' before putting other commands. And... consider having a backup first as well
[ July 01, 2008: Message edited by: Rodrigo Tomita ]

[Tim Holloway]

Hmmmm. You can get a list of candidate files with this:


for `find -type f -regex ".*/\s+.*$" -or -regex ".*\s$"`

However, if you've got directory names with leading spaces, it will give false positives, since my leading-space regex doesn't have the smarts to only look at the LAST slash in the path.

Still, it's a start. Usually I do this sort of stuff with a Perl script.

[Rodrigo Tomita]

Ahm... were you talking about removing spaces in the file names ?

In my post I thought you meant running your 'sed' command in several files editing the file content... Sorry about that.

[David O'Meara]

Yes, these are production files
Thanks Tim, I could barely understand it, so it must be right

I'll be running a local test before verifying in our TEST environment, and backing up clients before running against them.

[David O'Meara]

Word for the novice:
Since sed provides a line by line replacement, don't attempt to redirect back to the same file

Currently I am using

for i in $(find . -iname \*.jsp); do
    du -b $i
    sed 's/^[\t ]*//g' < $i > ${i}_tmp
    du -b ${i}_tmp
    rm $i
    mv  ${i}_tmp $i
done

[Carey Evans]

Some versions of sed provide a "-i" option to update files in place, optionally making backups, like Perl. For example, with GNU sed:

sed -i -e 's/foo/bar/' *.txt

will change "foo" to "bar" and update the original files. "-i~" would keep backups with .txt~ extensions.

By the way, mv will replace the destination if it exists, so you don't have to delete it first.