| Home | “This is the only version of the only program that I ever use to do a multiple file search and replace. There is no need for any other program. It does everything that you could possibly want to do!” |

Free Download & Tutorial: MFP, Version 42: Multiple File/Subfolder Parser (with Search and Replace Examples, Version 11)

By Howard Charles Best, January 28, 2012

Unicode file name feature added: December 19, 2014

(LLBest.com, )

On this web page:

A. Introduction
B. Assumptions
C. Get the Free Download
D. Main Features
E. Demo: Change the Sample Web Page File
F. Preview Mode
G. Subfolder Support
H. The Tutorial

A. Introduction

This web page is a tutorial on how to use MFP (Multiple File Parser) to do multiple-file search and replace operations of almost unlimited complexity, even if the files are located in multiple subfolders (subdirectories). This Perl script, in the form of a .bat file, never has to be changed. All of the code for how to do the search and replace operations are contained in a separate .perlcode file, which is to be dragged and dropped on top of the MFP_DragAndDropPerlCodeOnMe.bat - Shortcut icon on your computer’s desktop.

This is the only version of the only program that I ever use to do a multiple file search and replace. There is no need for any other program. It does everything that you could possibly want to do! And now that it can also handle files with names that contain Unicode characters, it is even more powerful.

Although the examples contained in this tutorial have to do with changing multiple web page files, the exact same methods may be applied to any group of text files. (By using the word, “Examples,” I am referring to the example (demo) below plus the many other examples which are “commented-out” in the included _MFP_SearchAndReplace1.perlcode file.)

B. Assumptions

This tutorial assumes the following:

1. That extensions for known file types are not hidden: a) Click Start. b) Click Computer. c) Click Organize. d) Click Folder and search options…. e) Click the View tab. f) Make sure that Hide extensions for known file types is unchecked. g) Click OK. h) Close the Computer window.

2. That you have Notepad2 or a shortcut to it on your desktop.

3. That you have installed Perl: a) Go to http://www.activestate.com/activeperl/downloads. b) Download the free version of ActivePerl for Windows. c) Install it, making sure that Add Perl to the PATH environment variable is checked (the default).

4. That _MFP_DragAndDropPerlCodeOnMe.zip (2.47 KB) has been downloaded, unzipped to a folder named _MFP_DragAndDropPerlCodeOnMe, that the _MFP_DragAndDropPerlCodeOnMe folder is located somewhere on your computer’s hard drive, and that a shortcut to the _MFP_DragAndDropPerlCodeOnMe.bat file within that folder is located on your computer’s desktop.

5. That the MFP_DragAndDropPerlCodeOnMe.bat file has a “Date modified” of 1/24/2015 or later.

C. Get the Free Download

1. Download _MFP_SearchAndReplace.zip (3.29 KB)

2. Unzip _MFP_SearchAndReplace.zip to a folder named _MFP_SearchAndReplace.

Caution: Be sure to keep unchanged versions of all of the files that you change. The backups should be in a separate folder, or, better yet, on a different drive.

D. Main Features

1. A log file, corresponding to the particular .perlcode file, is created which lists all files which were changed and how many changes were made to each.

2. Files are not rewritten unless they are actually changed. Therefore, changed files may be easily separated from unchanged files by sorting by “Date modified.”

3. Individual files are only opened once, even if, for example, multiple global search/replace operations are done in one fell swoop. (Much faster.)

4. “Preview mode” allows one to go through the motions of making changes without actually making them. Therefore, if the files changed and/or the number of changes being made is not what was expected, then the error can be corrected without “messing things up.”

5. There is no user interaction during run time to slow things down.

6. Any number of files can be changed in one fell swoop.

7. Multiple file types may be changed, in one fell swoop, without changing all file types.

8. The files may be a mixture of encodings: ANSI (ASCII), Unicode (UTF-16), and/or UTF-8 (with or without signature).

9. If the .perlcode file is encoded as UTF-8 with signature, then changed files encoded as ANSI are automatically converted to UTF-8 with signature.

10. If the .perlcode file is encoded as ANSI, then the encoding of ANSI files is not changed.

11. Changed files encoded as Unicode or UTF-8 without signature are always converted to UTF-8 with signature.

12. Files with names containing Unicode foreign language characters are handled seamlessly.

E. Demo: Change the Sample Web Page File

The way this demo works is that first the .demo file is copied to a .htm file. Then the Perl code in the _MFP_SearchAndReplace1.perlcode file is applied only to the .htm file. Thus, for the purposes of this demo only, the results will be exactly the same every time that it is run, as long as no changes are made to the .demo file or to the .perlcode files.

1. Open the _MFP_SearchAndReplace folder. It contains 5 files:

2. Double-click on the _MFP_PrepareTheDemo.bat file icon. Then you should see that a new .htm file has been created:

Please observe the contents of this .htm file:

<html><head> <title>Here's a test</title> <body> <pre>Test line 1 Test line 2 Test line 3</pre> <b>THIS WEB PAGE URL:</b> http://llbest.com/... </body> </html

3. Then drag and drop the _MFP_SearchAndReplace1.perlcode file icon on top of the MFP_DragAndDropPerlCodeOnMe.bat - Shortcut icon on your computer’s desktop. If Perl was installed correctly, then you should see:

Now there should be 7 files instead of 6:

$FileNameMask='\.htm$';$previewMode=0;$maxLevel=1;$readTheFiles=1; 1 CHANGE(S) MADE TO FILE: "C:\Users\Howard\Downloads\_MFP_SearchAndReplace\_MFP_SearchAndReplace.htm" TOTAL CHANGES MADE TO 1 "\.htm$" FILE(S): 1

The Log File (_MFP_SearchAndReplace1.log)

Please note how the contents of the .htm file has changed:

<html><head> <title>Here's a test</title> <body> <pre>Test line 1 Test line 2 Test line 3</pre> <b>THIS WEB PAGE URL:</b> http://llbest.com/_MFP_SearchAndReplace.htm </body> </html

…, and how the change was caused by the contents of the _MFP_SearchAndReplace1.perlcode file:

### $FileNameMask='\.htm$';$previewMode=0;$maxLevel=1;$readTheFiles=1; # In every web page (HTML) file which contains the string "http://llbest.com/...," replace the "..." with the actual file name of the HTML file: $changes+=($buffer=~s`(http://llbest.com/)\.\.\.`$1$fileName`); # What follows are more sample Perl search and replace statements. To try one of them, add a comment indicator ("#") to the beginning of the above, and remove it from the one that you want to try. # Some web page files may be missing the final">," but ignore the ones that already have it: #$changes+=($buffer=~s`</html`</html>`g); #$changes-=($buffer=~s`</html>>`</html>`g); # If there are any apostrophes ("'") within the title tags, change them to "’": #$buffer=~s`(<title>.*?</title>)`$t1=$1;$changes+=($t1=~s/'/’/g);$t1`sei; # If there are any tt tags within an HTML file, change them to pre tags, and, within the pairs of tt tags, change all of the <br> tags to newline characters: #$changes+=($buffer=~s`<tt(.*?)</tt>`$t1=$1;$t1=~s/<br>\n?/\n/gi;'<pre'.$t1.'</pre>'`gsei); # Change every occurrence of "TA5" to "TA4": #$changes+=($buffer=~s`\bTA5\b`TA4`g); # The following group of Regular Expression Perl statements correct common spelling mistakes and cause multiple web pages to be more consistent: #$changes+=($buffer=~s`\b([Hh])exidecimal\b`$1exadecimal`g); #$changes+=($buffer=~s`\b([Rr])echargable\b`$1echargeable`g); #$changes+=($buffer=~s`\b([Nn])eccessary\b`$1ecessary`g); #$changes+=($buffer=~s`\b([Ee])verytime\b`$1very time`g); #$changes+=($buffer=~s`\b([Uu])nlimitted\b`$1nlimited`g); #$changes+=($buffer=~s`\b([Oo])nline\b`$1n-line`g); #$changes+=($buffer=~s`\b([Rr])echargable\b`$1echargeable`g); #$changes+=($buffer=~s`\b([Dd])isply(s?)\b`$1isplay$2`g); #$changes+=($buffer=~s`\b([Rr])emoveable\b`$1emovable`g); #$changes+=($buffer=~s`\bunicode\b`Unicode`g); # Add the UTF-8 meta tag only if the web page is to be written as a UTF-8 file: #$changes+=($buffer=~s`<head>\n?<title>`<head>\n<meta http-equiv="content-type" content="text/html;charset=UTF-8">\n<title>`) if($writeChangedFileAsUTF8); # Run the following in preview mode to see if the above Regular Expression caused a duplicate UTF-8 meta tag to be added. The "$changes=0 if($changes<2);" statement causes cases of only one occurrence of the UTF-8 meta tag to be ignored: #$changes+=($buffer=~s`(<meta http-equiv="content-type" content="text/html;charset=UTF-8">)`$1`g); #$changes=0 if($changes<2); # Use to find all web pages that contain pictures. (If "$previewMode=0;" then the modification dates of these particular web pages are changed, so that they are easy to find by sing Windows to sort by "date modified."): #$changes+=($buffer=~s`(<img )`$1`g); # Change all ".TA" to "textarea.TA," except when it's already "textarea.TA": #$changes+=($buffer=~s`\.TA`textarea.TA`g); #$changes-=($buffer=~s`textareatextarea\.TA`textarea.TA`g); # See which web pages contain the string, "TS5": #$changes+=($buffer=~s`(\bTS5\b)`$1`g); # Use preview mode to see if any webpages which contain the string, "_LLBestUtilities.css" also contain the string "TA4" (The "if($previewMode)" at the end is an optional safeguard against accidental chages being made): #$changes+=($buffer=~s`(\bTA4\b)`$1`g) if($buffer=~m`_LLBestUtilities.css`) if($previewMode); # Use preview mode to see whether or not there are any web pages containing "class=change" within td tags: #$buffer=~s`(<td [^>]*)`$T1=$1;$changes+=($T1=~s/(class="?change"?\b)/$1/);"$T1"`e if($previewMode); # Change the string "fy5e34ds" to the name of the file that it's it, minus the extension: #$changes+=($buffer=~s`fy5e34ds`$T1=$fileName;$T1=~s/\.html?$//;$T1`e); # Eliminate preceding blanks only within paragraphs beginning with "\n<p>\n": #$buffer=~s`</p>`~</p>`g; #$buffer=~s`\n<p>`~~`g; #$buffer=~s`~~\n([^~]*)`$T=$1;$changes+=($T=~s/\n +/\n/g);"~~\n$T"`ge; #$buffer=~s`~</p>`</p>`g; #$buffer=~s`~~`\n<p>`g; # Make changes only within the "keywords" meta tags: #$buffer=~s`<meta name="keywords" content="([^"]*)">`$T1=$1;$changes+=($T1=~s/\bi'd like to see them make\b/I'd like to see them make/);"<meta name=\"keywords\" content=\"$T1\">"`e; # Eliminate the space between whole numbers and fractions: #$changes+=($buffer=~s`(\d) ([¼½¾])`$1$2`g); # Within "<p class=TBOX>" paragraphs, eliminate the newline characters and then put some back in under certain conditions: #$buffer=~s`class=TBOX>((?:.|\n)*?)<p `$T1=$1;$changes+=($T1=~s/\n/ /g);$changes+=($T1=~s/<br> /<br>\n/g);$changes+=($T1=~s/<br><span> /<br><span>\n/g);$changes+=($T1=~s/^<span> /<span>\n/);"class=TBOX>$T1<p "`ge; # Use preview mode to reveal which web pages contain a p{...} style declaration which contains "font-size:120%": #$buffer=~s`p\{((?:.|\n)*?)}`$T1=$1;$T2=($T1=~/font-size\:120%/);$changes++ if(!$T2);"p{$T1}"`ei if($previewMode); # Use preview mode to display, on the screen and in the log file, the path name, file name, and contents of all ".TS4{...}" style declarations: #$buffer=~s`(\.TS4{[^}]*})`$T1=$1;&printIt("\n$pathName$fileName: $T1\n");"$T1"`ei if($previewMode); # Display, on the screen and in the log file, the path name and file name of all web pages which contain the string, '<link rel="stylesheet"': #&printIt("\n$pathName$fileName\n") if(!($buffer=~/<link rel="stylesheet"/));

The above listed _MFP_SearchAndReplace1.perlcode file contains 84 lines. Except for the very first line, line #4, $changes+=($buffer=~s`(http://llbest.com/)\.\.\.`$1$fileName`);, is the only line of Regular Expression Perl code which was actually executed. The “`” characters are used to enclose the search string, (http://llbest.com/)\.\.\., and the replacement string, $1$fileName, and to separate them from each other. Back-slashes in front of the periods in the search string are necessary because periods in a Regular Expression normally have special meaning. The “$1” in the replacement string means, “Substitute the text which appears between the parentheses in the search string.” The “$fileName” in the replacement string indicates that the file name of the file containing the “\.\.\.” string should be inserted as a replacement.

4. Repeat steps 2 and 3 above, this time substituting _MFP_SearchAndReplace2.perlcode for _MFP_SearchAndReplace1.perlcode in step 3:

Now there should be 8 files instead of 7:

$FileNameMask='\.(txt|html?)$';$previewMode=0;$maxLevel=1;$readTheFiles=1; 1 CHANGE(S) MADE TO FILE: "C:\Users\Howard\Downloads\_MFP_SearchAndReplace\_MFP_SearchAndReplace.htm" TOTAL CHANGES MADE TO 1 "\.(txt|html?)$" FILE(S): 1

The 2nd Log File (_MFP_SearchAndReplace2.log)

The _MFP_SearchAndReplace2.perlcode file contains 3 lines:

# Some web page files may be missing the final">," but ignore the ones that already have it: $changes+=($buffer=~s`</html`</html>`g); $changes-=($buffer=~s`</html>>`</html>`g);

These 3 lines resulted in 1 change being made to the contents of the _MFP_SearchAndReplace.htm file:

<html><head> <title>Here's a test</title> <body> <pre>Test line 1 Test line 2 Test line 3</pre> <b>THIS WEB PAGE URL:</b> http://llbest.com/... </body> </html>

5. Repeat steps 2 and 3 above, this time substituting _MFP_SearchAndReplace3.perlcode for _MFP_SearchAndReplace1.perlcode in step 3:

Now there should be 9 files instead of 8:

HELLO! $FileNameMask='\.(txt|html?)$';$previewMode=1;$maxLevel=1;$readTheFiles=1; PREVIEW MODE: NO FILES ACTUALLY CHANGED. 3 CHANGE(S) MADE TO FILE: "C:\Users\Howard\Downloads\_MFP_SearchAndReplace\_MFP_SearchAndReplace.htm" TOTAL CHANGES MADE TO 1 "\.(txt|html?)$" FILE(S): 3 GOODBYE!

The 3rd Log File (_MFP_SearchAndReplace3.log)

As you can see, the contents of the _MFP_SearchAndReplace3.log file is quite a bit different from the contents of the _MFP_SearchAndReplace1.log or the _MFP_SearchAndReplace2.log files, and if you examine the contents of the _MFP_SearchAndReplace.htm file, you will see that nothing was changed this time. By carefully examining the 3 lines contained in the _MFP_SearchAndReplace3.perlcode file, see if you can explain these results:

###$previewMode=1;&printIt("HELLO!\n\n"); ###&printIt("GOODBYE!\n\n"); $changes+=($buffer=~s`Test l`L`g);

F. Preview Mode

If you want to do a find only or you want to “test the waters” to see if the results are what you expected without actually making any changes, then change $previewMode=0; to $previewMode=1; in the first ### line of the _MFP_SearchAndReplace1.perlcode file:

### $FileNameMask='\.htm$';$previewMode=1;$maxLevel=1;$readTheFiles=1;

G. Subfolder Support

If you want to include, for example, one level of subfolders, then change $maxLevel=1; to $maxLevel=2; in the first ### line of the _MFP_SearchAndReplace1.perlcode file:

### $FileNameMask='\.htm$';$previewMode=0;$maxLevel=2;$readTheFiles=1;

Note: If you want to insure that all levels of subfolders are included, then $maxLevel=99; would certainly do the trick.

H. The Tutorial

Now you are ready to make changes to any number of text files in one fell swoop:

1. Copy the files to be changed to a temporary folder such as C:\Temp2. (Be sure to keep backup copies until you are sure that the changes were made correctly.)

2. Copy the _MFP_SearchAndReplace1.perlcode file to that same folder.

3. Insert the appropriate line of code at the top of the _MFP_SearchAndReplace1.perlcode file (below the line beginning with ###, which must always be the first line of the file), making sure that all other lines are commented out. (It’s a good idea to save the other code for future reference. It is usually easier to make changes to previously written code than to start from scratch. It doesn’t hurt to have hundreds of lines of commented out code.)

4. Change $previewMode=0; to $previewMode=1; in the first line of the _MFP_SearchAndReplace1.perlcode file.

5. Drag and drop the _MFP_SearchAndReplace1.perlcode file icon on top of the MFP_DragAndDropPerlCodeOnMe.bat - Shortcut icon on your computer’s desktop. (No changes are actually made.)

6. If the number of changes made was about what you expected, then change $previewMode=1; back to $previewMode=0;. Otherwise, make corrections to the .perlcode file and try step 5 again.)

7. Again drag and drop the _MFP_SearchAndReplace1.perlcode file icon on top of the MFP_DragAndDropPerlCodeOnMe.bat - Shortcut icon. (Now the changes are actually made.)

Hopefully, all of your files will get changed according to plan.

Note: The Perl code within the very first line, beginning with ### is executed only once, at the very beginning. If there is a second line beginning with ### then it’s Perl code is also executed only once, but at the very end. The Perl code after these first one or two ### lines is exectuted once for each of the files found which match the $FileNameMask criteria.

Note: If the first ### line is missing, or if any of the 4 parameter assignments are missing, then one or more of the following 4 default values are used:
$FileNameMask='\.(txt|html?)$';$previewMode=0;$maxLevel=1;$readTheFiles=1;

Note: Except for the extension, the .log file always has the same name as the corresponding .perlcode file. If you make changes to the .perlcode file and then run it again, the corresponding .log file will be overwritten. Therefore, if you want to save the .log file, it must be moved to a different folder or renamed before the .perlcode file is run again.

Note: Here are 12 Perl variables which may come in handy. What these 12 variables are used for should be more or less obvious from their names, but please don't hesitate to experiment provided that $previewMode=1; is specified within the first ### line:

$0 (Path to the _MFP_DragAndDropPerlCodeOnMe.bat file.)
$buffer (Contains all of the text of the file currently being processed.)
$changes (Zeroed each time a new file is read. Only gets updated within the .perlcode file.)
$dirName
$fileName
$foldersOnly (If $foldersOnly=1;, the $readTheFiles=? parameter is ignored.)
$homeDirName (Always the same as $dirName if $maxLevel=1;.)
$logNoChangesAlso (To also log files which aren’t changed, $logNoChangesAlso=1;.)
$MFPdirName (Directory containing the _MFP_DragAndDropPerlCodeOnMe.bat file.)
$totalChanges (Total of all $changes values.)
$writeChangedFileAsUTF8 (Forces all changed files to be written as UTF8 files.)
$_ (Same as $dirName.$fileName or "$dirName$fileName".)

Note: The included _MFP_SearchAndReplace1.perlcode file also contains much commented-out sample code of varying degrees of complexity. By “commented-out,” I mean that the lines begin with a number sign (“#”), which indicates to the Perl script that those particular lines are to be ignored. It also contains comments which briefly explain the sample code. The first 3 lines of commented-out “Regular Expression” Perl code should, if un-commented-out, each cause one or more changes to the included sample HTML file, but the remaining ones probably won’t. By carefully studying this sample code, you can learn how to do just about everything that the MFP program is capable of doing.

Note: For applications which do not require reading the contents of the files, $readTheFiles=1; should be changed to $readTheFiles=0; (in the first ### line of the .perlcode file).

Note: If you get the message, ***** MFP_DragAndDropPerl CodeOnMe WARNING: FILE "???" NOT CHANGED. UNABLE TO DETERMINED THE ENCODING. SAVE THE FILE AS \"UTF-8 with signature\" AND TRY AGAIN, use Notepad2 to save that particular file with the encoding, “UTF-8 with Signature.” This will solve the problem without fail.

---

| Home | THIS WEB PAGE URL: http://LLBest.com/?P=7d |