Copyright © 2003-2012 by Prof. Timo Salmi  
Last modified Sat 10-Nov-2012 21:15:04

Assorted NT/2000/XP/.. CMD.EXE Script Tricks
From the html version of the tscmd.zip 1cmdfaq.txt file
To the Description and the Index

This page is edited from the 1cmdfaq.txt faq-file contained in my tscmd.zip command line interface (CLI) collection. That zipped file has much additional material, including a number of detached .cmd script files. It is recommended that you also get the zipped version as a companion.

Please see "The Description and the Index page" for the conditions of usage and other such information.

162} Finding duplicate filenames in a directory tree.

See the responses to this posting (if the Google message link fails try the link within the brackets)

  Google Groups Dec 3 2007, 2:04 am [M]
in the thread
  Finding duplicate files in a directory tree Google Groups


Below is the solution which I use myself. It is unrelated to the references in the above. This is a somewhat complicated a task which also is heavy on computing power. Therefore, my solution depends on a UNIX uniq.exe port of common GNU utilities to native Win32 collection UnxUpdates.zip. On my system I have renamed uniq.exe to unxuniq.exe
  @echo off & setlocal enableextensions

  :: Obviously, for debugging
  set debug_=true
  set debug_=

  call :ProgramTitle

  if "%~1"==""   (call :ProgramHelp & goto _out)
  if "%~1"=="?"  (call :ProgramHelp & goto _out)
  if "%~1"=="/?" (call :ProgramHelp & goto _out)

  :: Assign a value for the temporary folder variable temp_
  call :AssignTemp temp_
  if defined debug_ (call :ShowVarValue temp_& goto _out)

  :: Check that an essential auxiliary program is available
  call :IsProgramAtPath "unxuniq.exe" uniqfound_
  if not defined uniqfound_ (
    echo The UNIX-like utility uniq.exe from UnxUpdates.zip is not avalable
    echo Or has NOT been renamed unxuniq.exe as required by this script
    echo For UnxUpdates.zip see e.g. http://www.netikka.net/tsneti/info/tscmd.php#gawk
    goto :_out)

  :: Preset options of the starting root for the search, easy to customize
  set switchok_=
  if /i "%~1"=="/1" (
    set switchok_=true
    set topfolder_=%USERPROFILE%\My Documents\My Texts)
  if /i "%~1"=="/2" (
    set switchok_=true
    set topfolder_=%USERPROFILE%\My Documents\My Music)

  :: Check the switch's feasibility
  if not defined switchok_ (
    echo Error: Invalid parameter "%~1"
    call :ProgramHelp
    goto :_out)

  :: Check for the starting point folder's existence
  if not exist "%topfolder_%\" (
    echo Exiting: The assigned folder to start the search from not found
    echo %topfolder_%
    goto :_out)

  :: Remove possible leftover auxiliary files from previous runs
  call :CleanUp

  :: Search for the duplicates
  for /f "usebackq delims=" %%f in (`
    dir /s /b /a:-d "%topfolder_%\*.*"`) do (
      echo %%~nxf) >> "%temp_%\tmp1.lst"
  sort "%temp_%\tmp1.lst" > "%temp_%\tmp2.lst"
  rem The -d switch: only print duplicate lines
  rem The -i switch: ignore differences in case when comparing
  unxuniq -d -i "%temp_%\tmp2.lst" > "%temp_%\tmp3.lst"
  echo The results for %pcid_%
  call :TestForZeroFile "%temp_%\tmp3.lst" zero_
  if defined zero_ (
    echo No duplicate filenames found
    goto _out)
  for /f "usebackq delims=" %%a in ("%temp_%\tmp3.lst"
    ) do dir /s /b "%topfolder_%\%%~a" >> "%temp_%\tmp4.lst"
  for /f "usebackq delims=" %%f in (`type "%temp_%\tmp4.lst"`
    ) do echo "%%~f"

  if not defined debug_ call :CleanUp

  :: The main program ends
  if not defined cmdbox if defined PauseIfFromDesktop pause
  endlocal & goto :EOF

  echo +-----------------------------------------------------------+
  echo ^ FINDDUPL.CMD Find duplicate filenames on a directory tree ^
  echo ^ By Prof. Timo Salmi, Last modified Fri 9-Nov-2012         ^
  echo +-----------------------------------------------------------+
  goto :EOF

  echo Usage: FINDDUPL.CMD [StartFromFolder]
  echo /1 = %USERPROFILE%\My Documents\My Texts
  echo /2 = %USERPROFILE%\My Documents\My Music
  goto :EOF

  :IsProgramAtPath SearchFor found_
  setlocal enableextensions disabledelayedexpansion
  set found_=
  for %%f in ("%~1") do set found_="%%~$PATH:f"
  if exist "%found_%\" set found_=
  if exist "%~1" if not exist "%~1\" set found_="%~1"
  if [%found_%]==[""] set found_=
  endlocal & set "%~2=%found_%" & goto :EOF

  set return_=%temp%
  if defined mytemp if exist "%mytemp%\" set return_=%mytemp%
  endlocal & set "%1=%return_%" & goto :EOF

  :: For debugging, show verbally the value of an environment variable
  :ShowVarValue TheNameOfTheVariable
  set par1_=%~1
  call set value_=%%%par1_%%%
  echo The value of %~1 is %value_%
  endlocal & goto :EOF

  setlocal enableextensions
  set file_=%~1
  for %%f in ("%file_%") do set size_=%%~zf
  set return_=
  if %size_% EQU 0 set return_=true
  endlocal & set "%~2=%return_%" & goto :EOF

  for %%f in (
    "%temp_%\tmp4.lst") do (
      if exist "%%~f" del "%%~f")
  goto :EOF

The question naturally arises whether the task can be done without resorting to the uniq.exe UNIX port. Yes, but the solution is more sensitive to the potential irregularities in the file names. In particular, there should be no exclamation marks (!) in the file names. The pure solution is to substitute the line
  unxuniq -d -i "%temp_%\tmp2.lst" > "%temp_%\tmp3.lst"
  call :Uniq_id "%temp_%\tmp2.lst" > "%temp_%\tmp3.lst"
and add the following subroutine to the script
  :: Detect and display adjacent duplicate lines of a file
  setlocal enableextensions enabledelayedexpansion
  set prev_=
  for /f "usebackq delims=" %%a in (`type "%~1"`) do (
    set curr_=%%~a
    if /i !curr_!==!prev_! echo !curr_!
    set prev_=!curr_!)
  endlocal & goto :EOF

  About a potential Vista problem Google Groups 09 Nov 2012 20:26:54 +0800