How to Find Duplicate Files


Native Auditing vs. Netwrix Auditor for Windows File Servers

We never share your data. Privacy Policy
Native Auditing Netwrix Auditor for Windows File Servers
Steps
  1. Open the PowerShell ISE → Run the following script, adjusting the directory path:

    $Path = '\\PDC\Shared\Accounting' #define path to folders to find duplicate files
    $Files=gci -File -Recurse -path $Path | Select-Object -property FullName,Length
    $Count=1
    $TotalFiles=$Files.Count
    $MatchedSourceFiles=@()
    ForEach ($SourceFile in $Files)
    {
      Write-Progress -Activity "Processing Files" -status "Processing File $Count / $TotalFiles" -PercentComplete ($Count / $TotalFiles * 100)
      $MatchingFiles=@()
      $MatchingFiles=$Files |Where-Object {$_.Length -eq $SourceFile.Length}
      Foreach ($TargetFile in $MatchingFiles)
      {
        if (($SourceFile.FullName -ne $TargetFile.FullName) -and !(($MatchedSourceFiles |
          Select-Object -ExpandProperty File) -contains $TargetFile.FullName))
        {
          Write-Verbose "Matching $($SourceFile.FullName) and $($TargetFile.FullName)"
          Write-Verbose "File sizes match."
          if ((fc.exe /A $SourceFile.FullName $TargetFile.FullName) -contains "FC: no differences encountered")
          {
            Write-Verbose "Match found."
            $MatchingFiles+=$TargetFile.FullName
          }
        }
      }
      if ($MatchingFiles.Count -gt 0)
      {
        $NewObject=[pscustomobject][ordered]@{
          File=$SourceFile.FullName
          MatchingFiles=$MatchingFiles
        }
        $MatchedSourceFiles+=$NewObject
      }
      $Count+=1
    }
    $MatchedSourceFiles

  2. Review the results:

Powershell duplicate files report

  1. Run Netwrix Auditor → Navigate to "Reports" → Click "File Servers" → Choose "File Servers - State-in-Time" → Select "Duplicate Files" → Click "View". 
  2. Specify the appropriate path in the "Object UNC Path" filter.
  3. Click "View Report".


To save the report, click the "Export" button → Select a format, such as PDF or CSV → Click "Save as" → Choose a location to save it.

Netwrix Auditor Duplicate Files Report: shows files the same name and size.

Finding Duplicate Files Using PowerShell or Netwrix Auditor

Anyone who manages a file storage has to keep track of the size of files to ensure there is always enough free space. Documents, photos, backups and other can quickly eat up your shared file resources — especially if you have a lot of duplicates. Duplicate files are often the result of users’ mistakes, such as double copy actions or incorrect folder transfers. To avoid wasting space and driving up storage costs, you have to analyze your file structure, find duplicate files and remove them. A simple Windows PowerShell script can help you complete this tedious task faster. Just run the code provided above, being sure to specify the path to the folders you’re interested in.

Alternatively, you might try Netwrix Auditor for Windows File Servers. Ditch manual scripting and analyze your storage in mere minutes. Then simply subscribe to easy-to-use reports that list duplicates, stale data and large files and get them automatically by email, so you can continuously spot and remediate issues as they arise and keep your storage orderly and efficient.