26

Jaap Brasser’s Favorite PowerShell Tips and Tricks

When I met up with Aleksandar Nikolic at TechEd Europe I was asked if I wanted to write an article for PowerShell Magazine. I hope my tips are useful to you and feel free to leave a comment or question if you would like me to clarify anything.

 Using –ErrorVariable to quickly check effective permissions for a user

When working with permissions it is useful to know to which folders a user has access and to which folders access is denied. Although the Windows GUI does offer options for effective permissions, this is hardly appropriate when a large folder structure should be checked. This is where Get-ChildItem comes into play:

Get-ChildItem –Force –Recurse –ErrorAction SilentlyContinue –ErrorVariable AccessDenied

This will display all files and folders that are accessible, and stores all inaccessible folders in the AccessDenied array. For the purpose of this example we will focus on the files and folders that generated an error. We do this by omitting the output of Get-ChildItem by writing the output to the special variable $null:

$null = Get-ChildItem –Force –Recurse –ErrorAction SilentlyContinue –ErrorVariable AccessDenied

This gives us a list that contains all errors that have been generated by Get-ChildItem which conveniently gives us all the paths that were inaccessible.

$AccessDenied | ForEach-Object {$_.Exception}

This command displays all the inaccessible paths including the error messages. To just display paths we can use the TargetObject property.

$AccessDenied | ForEach-Object {$_.TargetObject}

This will display the names of the files and folders that were inaccessible. Because this catches all errors there might also be other reasons besides permissions which might be restricting access to a file or folder. Therefore it is important to verify the errors before assuming why access might have been denied to a folder.

As a bonus, since we should already be using PowerShell v3, we can shorten the last two commands:

$AccessDenied.Exception

$AccessDenied.TargetObject

Parse Robocopy output in PowerShell to find long path names – Workaround for 260 character limit in Windows

The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.

The dreaded error, I think everyone working with Windows has run into this at some point. Even in Windows 8/Server 2012 we are still limited by this. It might be a file server with an exotic folder structure or just a single file with a long name that we cannot get rid of. This tip will show how to use Robocopy to detect the files and folders that have a long path.

Let’s start out with a simple example displaying the difference in speed of using Get-ChildItem in combination with Measure-Object, or by using Robocopy to count the number of files and the total size of a folder:

(Measure-Command {1..10 | ForEach-Object {Get-ChildItem C:\Windows -Recurse -Force -ErrorAction SilentlyContinue | Where-Object {!$_.PSIsContainer} | Measure-Object Length -Sum}}).TotalSeconds

283.4675185

(Measure-Command {1..10 | ForEach-Object {robocopy.exe c:\windows c:\doesnotexist /l /e /b /copy:d /np /fp /nfl /ndl /njh}}).TotalSeconds

51.6469264

Both these commands generate similar output, but by using Robocopy we manage to get the results several times faster.

Now on to the more interesting stuff; using Robocopy to detect long file and folder paths. Since Robocopy uses a predictable output we can parse its output to extract the information we require. In this scenario we will use Robocopy to find files with a path longer than the maximum allowed. The next example will output a list of files that have a path longer than 260 characters:

robocopy.exe c:\deeppathtest c:\doesnotexist /l /e /b /np /fp /njh /njs /ndl | Where-Object {$_.Length -ge 286} | ForEach-Object {$_.Substring(26,$_.Length-26)}

To simplify this command I have written a short function to this process. It can be downloaded from the TechNet Script Repository: Get-LongPathName.ps1

Using this function, we can search for files and folders with long pathnames. The function outputs an object that contains the PathLength, Type (File or Folder) and the FullPath to the file or folder. Here is an example of how to utilize this function:

Get-LongPathName -FolderPath ‘C:\Deeppathtest’ -MaxDepth 200

This will output all files and folders with a path longer than 200 characters in the C:\Deeppathtest folder. And because this function returns an object the output can be sorted and otherwise manipulated making the output easy to work with.

Function Get-LongPathName

Function Get-LongPathNames {

	param(

	    [CmdletBinding()]

	    [Parameter(Position=0,Mandatory=$true,ValueFromPipeline=$true, ValueFromPipelineByPropertyName=$true)]
	    [string[]]$FolderPath,

	    [ValidateRange(10,248)][int16]$MaxDepth=248
	)

 

    begin {
        if (!(Test-Path -Path $(Join-Path $env:SystemRoot 'System32\robocopy.exe'))) {
            write-warning "Robocopy not found, please install robocopy"
            return
        }

        $Results = @()
    }
 

    process {

        foreach ($Path in $FolderPath) {

            $RoboOutput = robocopy.exe $Path c:\doesnotexist /l /e /b /np /fp /njh /njs 

            # Build Array for files. 26 is added to $MaxDepth the text string generated by Robocopy

            $RoboOutput | Select-String "New File" | Where-Object {$_.Line.Length -ge $MaxDepth+26} | ForEach-Object {

                $Props = @{
                    FullPath= $_.Line.Substring(26,$_.Line.Length-26)
                    Type = 'File'
                    PathLength = $_.Line.Length-26
                }

                $Results += New-Object -TypeName PSCustomObject -Property $Props
            } 

            # Append folders to array
            # Build Array for files

            $RoboOutput | Select-String "New Dir" | Where-Object {$_.Line.Length -ge ($MaxDepth+22)} | ForEach-Object {
                
                $Props = @{
                    FullPath = $_.Line.Substring(22,$_.Line.Length-22)
                    Type = 'Folder'
                    PathLength = $_.Line.Length-22
                }

                $Results += New-Object -TypeName PSCustomObject -Property $Props
            }
        }
    }

 

    end {
        $Results
    }

}
Filed in: Columns, Tips and Tricks Tags: ,

26 Responses to "Jaap Brasser’s Favorite PowerShell Tips and Tricks"

  1. mp says:

    It should be
    $RoboOutput = robocopy.exe $FolderPath c:doesnotexist /l /e /b /np /fp /njh /njs
    or im wrong?

  2. Guest says:

    You are correct, the version of the script posted here was outdated. It did not properly function when not called from the pipeline. The post has been updated to include the most recent version.

  3. kjhejdvoe lwaja says:

    The time it takes the robocopy command to run will be greatly affected by whether it is run first or second due to caching that takes place after the first run. Try reversing the order of the commands and you will see robocopy start to fall behind.

  4. DVS says:

    Great idea using “robocopy.” I ran into PowerShell’s 260 character limitation before, and tried using the “cmd /c dir” command

  5. ricktor says:

    thanks for the info.,you could also try long path tool. it helped me with error 1320 in Win 7.,:)

  6. Steve Hess says:

    I added /r:0 /xj /nc /ndl /bytes
    /r:0 disables retries
    /xj skips junction points
    /ndl removes directories from the list
    /nc removes the “new file” column
    /bytes makes all sizes in bytes
    You could also add /ns if your aren’t interested in sizes

  7. Yomodo says:

    Hi,

    PowerShell can be easily be used with AlphaFS.dll to do actual file I/O stuff
    without the PATH TOO LONG hassle.

    Please see at Codeplex: https://alphafs.codeplex.com/ for this .NET project.

  8. julias4 says:

    When working with permissions it is useful to know to which folders a user has access and to which folders access is denied. Then that time you can try Long Path Tool for following those error.

  9. Michael McNally says:

    Maybe I’m missing something here. Using this method I can generate a list of the folder paths that are too long for PowerShell to use. What do I do next, just say “Oh well, I just won’t do to those folders.”
    I mean, what’s the point? I guess knowing which folder paths you can’t use is better than not knowing, but it’s not better than a workaround that actually lets you use them.

  10. Often black dots, representing the blood vessels which feed the wart,
    are visible. They are the same color as your
    skin and have growths that lok liuke threads sticking outt
    of them. Another popular folk remedy is the use of duct tape.

  11. Matt Johnsen says:

    Jaap, I’m in an odd position where I need to do a Get-ACL on each subfolder in the file structure (which is deep and massive) to determine what Active Directory Groups are used.
    I’m able to access the folder names using robocopy.exe and export them to CSV but when I attempt to access the ACLs I’m running into an issue.
    Currently I am mapping a drive to each folder found and doing a -recurse to give me a depth of 520 but it results in the script taking forever to complete.
    Do you have any advice?

    • Jaap Brasser says:

      Absolutely, the mapped drive method is one way, another slightly more efficient method would be to use symbolic links instead of mapped drives. In PowerShell 5.0 the New-Item cmdlet supports the creation of symbolic links.

      As for gathering ACLs that are in deep paths, I would recommend you attempt to use the File System Security PowerShell Module, it allows for deep paths as it utilizes a different API, the AlphaFS project, to access files and folders. Have a look at it here and let me know if that solves your problem:

      https://ntfssecurity.codeplex.com/

      • Vinc says:

        Hi Jaap
        I am not a very experience scripting guy or programmer but my issue is this.
        I have downloaded NTFS Security modules and installed them i am trying to do the same thing as Matt.
        My question is;
        how are these module meant to help if in order to get GET-NTFSAccess output i need to run the GET-ChildItem.
        Did i not understand something ?
        I also read on your post about robocopy to bypass this limitation but i can’t figure out a way to link robocopy source path to something like a variable instead of destination path and then recall the variable in powershell and for last GET-NTFSAccess
        Thanks a million for any help

        • RBA says:

          Hi Vinc,

          Have you solved your question :
          “how are these module meant to help if in order to get GET-NTFSAccess output i need to run the GET-ChildItem.”

          >>> I came exactly the same conclusion as you and have no answer…

          Thanks a billion 😉 for any help..

          • Vinc says:

            HI RBA
            you can import the alphafs module (Get-Alphafschilditem) it worked for me for a while then stopped and did not have time to figure out the reason why it stopped. But again it worked about a couple of time

            v

          • Jaap Brasser says:

            Indeed Vinc, the problem with AlphaFS is that it might introduce other errors and bugs. If you are experiencing problems with the Get-AlphafsChilditem you could check the project page to see if your problems are known issues.

          • RBA says:

            I finally found a workaround by using robocopy as explained in this article :

            ## “Nouveau R” should be replaced by “New Dir” in english
            (robocopy.exe $Path c:\doesnotexist /l /e /b /np /fp /njh) | Select-String “Nouveau r” | ForEach-Object {
            $Props = @{
            FullName= $_.Line.Substring(22,$_.Line.Length-22)
            }
            $folders += New-Object -TypeName PSCustomObject -Property $Props
            }
            Write-Progress -Activity “Collecting folders” -Status $Path
            -PercentComplete 100

            $i = 0
            $FolderCount = $folders.count

            ForEach ($folder in $folders) {

            Write-Progress -Activity "Scanning folders" -CurrentOperation $folder
            -Status $Path -PercentComplete ($i/$FolderCount*100)
            $i++

            Try {
            # Get-ACL replaced by Get-NTFSAccess (248 max car workaround)
            #$acl = Get-ACL -Path $folder -ErrorAction Continue
            $acl = Get-NTFSAccess -Path $folder.FullName -ErrorAction Stop
            }
            Catch {
            $ErrorLog += New-Object PSObject `
            -Property @{CategoryInfo=$_.CategoryInfo;TargetObject=$folder.FullName}
            }

          • Vinc says:

            I see you are much more advanced then me in your script, i just started with Power Shell.

            However even your script returns me an error

            Missing closing ‘}’ in statement block or type definition.
            + CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException
            + FullyQualifiedErrorId : MissingEndCurlyBrace

          • RBA says:

            Here’s my final script (original version here : http://blogs.technet.com/b/ashleymcglone/archive/2014/03/17/powershell-to-find-where-your-active-directory-groups-are-used-on-file-shares.aspx)
            I just changed it to allow paths longer than 260 chars.
            I finally removed the robocopy listing and used Get-ChildItem2.
            Warning : you need to install NTFS-Security !
            Warning2 : Some quotes (“) have to be changed via notepad or notepad++ before running this script (caused by the comment textarea that transform them…)

            Function Get-ACE {
            Param (
            [parameter(Mandatory=$true)]
            [string]
            [ValidateScript({Test-Path -Path $_})]
            $Path
            )

            $ErrorLog = @()

            #on definit une barre de progression à 0%
            Write-Progress -Activity “Collecte des dossiers” -Status $Path
            -PercentComplete 0
            #initialise le tableau $folders
            $folders = @()
            #récupère en tout premier le dossier pour lequel on souhaite rechercher tous les enfants
            $folders += Get-Item $Path | Select-Object -ExpandProperty FullName
            Try{
            # recherche de tous les enfants de $Path
            $subfolders = Get-Childitem2 $Path -Recurse -ErrorVariable +ErrorLog -Directory -ErrorAction Stop |
            Select-Object -ExpandProperty FullName
            }catch{
            $ErrorLog += New-Object PSObject -Property @{CategoryInfo=$_.CategoryInfo;TargetObject=$_}
            }
            # (robocopy.exe $Path c:\doesnotexist /l /e /b /np /fp /njh) | Select-String "Nouveau rép." | ForEach-Object {
            # $Props = @{
            # FullName= $_.Line.Substring(22,$_.Line.Length-22)
            # }
            # $folders += New-Object -TypeName PSCustomObject -Property $Props
            # }
            #on met la barre de progression à 100%
            Write-Progress -Activity "Collecte des dossiers" -Status $Path

            -PercentComplete 100

            #on associe tous les enfants à $folders
            If ($subfolders) {$folders += $subfolders}
            $i = 0
            $FolderCount = $folders.count

            #parcours des dossiers
            ForEach ($folder in $folders) {

            Write-Progress -Activity “Scanning folders” -CurrentOperation $folder
            -Status $Path -PercentComplete ($i/$FolderCount*100)
            $i++

            Try {
            #récupère la liste des droits associés au dossier $folder et ErrorAction stop envoie
            # dans l'instuction Catch si une quelconque erreur se produit
            $acl = Get-NTFSAccess -Path $folder -ErrorAction stop
            }
            Catch {
            #En cas d'erreur on ajout une nouvelle ligne dans la variable ErrorLog
            $ErrorLog += New-Object PSObject

            -Property @{CategoryInfo=$_.CategoryInfo;TargetObject=$folder}
            }

            #DOMAIN_NAME should be replace by your company DOMAIN_NAME
            $acl | ? {$_.IsInherited -eq $false -AND $_.Account -like “DOMAIN_NAME*”} |
            Select-Object
            @{name='Racine';expression={$path}},

            @{name=’Dossier’;expression={$folder}},
            Account, AccessControlType, AccountType, AccessRights, IsInherited,InheritanceEnabled,

            InheritanceFlags, InheritedFrom, PropagationFlags

            }

            $ErrorLog |
            Select-Object CategoryInfo, TargetObject |
            Export-Csv “.\Errors_$($Path.Replace(‘\’,’_’).Replace(‘:’,’_’)).csv” -NoTypeInformation -Encoding unicode
            }

            # recupere le contenu du fichier path.txt et lance la fonction
            # pour chacun des chemins
            Get-Content .\paths.txt |
            ForEach-Object {
            If (Test-Path -Path $_) {
            Get-ACE -Path $_ |
            Export-CSV `
            -Path “.\ACLs_$($_.Replace(‘\’,’_’).Replace(‘:’,’_’)).csv” -NoTypeInformation -Encoding unicode
            } Else {
            Write-Warning “Invalid path: $_”
            }
            }

  12. Vincenzo says:

    Hi Jaap
    this is a good article
    I am trying to do the same as Matt using robocopy to get all the files and folders but can not get it properly working
    What i would like to do is get a list of all the file and folders and then Get-ACL and find out theone without permisision
    Can you give any help

  13. Roksolana says:

    Long Path Tool help me a lot when i have an issue like file deleting or renaming the file. Also good to use if file name and file extension is too long.

  14. barryk desteve says:

    Do not worry if you want to remove the blocked files or too long path files from your system, here I suggest a smooth way. Use “Long path tool” software and keep yourself cool.

  15. Johnraf says:

    I used to have similar problems too, but after using
    “long path tool” You can use to solve this problem.

Leave a Reply

Submit Comment

© 2016 PowerShell Magazine. All rights reserved. XHTML / CSS Valid.
Proudly designed by Theme Junkie.
%d bloggers like this: