Category Archives: RoboCopy

Some tips & tricks I picked up over the years

RoboCopy class

‘Sup PSHomies,

It all started a year ago… Always wanting to learn anything PowerShell related, classes caught my eye ever since it was introduced in v5.  I wanted to try my hand at classes with a real life application… So I got on twitter for some tips…


Doug was kind enough to reach out and point me in the right direction, for which I owe him a great debt! Appreciate it Doug!!!

Like I said, I wanted to try my hand at classes with a real life application… If you’ve read my blogs then you’ll know that I’m a fan of robocopy, seriously, huge fan! . Did I mention how awesome robocopy is? 😛 I think I found my real life application 😉

When I started out with my Robocopy class, it was just about logging initially, but it could be so much more! Classes are native to v5. Now that v5 is mainstream I decided to finish the class. Richard Siddaway’s article  was just the spark I needed to get me going (again)!

Here’s what the Robocopy class looks like:


Here a quick rundown on the properties:

The source/destination properties of the class are self explanatory (if you’re familiar with robocopy). The property logdir and JobID will be used to define a valid logfile name (with extension .log). Robocopy has quite a bit of options. I wanted to keep it as generic as possible. The property $this.Options is still a work in progress. The property $this.WildCards  is where you’ll define what will be filtered.  I’ll get back to rcCMDs and rcExitCodes later on…

These are the methods I came up with (so far, still a work in progress)

  • Mirror(). Mirrors $this.Source to $this.Destination with some default options
  • Move(). Moves this.Source to $this.Destination with some default options
  • RollBack(). Rollback $this.Destination to $this.Source with some default options
  • Sync(). Sync will synchronize the delta’s from $this.Source to $this.Destination using any additional $this.Options defined (at least that’s the idea). I’ve added a few options by default, mostly to exclude files and folders, think recycle.bin “System Volume Information” and the likes.
  • VerifyPaths(). This let’s you know if the $this.Source, $this.Destination and $this.LogDir are valid.
  • GetlistSource(). This will list the content of the $this.Source
  • GetListDestionation(). This will list the content of $this.Destination
  • GetLogSummary. This will return a summary of the log file (Hehe). The method is static so that you don’t have to instantiate the class in order to use it. (Thanks again Doug!)

The two methods: StartRCProcess and ExecuteRCCMD are actually helper methods. I just haven’t figured out how that works in classes. Ideally I’d like to have them hidden or as a function if that even makes sense. So here’s where they come in. At first I just executed robocopy with the necessary arguments. If you’re not interested in the exitcode then using ExecuteRCCMD is what you need. I wrote a blog about enumerating RoboCopy Exitcodes. Using $LastExitCode isn’t going to cut it if you decide to run robocopy jobs parallel. That’s where StartRCProcess comes in.Using Start-Process comes with an overhead of say 20 MB, which could add up in the long run. You do need to wait until the process has finished to retrieve the exitcode. If you really need the exitcode then StartRCProcess is what you need. The property $this.rcExitCodes will only be populated if StartRCProcess is used. Both will populate the $this.rcCMDs property.

Ok I think I’ve covered the basics, time to show some code! 😉

[Flags()] Enum RoboCopyExitCodes{
NoChange = 0
OKCopy = 1
ExtraFiles = 2
MismatchedFilesFolders = 4
FailedCopyAttempts = 8
FatalError = 16
Class RoboCopy{
#default constructor
$this.Source = $src
$this.Destination = $des
$this.JobID = $jid
$this.LogDir = $ld
$this.WildCards = $wc
return [PSCustomObject]@{
Source = $(Test-Path -Path $this.Source)
Destination = $(Test-Path -Path $this.Destination)
LogDirectory = $(Test-Path -Path $this.LogDir)
#Save RoboCopy Command
$MessageData = $('robocopy "{0}" "{1}" {2} {3}' -f $source,$target,$($params -join ' '),$logFile)
$this.rcCMDs += Write-Information -MessageData $MessageData 6>&1 -Tags $tag | Select-Object *
#Execute Process
$rcArgs = "`"$($source)`" `"$($target)`" $params $logFile"
$rcResult = Start-Process robocopy -ArgumentList $rcArgs -WindowStyle Hidden -PassThru -Wait
#Save ExitCode
$this.rcExitCodes += [PSCustomObject]@{
ExitCode = [RoboCopyExitCodes]$rcResult.ExitCode
StartTime = $rcResult.StartTime
ExitTime = $rcResult.ExitTime
rcCMD = $MessageData
#Save RoboCopy Command
$MessageData = $('robocopy "{0}" "{1}" {2} {3}' -f $source,$target,$($params -join ' '),$logFile)
$this.rcCMDs += Write-Information -MessageData $MessageData 6>&1 -Tags $tag | Select-Object *
#Execute Robocopy CMD
$rcArgs = @("$($source)","$($target)",$params,$logFile)
robocopy @rcArgs
$tag = @('Mirror',$this.JobID)
$params = New-Object System.Collections.Arraylist
$logFile = '/LOG:{0}\mirror-{1}.log' -f $this.LogDir,$this.JobID
$tag = @('Move',$this.JobID)
$params = New-Object System.Collections.Arraylist
$logFile = '/LOG:{0}\move-{1}.log' -f $this.LogDir,$this.JobID
$tag = @('RollBack',$this.JobID)
$params = New-Object System.Collections.Arraylist
$logFile = '/LOG:{0}\rollback-{1}.log' -f $this.LogDir,$this.JobID
$tag = @('Sync',$this.JobID)
#Excluded Files & Directories
$XF = @(
$XD = @(
'Boot Recycler'
"`"System Volume Information`""
$Excluded = "$($XF -join ' ') $($XD -join ' ')"
$params = New-Object System.Collections.Arraylist
$logFile = '/LOG:{0}\sync-{1}.log' -f $this.LogDir,$this.JobID
#Save RoboCopy Command
$arrMessageDataInput = @(
$($params -join ' ')
$($this.Options -join ' ')
$MessageData = $('robocopy "{0}" "{1}" {2} {3} {4} {5}' -f $arrMessageDataInput )
$this.rcCMDs += Write-Information -MessageData $MessageData 6>&1 -Tags $tag | Select-Object *
#Execute Robocopy CMD
$rcArgs = @("$($this.Source)","$($this.Destination)",$params,$XF,$XD,$logFile)
robocopy @rcArgs
$tag = @('ListSRC',$this.JobID)
$params = New-Object System.Collections.Arraylist
$params.AddRange(@($($this.WildCards),'/L','/S','/E', '/BYTES','/FP','/NC','/NDL','/TS','/R:0','/W:0'))
$logFile = '/LOG:{0}\listSRC-{1}.log' -f $this.LogDir,$this.JobID
$tag = 'ListDES'
$params = New-Object System.Collections.Arraylist
$logFile = '/LOG:{0}\listDES-{1}.log' -f $this.LogDir,$this.JobID
static [PSCustomObject]GetLogSummary([String]$rcLogFile){
filter Get-CapacitySize {
'{0:N2} {1}' -f $(
if ($_ -lt 1kb) { $_, 'Bytes' }
elseif ($_ -lt 1mb) { ($_/1kb), 'KB' }
elseif ($_ -lt 1gb) { ($_/1mb), 'MB' }
elseif ($_ -lt 1tb) { ($_/1gb), 'GB' }
elseif ($_ -lt 1pb) { ($_/1tb), 'TB' }
else { ($_/1pb), 'PB' }
$rcLogSummary = [PSCustomObject]@{
Start = $null
End = $null
LogFile = $null
Source = $null
Destination = $null
TotalDirs = $null
CopiedDirs = $null
FailedDirs = $null
TotalFiles = $null
CopiedFiles = $null
FailedFiles = $null
TotalBytes = $null
CopiedBytes = $null
FailedBytes = $null
TotalTimes = $null
Speed = $null
$rcLogSummary.LogFile = $rcLogFile.Split('\')[-1].ToLower()
$logFileContent = Get-Content $rcLogFile -Raw
[regex]$regex_Start = 'Started\s:\s+(?<StartTime>.+[^\n\r])'
if ($logFileContent -match $regex_Start){
$rcLogSummary.Start = $Matches['StartTime']
[regex]$regex_End = 'Ended\s:\s+(?<EndTime>.+[^\n\r])'
if ($logFileContent -match $regex_End){
$rcLogSummary.End = $Matches['EndTime']
[regex]$regex_Source = 'Source\s:\s+(?<Source>.+[^\n\r])'
if($logFileContent -match $regex_Source){
$rcLogSummary.Source = $Matches['Source'].Tolower()
[regex]$regex_Target = 'Dest\s:\s+(?<Target>.+[^\n\r])'
if($logFileContent -match $regex_Target){
$rcLogSummary.Destination = $Matches['Target'].ToLower()
[regex]$regex_Dirs = 'Dirs\s:\s+(?<TotalDirs>\d+)\s+(?<CopiedDirs>\d+)(?:\s+\d+){2}\s+(?<FailedDirs>\d+)\s+\d+'
if ($logFileContent -match $regex_Dirs){
$rcLogSummary.TotalDirs = [int]$Matches['TotalDirs']
$rcLogSummary.CopiedDirs = [int]$Matches['CopiedDirs']
$rcLogSummary.FailedDirs = [int]$Matches['FailedDirs']
[regex]$regex_Files = 'Files\s:\s+(?<TotalFiles>\d+)\s+(?<CopiedFiles>\d+)(?:\s+\d+){2}\s+(?<FailedFiles>\d+)\s+\d+'
if ($logFileContent -match $regex_Files){
$rcLogSummary.TotalFiles = [int]$Matches['TotalFiles']
$rcLogSummary.CopiedFiles = [int]$Matches['CopiedFiles']
$rcLogSummary.FailedFiles = [int]$Matches['FailedFiles']
[regex]$regex_Speed = 'Speed\s:\s+(?<Speed>.+\/min)'
if ($logFileContent -match $regex_Speed){
$rcLogSummary.Speed = $Matches['Speed']
$arrBytes = @(
'Bytes\s:\s+(?<TotalBytes>(\d+\.\d+\s)[bmg]|\d+)\s+' #TotalBytes
'(?<CopiedBytes>\d+.\d+\s[bmg]|\d+)\s+' #CopiedBytes
'(?:(\d+.\d+\s[bmg]|\d+)\s+){2}' #Skip two
'(?<FailedBytes>\d+.\d+\s[bmg]|\d+)' #FailedBytes
[regex]$regex_Bytes = -join $arrBytes
if ($logFileContent -match $regex_Bytes){
$rcLogSummary.TotalBytes = [int64]$Matches['TotalBytes'] | Get-CapacitySize
$rcLogSummary.CopiedBytes = [int64]$Matches['CopiedBytes'] | Get-CapacitySize
$rcLogSummary.FailedBytes = [int64]$Matches['FailedBytes'] | Get-CapacitySize
[regex]$regex_Times = 'Times\s:\s+(?<TotalTimes>\d+:\d+:\d+)'
if ($logFileContent -match $regex_Times){
$rcLogSummary.TotalTimes = $Matches['TotalTimes']
return $rcLogSummary
$paramRoboCopy = @(
$($this.WildCards -join ' ')
$('/LOG:{0}\{1}.log' -f $this.LogDir,$this.JobID)
return $('robocopy "{0}" "{1}" {2} {3} {4}' -f $paramRoboCopy)

Here what’s happening in the List methods:


GetListSource() is using $this.StartRCProcess to generate a list of $this.Source using some default option. While writing I noticed that I forgot to add the wildcards to the parameter. All I had to do was add it!. I added it at the beginning so it lines up accordingly… Robocopy is fickle like that…  GetListDestination does the same only it uses ExecuteRCCMD instead.

Here’s what’s going on in StartRCProcess and ExcuteRCCMD


Both StartRCProcess and ExcuteRCCMD will save the robocopy command using Write-Information. I’m loving Write-Information more and more! StartRCProcess saves the exitcode with some extra information. Here’s where the robocopy exitcode enumeration came in handy! ExecuteRCCMD will run robocopy with the specified arguments. Truth be told I’m more partial to the ExecuteRCCMD method. I added the StartRCProcess more for demo purposes and finally getting to use my Robocopy exitcode enumeration!

For Mirror(),Move() and RollBack(), I omitted the Wildcards. These methods all or nothing in my opinion. If omitted, . will be the default.

Sync() had me going for a while. I still have some issues with Options. For now Sync() uses some default switches. Like I said work in progress…

Quite a bit of code, so does it work? Here’s some code to play with. be sure to edit the source,destination and logdir to your liking. Just remember that robocopy is unforgiving so make sure not to use it of production folders!

#region Main
$rc = [RoboCopy]::New('C:\scripts\move','C:\temp\move','rc-0001','c:\scripts\log',@('*.*'))

#Run RoboCopy methods

#Get RoboCopy LogFile Summary

#Get RoboCopy executed CMDs

First I instantiate the class with arguments. I then run the methods Sync(),GetListSource() and GetListDestination(). Up next is retrieve the LogSummaries from the methods. Here’s a screenshot of the Sync LogSummary


I did a select of $rc.rcCMDs to give you an idea what is being stored


Only want ListDES?

$rc.rcCMDs |
Where-Object{$_.Tags -contains 'ListDes'} |
Select-Object -Property Time*,Tag*,Mess*

The information stream is quite handy! The tags will definitely come in handy when you need to filter on action verb or job ID.

The methods GetListSource() & Mirror() both make use of StartRCProcess(), so let’s see what $rc.rcExitcode has shall we?



This is by far my longest blog, if you made this far then… Congratulations! There’s still so much to discover when it comes to classes.

Classes definitely  makes your code look and feel more like a developer 😉 . I feel more comfortable giving a colleague this class than a bunch of scripts. In Richard’s article he’s using both classes and modules. There are sure to be some gotcha’s… Do you go all in with classes or only partial?

I’m hoping that the community can shed some light on the subject. I’d love to hear from you guys on how to improve on this… Let’s make this year, a year of PowerShell Classes! 😛

Hope it’s worth something to you…




RoboCopy ExitCodes the PowerShell way

‘Sup PSHomies!

Boe Prox recently blogged about building a enum that supports bitfields in PowerShell, definitely worth the read!

The first thing I thought about was RoboCopy! As you may or may not know, I’m a huge fan (no pun intended) of RoboCopy! I know that RoboCopy returns exitcodes, I just  didn’t do anything with it. Instead of relying on the exitcode, I’ve created a Get-LogSummary script to give me an overview of what failed or succeeded. Now that I see how easy it can be, why not give it a try? 😉

I’m running Windows 10 so I went with the PowerShell 5.0 version

[Flags()] Enum RoboCopyExitCodes{
NoChange = 0
OKCopy = 1
ExtraFiles = 2
MismatchedFilesFolders = 4
FailedCopyAttempts = 8
FatalError = 16
Number = $_
ExitCodes = [RoboCopyExitCodes]$_


So let’s give it a run!

#Start RoboCopy
$source  = '.\c#'     #Choose a source
$target  = '.\temp'   #Choose a target

robocopy $source $target /MIR


The target isn’t empty. I’m using /MIR to make a point. Be careful when choosing your target!

So the first run gave an exitcode of 3



A second run gave an exitcode of 0


No change indeed!

As a first impression, could be useful… So there you have it! RoboCopy exitcodes the *ahem* PowerShell way! 😛

Hope it’s worth something to you



Get-RCLogSummary take II

Trying to figure out this regular expression thing…. I just can’t stand the idea of not getting it!

Regular Expression meme

Joakim Svendsen Get-FolderSize uses regular expression, so I did some investigating. Why reinvent the wheel eh? 😉 Turns out there’s a lot going on under the hood here. First I had to figure out how the regular expression worked so I isolated just one…

[Regex]$regex_Dirs = 'Dirs\s:\s+(?<TotalDirs>\d+)\s+(?<CopiedDirs>\d+)(?:\s+\d+){2}\s+(?<FailedDirs>\d+)\s+\d+'

Turns out you can give your result a label if a match is found. Running the regular expression with -match operator will populate the $Matches variable with found label/value, which allows you to retrieve that value later on.

if ($_ -match $regex_Dirs){
  $rcLog.TotalDirs = [int]$Matches['TotalDirs']
  $rcLog.CopiedDirs = [int]$Matches['CopiedDirs']
  $rcLog.FailedDirs = [int]$Matches['FailedDirs']

Quick side note: This script works with the /Bytes option. The plus side of using /Bytes is that you can do some fun stuff like calculating the size in GB or MB

TotalBytes  = $Matches['ByteCount']
TotalMBytes = ([int64] $Matches['ByteCount'] / 1MB).ToString('N')
TotalGBytes = ([int64] $Matches['ByteCount'] / 1GB).ToString('N')


There’s yet another option, using ConvertFrom-String, an excellent blog by Bartek Bielawski brought to my attention by Dexter Dhami. I’m working on that as well. That one will only work with PowerShell version 5 so there’s that. Actually, I saw this feature in ISESteroids first at the PowerShell Summit. What I did notice is when using ConvertFrom-String performance took quite a hit…

I have mixed feelings when it comes to ConvertFrom-String. At times he just doesn’t get it! I love the template idea but you need to be specific, otherwise some data might fall through the cracks…You never know for sure… Like all tools you should be skilled at several. Think of regular expression as driving a stick and ConvertFrom-String as an automatic. Here in the Netherlands you can get your driver license on an automatic, but then you’re not allowed to drive stick ever! But if you pass driving a stick you’re allowed to drive automatic. Best bet, go with stick!

Here’s the code using regular expression, with a lil’ help from my friends… 😉

#region Hash with the Robocopy Log properties
$rcLogProperties = [Ordered]@{
  rcLogFile = ''
  Source = ''
  Target = ''
  TotalDirs = ''
  CopiedDirs = ''
  FailedDirs = ''
  TotalFiles = ''
  CopiedFiles = ''
  FailedFiles = ''
  TotalBytes = ''
  CopiedBytes = ''
  FailedBytes = ''
  StartTime = ''
  EndTime = ''
  Speed = ''

#region Main
#Get Logfiles from folder
Get-ChildItem '.\temp\23-06-2015' -File |
ForEach-Object {
  #Get Lofile Header & Footer
  $arrSummary  = (Get-Content $_.FullName)[5..8] 
  $arrSummary += (Get-Content $_.FullName)[-10..-1]

  $rcLog = New-Object -TypeName psobject -Property $rcLogProperties
  $rcLog.rcLogFile = $_.Name

  Foreach($line in $arrSummary) {
    switch ($line){
      {$_ -like  '*Source :*'}
        $rcLog.Source = ($_ -replace '(\s\w+)(\s.)','').Trim()
      {$_ -like  '*Dest :*'}
        $rcLog.Target = ($_ -replace '(\s\w+)(\s.)','').Trim()
      {$_ -like  '*Dirs :*'}
        [regex]$regex_Dirs = 'Dirs\s:\s+(?<TotalDirs>\d+)\s+(?<CopiedDirs>\d+)(?:\s+\d+){2}\s+(?<FailedDirs>\d+)\s+\d+'
        if ($_ -match $regex_Dirs){
          $rcLog.TotalDirs = [int]$Matches['TotalDirs']
          $rcLog.CopiedDirs = [int]$Matches['CopiedDirs']
          $rcLog.FailedDirs = [int]$Matches['FailedDirs']
      {$_ -like  '*Files :*'}
        [regex]$regex_Files = 'Files\s:\s+(?<TotalFiles>\d+)\s+(?<CopiedFiles>\d+)(?:\s+\d+){2}\s+(?<FailedFiles>\d+)\s+\d+'
        if ($_ -match $regex_Files){
          $rcLog.TotalFiles = [int]$Matches['TotalFiles']
          $rcLog.CopiedFiles = [int]$Matches['CopiedFiles']
          $rcLog.FailedFiles = [int]$Matches['FailedFiles']
      {$_ -like  '*Bytes :*'}
        [regex]$regex_Bytes = 'Bytes\s:\s+(?<TotalBytes>\d+)\s+(?<CopiedBytes>\d+)(?:\s+\d+){2}\s+(?<FailedBytes>\d+)\s+\d+'
        if ($_ -match $regex_Bytes){
          $rcLog.TotalBytes  = $Matches['TotalBytes']
          $rcLog.CopiedBytes = $Matches['CopiedBytes']
          $rcLog.FailedBytes = $Matches['FailedBytes']
      {$_ -like  '*Ended :*'}
        [regex]$regex_End = 'Ended\s:\s+(?<EndTime>.+)'
        if ($_ -match $regex_End){
          $rcLog.EndTime = $Matches['EndTime']
      {$_ -like  '*Started :*'}
        [regex]$regex_Start = 'Started\s:\s+(?<StartTime>.+)'
        if ($_ -match $regex_Start){
          $rcLog.StartTime = $Matches['StartTime']
      {$_ -like  '*Speed :*'}
        [regex]$regex_Speed = 'Speed\s:\s+(?<Speed>.+\/min)'
        if ($_ -match $regex_Speed){
          $rcLog.Speed = $Matches['Speed']

Hope it’s worth something to you!



‘Sup PSHomies?

Got a lil’ somethin’ for ya… Get-RCLogSummary! As you know I’m a big fan of RoboCopy! I thought I’d share one of the perks of using RoboCopy: the LogFile.

Here’s a list of RoboCopy Logging options, courtesy of

   Logging options
                /L : List only - don’t copy, timestamp or delete any files.
               /NP : No Progress - don’t display % copied.
          /unicode : Display the status output as Unicode text.  ##
         /LOG:file : Output status to LOG file (overwrite existing log).
      /UNILOG:file : Output status to Unicode Log file (overwrite)
        /LOG+:file : Output status to LOG file (append to existing log).
     /UNILOG+:file : Output status to Unicode Log file (append)
               /TS : Include Source file Time Stamps in the output.
               /FP : Include Full Pathname of files in the output.
               /NS : No Size - don’t log file sizes.
               /NC : No Class - don’t log file classes.
              /NFL : No File List - don’t log file names.
              /NDL : No Directory List - don’t log directory names.
              /TEE : Output to console window, as well as the log file.
              /NJH : No Job Header.
              /NJS : No Job Summary.

My preference when it comes to logging is to have seperate logfiles instead of appending to one big file. The option /NP is a no brainer, displaying ‘%’ will give you an indication how long it took for that specific file/folder, but who wants that right?It will only increase your logfile size taking more time to parse it down the line. I recently used /NDL and I must say this will keep your logfile footprint small. I did include /FP to still have an idea where the file is being copied from. I’d go with /NDL in combination with /FP when doing a delta-sync. A delta-sync is a robocopy job that will copy the differences once a full-sync has taken place. If the file hasn’t changed robocopy will skip it. Only new and newer files will be copied… Ok enough background, let get scripting shall we? 😛

Function Get-RCLogSummary{

  $objLogSummary = @{
    rcLogFile = $LogFileName
    Speed = ''

  Foreach($line in $logSummary) {
    switch ($line){
      {$_ | select-string '   Source :'}
          $_= $_.ToString()
      {$_ | select-string '     Dest :'}
          $_= $_.ToString()
      {$_ | select-string '  Started :'}
          $_= $_.ToString()
      {$_ | select-string '    Dirs :'}
          $_= $_.ToString()
      {$_ | select-string '   Files :'}
          $_= $_.ToString()
      {$_ | select-string '   Bytes :'}
          $_= $_.ToString()
      {$_ | select-string '   Ended :'}
          $_= $_.ToString()
      {$_ | select-string '   Speed :'}
          $_= $_.ToString()
          $objLogSummary.Speed = $($_.Substring(11).Trim())
      {$_ | select-string '   Times :'}
          $_= $_.ToString()
          $objLogSummary.Add('Time Total',$($_.Substring(11,10).Trim()))

  #return $objLogSummary

#region:array with all LogSummary Object Properties
$arrRCProperties = @(
  'Time Total',

#region: Get all robocopy LogFiles in specified folder and get Summary
get-childitem '.\log\rc\home\22-06-2015' -File |
ForEach-Object {
  #region: Get File Header & Footer
  $arrSummary  = (Get-Content $_.FullName)[5..8] #Header
  $arrSummary += (Get-Content $_.FullName)[-11..-1] #Footer

  Get-RCLogSummary -LogFileName $_.Name -LogSummary $arrSummary
Select-Object $arrRCProperties |

First I’ll get a list of logFiles and retrieve the first 5-8 lines and the last 10 lines of each file for processing. The LogFileName & array Summary are then passed as parameters to Get-RCLogSummary. I did a select to get the parameters in a certain order. It was a toss up between using [Ordered] Hash or  defining a [PSCustomObject] beforehand. I figured you could minimize the Properties you want by tweaking the $arrRcProperties yourself. last but not least use Out-Gridview or Export-Csv to see the endresult.

I’m working on my pipeline skills, trust me my previous version was more ‘elaborate’, and by elaborate I mean over engineered…

So I guess you’ve noticed that regular expression is missing? Robocopy labels are fixed which is a good thing for me. I’m looking into it…


This regular expression isn’t as easy as it seems… This works, just don’t include /Bytes in your robocopy parameter list. In that case you’ll definitely need regular expression. Version 2.0 I guess…

Hope it’s worth something to you



SDDL gives more NTFS insight

I’ve been doing migrations, oh say for the past 10 years (Hmmm, that’s long if I do say so myself) Data Migrations can be complex depending what needs to be achieved. I remember using ScriptLogic to map drives depending on which subnet a user was on, that was way before DFS was available… Good times…

I’ve had my share of headaches when it comes to Data migrations. The biggest challenge is interoperability, when Target Resources keeps on using Source Resources until all Source Resources have been migrated. Sometimes it’s just not possible to migrate all Source Resources at once (what we affectionately call ‘big bang’). If data is being mutated by different departments/projects that aren’t migrated at the same time then interoperability is your only choice… Still tricky though…

Ok so here’s the scenario: Migrate Resources from one AD Forest to another (with a trust in place). I’ll take you through the Data part 🙂

The key component is to use SIDHistory. SIDHistory will help resolve whether you have access or not to a Source Resource. My favorite replication tool has to be robocopy! It wasn’t love at first sight, but once I figured out all the parameters, then there isn’t much you can’t accomplish with it!

For interoperability we usually redirect Target Resources to the Source. This way Data mutation can still be achieved without disturbing Production. In the mean time data is being synced to the Target Domain with ACLs intact! Why? We’ll get to that later… Or might as well get into it now… 🙂

Ok so ACL (Access Control List) is that list you get when you open up a file or folder security tab. The accounts are referred to as ACE (Access Control Entry). That’s where you’d grant/remove an account read/write/full/etc access to said file or folder. When using SIDHistory you’re token access will resolve correctly, but here’s where it gets tricky

I’ve copied Data with robocopy keeping security intact. When I opened a folder security tab I noticed the Target account name being displayed. That threw me off because I didn’t reacl the target resource yet.

Quick sidestep ReACL is a term I came across using Quest Active Directory Manager (now DELL). ReACL can be done by adding the Target Account (doubling the amount of ACEs) or doing a cleanup by first adding the Target account and removing the Source Account. You can also rollback if needed but that one is tricky, especially if SIDHistory has more than one entry.

But you wouldn’t know that by looking at the folder Security tab.

If you really want to find out who has access, SDDL will let you know. SDDL uses an object SID to grant or deny access. Thing is SDDL is hard to read hence the Security tab. So the first time I ReACLed a folder adding the Target Account I saw that the ACEs did double, but I only saw the Target Account. I expected to see SOURCE\ACCOUNT;TARGET\ACCOUNT instead I was seeing the TARGET\ACOUNT twice. Here’s where looking at SDDL will give you more insight… Suffice to say we’ll be doing this the PowerShell way… Oh come on! don’t act so surprised! 😛

So first let’s get the ACL of the folder you want to inspect (try this on your folder):

$acl = get-acl '\\\g$\GRP\DATA\DEPT-001-XYZ'

To find out who has access  type $acl.Access. This will give you a list of all ACEs in the ACL. This is the list you’d also see in Explorer security tab (advance mind you, I noticed that). Now for the fun part $acl.sddl… Tada!!!



Seems complicated, well yes it is, still it’s worth figuring out… Have a look at MSDN for more information.

The tell tale is the Domain SID, every Account begins with it. Looking at the Domain SID tells you who actually has access (or not) to said resource and which Domain that account belongs to.

The Domain SID for the current domain I’m inspecting is:
DomainSID : S-1-5-21-602145358-1453371165-789345543
You can get the Domain SID using Get-ADDomain cmdlet… 😉

I picked an ACE from the $acl.access list:

FileSystemRights : Modify, Synchronize
AccessControlType : Allow
IdentityReference : SOURCE\DEPT-001-XYZ-RXWR
IsInherited : False
InheritanceFlags : ContainerInherit, ObjectInherit
PropagationFlags : None

Let’s get some AD properties from this acount

Get-ADGroup -Identity DEPT-001-XYZ-RXWR -Server -Properties SID,SIDHistory
SamAccountName : DEPT-001-XYZ-RXWR
SID : S-1-5-21-602145358-1453371165-789345543-35829
SIDHistory : S-1-5-21-103234515-1370883554-928726630-4307

Here’s the sddl string once more:


This group has access using SIDHistory!!!

Ok now what? Well in an ideal situation the data would have been ReACLed using the current SID instead of the SIDHistory. The reason for that is to cleanup your SIDHistory to avoid tokenbloat. Here’s an excellent blog by the dirteam discussing the perils of tokenbloat.

This only scratched the surface of what you could investigate! There aren’t many tools (Free) that can help. Ashley Mcglone has an excellent series on the matter definitely worth reading.

I’m currently doing a Data migration (surprise!) so I’ll be adding more tips/tricks/gotchas as the Data migration progresses so stay tuned!

Hope this will steer you in the right direction when it comes to figuring out who has access…
The rabbit hole goes deep…