The lowdown on SIDHistory

Sup’ PSHomies,

SIDHistory is one of those Active Directory attributes you love to hate. When migrating from one domain to another, it let’s you retain access to resources in the Source Domain. This is a great way to transition, but in my experience it also makes for quick-shift migrations.

The first thing I do whenever we start a migration is have a look at SIDHistory. This will let me know quickly what we’re dealing with:

  • Has there been a previous migration? (I’ve seen objects in excess of 5 entries)
  • Did they clean up? (Obviously they didn’t or I wouldn’t see any entries)
  • Do I need to worry about Token-bloat?

Remember the blog I did about SDDL? Well SDDL deals with access based on SIDs. When a user logs on to the system, not only the new SID, but also the old SID is retrieved from the SIDHistory attribute and is added to the user’s access token and used to determine the user’s group memberships. The SIDs of the groups of which the user is a member through either the new SID or the old SID are then also added to the access token, together with any SIDHistory those groups might have.

This is also the reason tokenbloat can be an issue if it isn’t cleaned up after a migration.

So how do you find out about SIDHistory?

On the subject of removing SIDHistory

This is tricky. Having and keeping SIDHistory intact will keep many a pesky helpdesk calls at bay… But is it wise to keep it?

From a Data (Read NTFS) perspective, you’ll need to Re-Acl your data structure. If you’ve kept you NTFS ACLs (Access Control List) nice and tidy (Wait, gimme a second to catch my breath from laughing) then you’re golden! This has never been the case in all my migrations  so far. My advice when it comes to Re-Acl, is to recreate the data structure (empty) and assign the correct ACEs (Access Control Entry) to the ACLs. Maybe I need to explain what Re-acl a bit more…

Re-ACL is the process of translating SIDs on Resources. I first came across the term using Quest Migration tools. This gave me the option to:

  • add the target SID to a resource
  • replace source SID on said resource
  • remove source SID from resource if everything is working

Here are the things you need to consider for each option

Adding the targetSID to a Resource

This gives the AD Object  access without having to rely on SIDHistory. This means that once the target SID has been added you can safely clean up SIDHistory. A target SID can only be added if a valid source SID has been found. I’ve seen too many ACLs with unknown ACEs in migrations I did over the years. This does nothing to clean up those unknown ACEs. Adding a target SID will expand you ACLs, which can have an impact on processing time

Replacing the sourceSID on a Resource

This makes for a cleaner ACL. Again, this does nothing for unknown ACEs. Replacing adds the targetSID and removes the sourceSID in the same process. A bold move, reverting SIDHistory isn’t as easy a writing to other AD object attributes and for good reason.

Remove sourceSID from Resource once everything has been verified to be working

Most are quite content that everything is working and don’t bother with this. Again, if your structure is up to date, this shouldn’t be an issue. What I’ll usually hear is: “We’ll create a new structure later on and get that cleaned up…” This rarely happens…

To wrap up

SIDHistory is a great way to retain access to source Resources, just make cleanup a part of the migration (If possible).This will vastly improve tokensize and improve your security

Re-Acl only makes sense if you’re content with your current NTFS data structure. If not, then I’d suggest redefining your Data structure. It’s a chore but well worth it.

Hope it’s worth something to you…



Venturing into the world of PowerShell & DataTables

‘Sup PSHomies,

Been rethinking my Active Directory snapshot approach…

There are many ways to Rome, so which one to you choose? I’ve always been a fan of CSV and the Export-Csv cmdlet. Then I found out that Export-CliXML makes for a better experience when it comes to saving PS Objects (just don’t try figuring the tags out). Use the Import-CliXML cmdlet and you’ve got your PS Objects with Types back and in tact! Kevin Marguette has a great blog about this. While you’re at it, Jeff Hicks also has a great series on this topic, sharing is caring… 🙂

Gathering all you Active Directory data in one PS Object might not be the best approach when it comes to performance on a sizable AD. In my test lab, that wasn’t an issue. When I ran my script against a sizable AD, let’s just say I ran into some memory issues… 😉

My objective is to:

  • Gather AD data for offline reporting  purposes.
  • Keep data fragmentation to a minimum.

Let’s gather ADUser data for this exercise…

I want to collect the following data on AD users:

  • Disabled
  • Expired
  • NoExperationDate
  • Inactive
  • MustChangePassword
  • CannotChangepassword
  • All User & Properties

Approach #1

Pretty straightforward. Use the pipeline to keep resource usage to a minimum. When processing large collections you should always think pipeline!

Get-ADUser -Filter * -Properties * |
Export-Clixml .\export\dsa\dsADUsers-CliXML.xml -Encoding UTF8

Exporting directly using Export-CliXML will get the job done. I could do this for each AD User query:

  • Search-ADAccount -AccountDisabled
  • Search-ADAccount -AccountExpired
  • Get-ADUser -LDAPFilter ‘(|(accountExpires=0)(accountExpires=9223372036854775807))’
  • Search-ADAccount -AccountInactive
  • Get-ADUser -Filter {pwdLastSet -eq 0}
  • Get-ADUser -Filter * -Properties CannotChangePassword |Where-Object {$_.CannotChangePassword}
  • Get-ADUser -Filter * -Properties *

Nothing wrong with that, other than having a bunch of CliXML files. Collecting everything first isn’t an option performance wise.

Approach #2

Use a data Set/Table.

Full disclosure: This is my first attempt using data set/tables. I’m pretty sure that there’s a better approach and I’d love to here all about it!!!

I (re)found Chrissy LeMaire blog on basic .NET Datasets. Now why doesn’t that surprise me? 😛 Hehe…


Quick breakdown: First I created a Dataset (prefix ds just in case you were wondering…) and the data table. I wanted all available properties. I realize this might be more than you need so feel free to change that to your needs. To get the necessary column Names I’m using the Get-Member cmdlet with parameter MemberType Property. A quick select and expand and I have column names, beats hard coding column names…

Ok here’s where it gets a bit tricky, So I’m processing each object as it goes through the pipeline, but I do have a datatable object… Like I said first attempt…

Once the data table is ready just add it to the dataset! A dataset can hold multiple datatables which helps in solving my data gathering fragmentation.

Fun fact

Did you know that you can write/export your datatable to XML? Unlike CliXML output this one is legible…

dataTable ADuser

To write the dataTable to XML:


Same command applies for the dataSet 🙂 Added bonus is the file size, much smaller than CliXML, but then again not as rich…

dataTable ADuser size

Take away

I love the fact that there’s more than one way to work with data. I guess it comes down to preference. I was pleasantly surprised by the datatable XML formatting, nice and clean! If  export file size is an issue, then datasets can help, the trade off being you’ll loose rich data of the PS objects. The *CliXML cmdlets are sufficient for my needs, if I’m honest with myself, still glad I looked into data sets/tables…

Now If you have a full fledged SQL environment you can take advantage of, then, Go for it!!! Just ask Rob Sewell aka sqldbawithbeard  or Chrissy LeMaire, our PowerShell SQL experts to point you in the right direction!

The more you know! 😉

Hope it’s worth something to you…



ACLs Folder class

‘Sup PSHomies,

I recently had to make a quick Backup & Restore of ACLs three levels deep. Nothing fancy, just two functions, but that got me thinking…

Why not make a class of this?

And so I did! Here’s the framework for the class:


Here’s a list of the methods:

  • Backup. Backup SDDL from the property $Folder of the class
  • Restore. Restore SDDL to the property $Folder of the class
  • Clone. Clone will take a target Folder and clone the SDDL of $Folder to it
  • ConvertSDDLToAccess. This will enumerate what the SDDL stands for

Default Constructor


The default constructor will evaluate the folder used to instantiate the object. If successful, the SDDL, Owner and Access is retrieved using  the Backup() method. All actions are registered for future reference.

Instantiating is pretty straightforward:




This will retrieve the SDDL for the folder and enumerate the Access.



Restore is a bit tricky. For one you need to make sure it isn’t empty. Set-Acl has no problem setting an empty SDDL, blowing everything open (worst case scenario, and that’s why you should test this in a lab first!). The other challenge is having a valid SDDL string. You can change the SDDL string if you want to something gibberish, hence the typecast as a precaution.



The same goes for cloning. In this case we need to test the target path. Alternatively, you could  also change the Folder to a new path… It works, you’d just have misleading ActionHistory entries… I wonder if there’s a way to make this read-only,  just thinking out loud here… (note to self)


This is just a lil’ something extra. Like I said in a previous blog SDDL really gives more information. For one, the SID will let you know which domain this object belongs to. One thing I ran into with ReACL is that SIDHistory will resolve to the current NTAccount. This had me puzzled for a while until I saw that the SIDs in SDDL where different.

Here’s what the ouput looks like:


Now for those of you that are wondering just what is this AccessMask, wonder no more! 🙂

Remember the RoboCopy ExitCodes blog I did a while back? Well it’s the same principal 🙂 This is why classes & everything related should be on your radar…


Here’s how this works…

Say I wanted to evaluate the AccessMask of a SDDL entry


Here I have the SID & the NTAccount. This is the builtin administrators account but it also works for Domain accounts.classACLConvertSDDLToAccess

There’s a private function that will translate the SID accordingly.

To see what the account can actually do we can enumerate the AccessMask


This is what we’d see using the advanced Security GUI tab of a folder.AdvancedPermissions

Not bad… Not bad at all…

I can’t state this enough, SDDL is NOT to be be trifled with. Yes you need admin rights and please try this is a testlab first.  SDDL is very potent, if used with caution, it could do a whole bit of good!

So finally, here’s the code…

Hope it’s worth something to you…



AD Operation Validation class

‘Sup PSHomies,

I’m like a dog with a bone… 😛

2016 was all about operation validation for me. I did a series on Active Directory snapshot, report and validation that was well received by the community! Classes will definitely make the user experience more pleasant! I decided to refactor the code to a class 😉 Got a lot of ground to cover so let’s dive in!

Here’s a screenshot of the ADInfrastructure (was what popped in my mind at the time) class properties and methods:


The focus will be on Active Directory’s forest, domain, sites, sitelinks, subnets and domaincontrollers.

These are the methods I’ve worked out so far (work in progress):

GetCurrentConfig() will populate the necessary properties using the right activedirectory cmdletsgetcurrentconfig

ImportADSnapshot() will import a saved XML file to ADSnapshot property.importadsnapshot

ExportADSnapshot() saves the class object as a XML file. This will generate a new XML file using the current $exportDate valueexportadsnapshot

RunValidation() deserves it own section… 😉


This is where the the operation validation will take place. This was a bit of a challenge getting the method right, but I think it worked out just fine… I should explain…

In my first attempt I ran the validation directly from the method. You can invoke the Describe block just like a function. That wasn’t the challenge, have a look at the It blockitblock

The test is hard coded to use $this as source and $this.ADSnapshot as target. I blogged about some possible validation gotchas a while back. To remedy this, I decided to use a scriptblock. You can also provide parameters to a scriptblock . In this case I provided ($Source, $Target) as parameters. This will make interchanging  input easier of which I’ll explain the advantages later on.


Ok so the scriptblock was a good idea. One of the things I wanted to do was save the validation results. That’s where I ran into something interesting. The input has to be a *.tests.ps1 file(s). I tried using the scriptblock as input but that didn’t work. I visited the github page to see if scriptblock is supported as a feature, it isn’t. In order to save the results I would first have to save the test to a file. The scriptblock made that part a lot easier. As a workaround, this isn’t an issue.The file is generate every time RunValidation is executed, an inconvenience at most.  A scriptblock feature  would make for a cleaner approach.

Quick side step: There’s a poll on twitter for anyone interested in casting a vote 🙂 Only 5 more days left…


The test results are saved in ValidationResults. That’s RunValidation in a nutshell.

It’s quite a bit of code, so I’ll post that at the end of the blog. Here’s what you can expect if you try it out. First up, a simple verification of the current configuration against a saved snapshot

#region Verify Current Configuration against a snapshot
$snapShotDate = '12012017'
$ADInfra = [ADInfrastructure]::New()
$ADInfra.snapshotDate = $snapShotDate
$ADInfra.ActionHistory | Select-Object -Property TimeGenerated,Tags,MessageData

Before you get startetd, you need to instantiate the class. GetCurrentConfig() will save the information to the properties. ExportADSnapshot() will create a ADSnapshot-($exportDate).xml file. ImportADSnapshot will import any existing snapshot file of a given $snapshotDate formatted as ‘ddMMyyyy’.

Because I’m verifying the current configuration with a snapshot without any changes all the tests will pass.


No surprises.

For the next example, I wanted to validate against a manual configuration. This is where the scriptblock really made a difference. I added a non-existent DC to  $ADVerifyConfig for testing purposes.


The class is instantiated differently this time. The values are added externally. If you run GetCurrentConfig() at any point, it will rewrite the default values. Here are the results of the snapshot vs manual source first.



The snapshot only has one DomainController, we never get to the second DomainController. Now if we switch parameters from positions…



Ah! DC-DSC-02 doesn’t exist in the snapshot so it will fail! There are always two sides to consider. RunValidation() makes it easier to test and verify both sides…

Bonus round


I recently discovered the Information stream in PowerShell v5. I decided to make use of Write-Information to log activities as I go along. This makes for easier troubleshooting of actions and/or sequences of methods being executed, couldn’t hurt… 😉



Saving the validation test result enables you to process the results in different ways.

For starters you can use Format-Pester by Erwan Quélin to generate documentation of the results. Now because it’s an object you can just as easily run a query:

$ADVerifyInfra.validationResults.Results.TestResult.Where{$_.Passed -eq $false}

You can even send a high-level overview to Slack (It’s on my to-do list).

Whew! I think I’ve covered all the essentials… Ok as promised the code:

Classes will definitely enhance your end-user’s experience…

Hope it’s worth something to you…



RoboCopy class

‘Sup PSHomies,

It all started a year ago… Always wanting to learn anything PowerShell related, classes caught my eye ever since it was introduced in v5.  I wanted to try my hand at classes with a real life application… So I got on twitter for some tips…



Doug was kind enough to reach out and point me in the right direction, for which I owe him a great debt! Appreciate it Doug!!!

Like I said, I wanted to try my hand at classes with a real life application… If you’ve read my blogs then you’ll know that I’m a fan of robocopy, seriously, huge fan! . Did I mention how awesome robocopy is? 😛 I think I found my real life application 😉

When I started out with my Robocopy class, it was just about logging initially, but it could be so much more! Classes are native to v5. Now that v5 is mainstream I decided to finish the class. Richard Siddaway’s article  was just the spark I needed to get me going (again)!

Here’s what the Robocopy class looks like:


Here a quick rundown on the properties:

The source/destination properties of the class are self explanatory (if you’re familiar with robocopy). The property logdir and JobID will be used to define a valid logfile name (with extension .log). Robocopy has quite a bit of options. I wanted to keep it as generic as possible. The property $this.Options is still a work in progress. The property $this.WildCards  is where you’ll define what will be filtered.  I’ll get back to rcCMDs and rcExitCodes later on…

These are the methods I came up with (so far, still a work in progress)

  • Mirror(). Mirrors $this.Source to $this.Destination with some default options
  • Move(). Moves this.Source to $this.Destination with some default options
  • RollBack(). Rollback $this.Destination to $this.Source with some default options
  • Sync(). Sync will synchronize the delta’s from $this.Source to $this.Destination using any additional $this.Options defined (at least that’s the idea). I’ve added a few options by default, mostly to exclude files and folders, think recycle.bin “System Volume Information” and the likes.
  • VerifyPaths(). This let’s you know if the $this.Source, $this.Destination and $this.LogDir are valid.
  • GetlistSource(). This will list the content of the $this.Source
  • GetListDestionation(). This will list the content of $this.Destination
  • GetLogSummary. This will return a summary of the log file (Hehe). The method is static so that you don’t have to instantiate the class in order to use it. (Thanks again Doug!)

The two methods: StartRCProcess and ExecuteRCCMD are actually helper methods. I just haven’t figured out how that works in classes. Ideally I’d like to have them hidden or as a function if that even makes sense. So here’s where they come in. At first I just executed robocopy with the necessary arguments. If you’re not interested in the exitcode then using ExecuteRCCMD is what you need. I wrote a blog about enumerating RoboCopy Exitcodes. Using $LastExitCode isn’t going to cut it if you decide to run robocopy jobs parallel. That’s where StartRCProcess comes in.Using Start-Process comes with an overhead of say 20 MB, which could add up in the long run. You do need to wait until the process has finished to retrieve the exitcode. If you really need the exitcode then StartRCProcess is what you need. The property $this.rcExitCodes will only be populated if StartRCProcess is used. Both will populate the $this.rcCMDs property.

Ok I think I’ve covered the basics, time to show some code! 😉

Here what’s happening in the List methods:


GetListSource() is using $this.StartRCProcess to generate a list of $this.Source using some default option. While writing I noticed that I forgot to add the wildcards to the parameter. All I had to do was add it!. I added it at the beginning so it lines up accordingly… Robocopy is fickle like that…  GetListDestination does the same only it uses ExecuteRCCMD instead.

Here’s what’s going on in StartRCProcess and ExcuteRCCMD


Both StartRCProcess and ExcuteRCCMD will save the robocopy command using Write-Information. I’m loving Write-Information more and more! StartRCProcess saves the exitcode with some extra information. Here’s where the robocopy exitcode enumeration came in handy! ExecuteRCCMD will run robocopy with the specified arguments. Truth be told I’m more partial to the ExecuteRCCMD method. I added the StartRCProcess more for demo purposes and finally getting to use my Robocopy exitcode enumeration!

For Mirror(),Move() and RollBack(), I omitted the Wildcards. These methods all or nothing in my opinion. If omitted, . will be the default.

Sync() had me going for a while. I still have some issues with Options. For now Sync() uses some default switches. Like I said work in progress…

Quite a bit of code, so does it work? Here’s some code to play with. be sure to edit the source,destination and logdir to your liking. Just remember that robocopy is unforgiving so make sure not to use it of production folders!

#region Main
$rc = [RoboCopy]::New('C:\scripts\move','C:\temp\move','rc-0001','c:\scripts\log',@('*.*'))

#Run RoboCopy methods

#Get RoboCopy LogFile Summary

#Get RoboCopy executed CMDs

First I instantiate the class with arguments. I then run the methods Sync(),GetListSource() and GetListDestination(). Up next is retrieve the LogSummaries from the methods. Here’s a screenshot of the Sync LogSummary


I did a select of $rc.rcCMDs to give you an idea what is being stored


Only want ListDES?

$rc.rcCMDs |
Where-Object{$_.Tags -contains 'ListDes'} |
Select-Object -Property Time*,Tag*,Mess*


The information stream is quite handy! The tags will definitely come in handy when you need to filter on action verb or job ID.

The methods GetListSource() & Mirror() both make use of StartRCProcess(), so let’s see what $rc.rcExitcode has shall we?



This is by far my longest blog, if you made this far then… Congratulations! There’s still so much to discover when it comes to classes.

Classes definitely  makes your code look and feel more like a developer 😉 . I feel more comfortable giving a colleague this class than a bunch of scripts. In Richard’s article he’s using both classes and modules. There are sure to be some gotcha’s… Do you go all in with classes or only partial?

I’m hoping that the community can shed some light on the subject. I’d love to hear from you guys on how to improve on this… Let’s make this year, a year of PowerShell Classes! 😛

Hope it’s worth something to you…




A simple logger using PowerShell Class

‘Sup PSHomies,

At our last DuPSUG meeting, Jaap Brassers did a short Demo on the latest addition to PowerShell Streams, Information. How did I miss this?

So what’s so great about the new Information stream? The Information stream solves the Write-Host issue of not sending output to a stream like: error;verbose;warning etc. As such we were told that Write-Host is evil and every time you use it a puppy dies… Who wants that on their conscious right?

Well with the Information stream Write-Host is allowed… no more puppy genocide!

Jaap showed us what the stream output looks like and how to write to the information stream. Here’s what the output looks like


Here are a few properties we can play with: TimeGenerated,Source,User,Computer,Tags and MessageData. You get all of this by writing/redirecting to the information stream. The TimeGenerated property caught my eye, imagine creating a timeline of sorts. While googling (don’t act like it’s just me…) I came across a great article by Jeff Hicks on the subject (Seriously, how did I miss this?) In it he also talked about generating a timeline view… 😉

After I had some time to think about how to make use of this, it dawned on me… You know what else is native to PowerShell v5? Classes! When Classes were introduced in v5, they were another way to write DSC Resources. I’ve always thought that there’s more to classes than meets the eye.

If you’re interested in classes, Richard Siddaway has a great article taking you through each step, worth trying out.

Jaap’s demo gave me the idea to try my hand at a simple logger using classes. Here’s what I came up with…

Here’s a quick rundown:


The idea is to gather output from the Information stream. Constructors are a way of initiating an object for use (Shout out to Doug Finke for breaking this down for me!)


I “borrowed” Jeff Hicks idea of validating the preference first. This way you can toggle between showing Write-Information and/or Write-Host data. You can use Write-Host for a more colorful experience. Did I already mention that Write-Host is cool again? :-P.  I also added the possibility to show data in the console if you choose to, a verbose action of sorts…


I also added two methods of exporting: text and xml. That’s where LogDirectory and FileBaseName come in. For the text export, I selected TimeGenerated and MessageData for a timeline view (I know I know Jeff Hicks did it first :-P). Export to xml saves InfoMessages in raw format.

Here’s some code to play with.

function Get-PowerShellProcess{

   Write-Verbose "InformationPreference = $InformationPreference"
   $Logger.Preference = $InformationPreference
   $Logger.WriteInformation("Starting $($MyInvocation.MyCommand)")

   Get-Process -Name *PowerShell*
   $Logger.WriteInformation("Finishing $($MyInvocation.MyCommand)")

#region Main
#Initiate variable
$Logger = [WriteLog]::New('psProcess','C:\scripts\temp')

#Run with InformationAction. Information will be saved and not shown
Get-PowerShellProcess -InformationAction Continue -Verbose

#Run without InformationAction with Console set to $true
$Logger.Console = $true
Get-PowerShellProcess -Verbose

#Run without InformationAction with Console set to $false
$Logger.Console = $false
Get-PowerShellProcess -Verbose

Here are some screenshots of the output

I initiated the object a FileBaseName value ‘psProcess’ and LogDirectory ‘C:\scripts\temp’. Default Preference is ‘SilentContinue’ and no ouput to the console


First up run with action set to continue


$Logger.InfoMessages has redirected data.


When console is set to true we’ll see the Information message without explicitly setting InformationAction to ‘continue’. Using InformationAction will save the Information data to InfoMessages property.


Setting the console preference back to $false stops displaying output to the console


I hope that you were able to follow it all. 2016 was all about Operation Validation for me. I think classes are going to be mainstream in 2017! PowerShell + Classes will definitely give you an edge and not to mention make you feel more like a developer! 😉

The line between Dev and Ops is being blurred…

Hope it’s worth something to you…



Verify GroupMembership with Pester

‘Sup PSHomies,

Here’s another advantage of adding members with a different approach, Pester validation!

This makes for an easy way to process validation of  each member of a group.

Quick update: I’ve added  some extra code (at the end of the previous blog code) to export added- and revoked members.

#region Export for futher processing
$GroupMembers =@{
  Groups  = $Header
  Added   = $addADGroupMembers
  Revoked = $delADGroupMembers
$GroupMembers |
Export-Clixml .\export\dsa\ADGroupMembers-$exportDate.xml -Encoding UTF8

Quick rundown, first we’ll import the saved object and used that to get a snapshot of the current group members. Then it’s time to vaildate who has been added or revoked.


In this case I wanted to generate different Describe blocks. This makes for a better distribution in the HTML report.


Here’s the code to generate the HTML report using reportunit.exe

$exportDate = Get-Date -Format ddMMyyyy

#region Main
$pesterGroupMembers = Invoke-Pester .\ps1\dsa\ADGroupMembers*  -OutputFile .\export\dsa\ADGroupMembers.NUnit.xml -OutputFormat NUnitXml -PassThru

#run reportunit against ADgroupMembers.NUnit.xml and display result in browser
& .\tools\ReportUnit\reportunit.exe .\export\dsa\ADGroupMembers.NUnit.xml
Invoke-Item .\export\dsa\ADGroupMembers.NUnit.html

#Export Pester results to xml
$pesterGroupMembers | Export-Clixml .\export\dsa\PesterResults-GroupMembers-$($exportDate).xml -Encoding UTF8

Making sure a user is a member can be tricky at times especially when the members list is a few hundred.

As always, snapshots are your friend! When I exported the groups the first time I did it without validating if they existed. I recently ran into a situation where AD Objects were being deleted and recreated using the same SamAccountName! So having a little more information than just the SamAccountName can help when troubleshooting now and in the future.

When my project manager asked for logs and I handed him the HTML generated report of the group members… You should have seen the glee on his face!

So there you have it, verfying group membership using Pester!

Hope it’s worth something to you…