Graph, the endpoint to end all endpoints

‘Sup PSHomies,

My first introduction to Graph was at the Microsoft Technical Summit here in the Netherlands at the RAI march 23,24 this year. It was actually the last session of the day. The intended audience was for developers… I thought to myself, why not? I might pick up a thing or two… It blew my mind!

Then at the PSConfEU at Hannover, Jeffrey Snover’s Keynote ‘State of Union’ really made me reassess my career, so much so that I left my previous employment of 23 years for a much smaller company, Methos! Methos is owned by Jeff Wouters aka “The scripting dutchman”. It’s like Jeffrey Snover said in his keynote: “With transformative change, you need to change the way you think about things.” Graph is one of those transformative thing… Here’s how I got started.

So Jeff came up with an idea:

Jeff : Say Urv, I got something I’d like you to look into…

Me: Sure, what’s up?

Jeff: Would it be possible to get a hierarchy of the entire company from Azure Active Directory?

Me: I don’t see why not?

Jeff: Once you have the data the next step is to visualize it!

Me: Like an organizational chart?

Jeff: Uhuh! (Grinning) Here’s a tip, look into Graph…

I should have known… 🙂 Now at Methos we don’t have an on-premise Active Directory so Azure AD it is. Now as part of my “transformative change” I asked for a Macbook pro. I thought: “Hey if you’re gonna go through the change (wait that doesn’t sound right, you know what I mean) might as well go big or go home! Jeff didn’t disagree, he only grinned and let out a subtle evil laugh… I got a sneaky suspicion that I’m in for a surprise… My other Transformative change: PowerShell Core!

I installed Azure CLI 2.0 as a first attempt. To get Azure AD users:

az ad user list

This gave me a JSON list with few attributes. Maybe I need to add a property list or something. Let do a quick help, Hmmm… Ok… Next approach.

Let’s install AADPreview Core. Say, Get-AzureADUser doesn’t have a -Properties attribute… Huh? So that’s why Jeff was grinning about… Well played Jeff…

So here’s the deal, I’m using PowerShell core, not all modules are applicable. The good thing is that I could login to Graph Explorer.  Graph explorer gives you the opportunity to explore the endpoint and browse around.

“Hey look at that, there’s a get manager query! Maybe there’s a way to get all the data and filter out what I need (Like I always do with PSCustomObject)”

I saw one of the examples use a $select option so I decided to give it a try.

$select=displayName, GivenName, surName, department, officeLocation, jobTitle, userPrincipalName, id, manager

I decided to give it a try. Ok that narrowed things down a bit still no manager in sight! Turns out manager is a special kind of property, a Navigation Property. You can extract it from the MetaData

$graphMetadata = [XML](Invoke-WebRequest -Uri 'https://graph.microsoft.com/v1.0/$metadata').Content
$userMetaData = $graphMetadata.Edmx.DataServices.Schema.EntityType.Where{ $_.Name -eq 'User' }
$userMetaData

Graph-NavigationProperty

Ah, that’s why manager is omitted, it isn’t part of the collection! My next challenge was to retrieve the JSON payload. That’s where I ran into the accessToken dilemma.

 

AccessToken

From the online resources I inspected, authentication is the key to access (see what I did there?). I found Mark Kraus’s module PSMSGraph quite useful to follow his trail of thoughts. It didn’t work for me because I’m using PowerShell core. Now it’s time to read the document! When all else fails RTFM!

I ran into some snags with my application registration, so my next step was to reach out to the Graph team! They are on twitter just no tweets… yet! That got me in contact with Dmitry Pimenov, the PM of Microsoft Graph! I explained him my dilemma and he showed me an easy way to get the accesstoken for proto-type purposes. Finally! I can get some data to play with!

My first attempt was straight forward: get the user, get the manger for each user and then create a PSCustomObject. This isn’t optimal ofcourse. So I sent Dmitry a DM asking what is the optimal way of retrieving manager in an ODATA query. That’s when he explained that the Beta version supports the $expand option in the ODATA query.

Quick side step: Microsoft Graph supports ODATA queries, not all queries are supported yet, it’s a work in progress. ODATA is definitely something to look into… Here’s a link to get you started… Transformative change… 😉

Alright, so the /Beta returned  quite a bit more than I bargained for, but at least I didn’t have to do multiple queries. That’s when I decided to trying something new… JMESPath Query!

JMESPath

While using az I noticed a link to JMESpath query, what’s that?  JMESPath is a query language for JSON. With it, you can extract and transform elements from a JSON document… Sweet! Kinda like what we do with PSCustomObject, just on JSON. Here’s where having a Mac wasn’t a liability after all… I even found a vscode extension for jmespath queries! Many ways to Rome… Transformative change… (Jeffrey Snover was right, you need to rethink the way you think about things)

Using this query on the JSON document gave me the following results:

value[].{DisplayName: displayName, GivenName: givenName, SurName: surname, Department: department, JobTitle: jobTitle, Manager:manager.displayName, Location:officeLocation}

JSOn-ManagerResults

Alright! Prototyping mission accomplished! What I thought was straight forward had quite a few curves, but I wouldn’t have it any other way!

Take away:

Learn to love Graph! Be open to learning new and interesting way to accomplish tasks in a new way. Transformative change  won’t be easy, but it’ll be worth it!!!

Hope it’s worth something to you…

Ttyl,

Urv

Advertisements

PSDay.uk Slides & Code

I had the pleasure of presenting at PSDay.uk recently. PSDay uk was sponsored by Ticketmaster and G-Research in London. I presented the following topic:

PowerShell Classes for Ops transitioning to Dev

I’ve been fascinated by PowerShell classes since its introduction in v5. When Rob Sewell ask me to do a session at the PSDay.uk, I didn’t have to think too long about the topic 🙂

I’ll be honest that the first 5 minutes was kinda nerve wrecking. I wasn’t nervous at first until I stood there and had some kind of outer-body experience…

Holy crap this is really happening!!! Calm down Urv… Whooza… Get to the code…

Once I got settled in, my nerves calmed down…

The code is bases on an interaction I had with Jaap Brasser (The legend himself 😉 ) on his blog a while back. At first I thought I didn’t have enough code but it all timed-out perfectly in the end.

Here’s the link to the code & presentation.

Shout out to Doug Finke for taking the time to break down classes for me. Doug is a true Developer and an great guy! I truly believe that the right people are placed in our paths when we need them most. Yet only if we remain open-minded and willing to learn can these mentors help us reach new levels of development. I can’t thank you enough Doug, your contribution to the community is plain awesome! You make me want to be a better coder!

Thank you PSDay.uk for this great experience! Til’ next time? 😛

Ttyl,

Urv

 

Protecting output content from prying eyes

Sup PSHomies,

Security is on everyone’s mind these days. It’s no wonder, with GDPR  going in effect as of may 2018.  My quick take on GDPR? It’s just Data Protection & ILM (Information Life-cycle Management) with some serious penalty consequences.  That archiving solution isn’t looking that expensive anymore eh? Having a RBAC model in place makes a whole lot of sense right about now huh? But I digress…

I get asked a lot to create reports. GDPR made me stop and reflect on how well (or maybe not so well) I’m handling/sharing these reports. Let’s just say I didn’t give myself a high score…

So my next challenge is about security, how do I keep prying eyes off my data? How do I make sure only the intended audience has access to the data?

First thing that came to mind was encryption. Come to find out that PowerShell v5 has some new cmdlets, Protect/Unprotect-CMSMessage,for this very purpose! Keith Hill has an excellent blog about getting started with them (seriously how did I miss this???). Oh and be sure to read Dave Wyatt’s comment.

Another must read blog is that of Cyber Defense. I tried the code and was able  to recreate the limitations, errors and performance issues. I also tried exporting to XML using Export-CliXML, but that gave me some issues with UTF8 encoding.  I was able to protect a 500 MB file, but unable to unprotect it. Found yet another great tip on Compressing files using the 7zip Module (Could it get any better???) The cmdlets are great for encrypting simple text.

Best route for protecting your data is to encrypt your password and use that password to protect your data as a zip file. Here’s what I came up with…

Let’s break down the function. First you’re going to need the public key of the intended party. Keith’s blog covers how to do that. The function will generate a 100 random char password, encrypt it and save it to specified file. Last but not least it will also return said Password as a Credential object. This will be used later on to password protect the zip file.

This will at least protect you data from prying eyes. Like Dave said in his comment:

“Anyone with admin access to a computer, or physical access to an unencrypted hard drive, can steal those private keys in most cases.”

It’s not fool-proof but it’s a start…

Take away

Make sure you’re not the weakest link when it comes to security and protecting/sharing data. Make sure you understand your company’s GDPR compliance policies. I’m seeing quite a few Security officer job offers on LinkedIn if that’s your thing… 😉

One more blog I can definitely recommend reading is David das Neves blog on Powershell Security at Enterprise customers . Granted it’s quite the read but well worth it!

Ok, there’s just one more blog you should also read, Don Jones’ blog on stop using self-signed certificates

It’s going to be a challenge for all of us, but one well worth it…

Hope it’s worth something to you…

Ttyl,

Urv

The lowdown on SIDHistory

Sup’ PSHomies,

SIDHistory is one of those Active Directory attributes you love to hate. When migrating from one domain to another, it let’s you retain access to resources in the Source Domain. This is a great way to transition, but in my experience it also makes for quick-shift migrations.

The first thing I do whenever we start a migration is have a look at SIDHistory. This will let me know quickly what we’re dealing with:

  • Has there been a previous migration? (I’ve seen objects in excess of 5 entries)
  • Did they clean up? (Obviously they didn’t or I wouldn’t see any entries)
  • Do I need to worry about Token-bloat?

Remember the blog I did about SDDL? Well SDDL deals with access based on SIDs. When a user logs on to the system, not only the new SID, but also the old SID is retrieved from the SIDHistory attribute and is added to the user’s access token and used to determine the user’s group memberships. The SIDs of the groups of which the user is a member through either the new SID or the old SID are then also added to the access token, together with any SIDHistory those groups might have.

This is also the reason tokenbloat can be an issue if it isn’t cleaned up after a migration.

So how do you find out about SIDHistory?

On the subject of removing SIDHistory

This is tricky. Having and keeping SIDHistory intact will keep many a pesky helpdesk calls at bay… But is it wise to keep it?

From a Data (Read NTFS) perspective, you’ll need to Re-Acl your data structure. If you’ve kept you NTFS ACLs (Access Control List) nice and tidy (Wait, gimme a second to catch my breath from laughing) then you’re golden! This has never been the case in all my migrations  so far. My advice when it comes to Re-Acl, is to recreate the data structure (empty) and assign the correct ACEs (Access Control Entry) to the ACLs. Maybe I need to explain what Re-acl a bit more…

Re-ACL is the process of translating SIDs on Resources. I first came across the term using Quest Migration tools. This gave me the option to:

  • add the target SID to a resource
  • replace source SID on said resource
  • remove source SID from resource if everything is working

Here are the things you need to consider for each option

Adding the targetSID to a Resource

This gives the AD Object  access without having to rely on SIDHistory. This means that once the target SID has been added you can safely clean up SIDHistory. A target SID can only be added if a valid source SID has been found. I’ve seen too many ACLs with unknown ACEs in migrations I did over the years. This does nothing to clean up those unknown ACEs. Adding a target SID will expand you ACLs, which can have an impact on processing time

Replacing the sourceSID on a Resource

This makes for a cleaner ACL. Again, this does nothing for unknown ACEs. Replacing adds the targetSID and removes the sourceSID in the same process. A bold move, reverting SIDHistory isn’t as easy a writing to other AD object attributes and for good reason.

Remove sourceSID from Resource once everything has been verified to be working

Most are quite content that everything is working and don’t bother with this. Again, if your structure is up to date, this shouldn’t be an issue. What I’ll usually hear is: “We’ll create a new structure later on and get that cleaned up…” This rarely happens…

To wrap up

SIDHistory is a great way to retain access to source Resources, just make cleanup a part of the migration (If possible).This will vastly improve tokensize and improve your security

Re-Acl only makes sense if you’re content with your current NTFS data structure. If not, then I’d suggest redefining your Data structure. It’s a chore but well worth it.

Hope it’s worth something to you…

Ttyl,

Urv

Venturing into the world of PowerShell & DataTables

‘Sup PSHomies,

Been rethinking my Active Directory snapshot approach…

There are many ways to Rome, so which one to you choose? I’ve always been a fan of CSV and the Export-Csv cmdlet. Then I found out that Export-CliXML makes for a better experience when it comes to saving PS Objects (just don’t try figuring the tags out). Use the Import-CliXML cmdlet and you’ve got your PS Objects with Types back and in tact! Kevin Marguette has a great blog about this. While you’re at it, Jeff Hicks also has a great series on this topic, sharing is caring… 🙂

Gathering all you Active Directory data in one PS Object might not be the best approach when it comes to performance on a sizable AD. In my test lab, that wasn’t an issue. When I ran my script against a sizable AD, let’s just say I ran into some memory issues… 😉

My objective is to:

  • Gather AD data for offline reporting  purposes.
  • Keep data fragmentation to a minimum.

Let’s gather ADUser data for this exercise…

I want to collect the following data on AD users:

  • Disabled
  • Expired
  • NoExperationDate
  • Inactive
  • MustChangePassword
  • CannotChangepassword
  • All User & Properties

Approach #1

Pretty straightforward. Use the pipeline to keep resource usage to a minimum. When processing large collections you should always think pipeline!

Get-ADUser -Filter * -Properties * |
Export-Clixml .\export\dsa\dsADUsers-CliXML.xml -Encoding UTF8

Exporting directly using Export-CliXML will get the job done. I could do this for each AD User query:

  • Search-ADAccount -AccountDisabled
  • Search-ADAccount -AccountExpired
  • Get-ADUser -LDAPFilter ‘(|(accountExpires=0)(accountExpires=9223372036854775807))’
  • Search-ADAccount -AccountInactive
  • Get-ADUser -Filter {pwdLastSet -eq 0}
  • Get-ADUser -Filter * -Properties CannotChangePassword |Where-Object {$_.CannotChangePassword}
  • Get-ADUser -Filter * -Properties *

Nothing wrong with that, other than having a bunch of CliXML files. Collecting everything first isn’t an option performance wise.

Approach #2

Use a data Set/Table.

Full disclosure: This is my first attempt using data set/tables. I’m pretty sure that there’s a better approach and I’d love to here all about it!!!

I (re)found Chrissy LeMaire blog on basic .NET Datasets. Now why doesn’t that surprise me? 😛 Hehe…

 

Quick breakdown: First I created a Dataset (prefix ds just in case you were wondering…) and the data table. I wanted all available properties. I realize this might be more than you need so feel free to change that to your needs. To get the necessary column Names I’m using the Get-Member cmdlet with parameter MemberType Property. A quick select and expand and I have column names, beats hard coding column names…

Ok here’s where it gets a bit tricky, So I’m processing each object as it goes through the pipeline, but I do have a datatable object… Like I said first attempt…

Once the data table is ready just add it to the dataset! A dataset can hold multiple datatables which helps in solving my data gathering fragmentation.

Fun fact

Did you know that you can write/export your datatable to XML? Unlike CliXML output this one is legible…

dataTable ADuser

To write the dataTable to XML:

$dtADUsers.WriteXml('c:\scripts\export\dsa\dtADUsers.xml')

Same command applies for the dataSet 🙂 Added bonus is the file size, much smaller than CliXML, but then again not as rich…

dataTable ADuser size

Take away

I love the fact that there’s more than one way to work with data. I guess it comes down to preference. I was pleasantly surprised by the datatable XML formatting, nice and clean! If  export file size is an issue, then datasets can help, the trade off being you’ll loose rich data of the PS objects. The *CliXML cmdlets are sufficient for my needs, if I’m honest with myself, still glad I looked into data sets/tables…

Now If you have a full fledged SQL environment you can take advantage of, then, Go for it!!! Just ask Rob Sewell aka sqldbawithbeard  or Chrissy LeMaire, our PowerShell SQL experts to point you in the right direction!

The more you know! 😉

Hope it’s worth something to you…

Ttyl,

Urv

ACLs Folder class

‘Sup PSHomies,

I recently had to make a quick Backup & Restore of ACLs three levels deep. Nothing fancy, just two functions, but that got me thinking…

Why not make a class of this?

And so I did! Here’s the framework for the class:

classACLFolders

Here’s a list of the methods:

  • Backup. Backup SDDL from the property $Folder of the class
  • Restore. Restore SDDL to the property $Folder of the class
  • Clone. Clone will take a target Folder and clone the SDDL of $Folder to it
  • ConvertSDDLToAccess. This will enumerate what the SDDL stands for

Default Constructor

classACLDefConstructor

The default constructor will evaluate the folder used to instantiate the object. If successful, the SDDL, Owner and Access is retrieved using  the Backup() method. All actions are registered for future reference.

Instantiating is pretty straightforward:

instantiateClass

Backup()

classACLBackup

This will retrieve the SDDL for the folder and enumerate the Access.

Restore()

classACLRestore

Restore is a bit tricky. For one you need to make sure it isn’t empty. Set-Acl has no problem setting an empty SDDL, blowing everything open (worst case scenario, and that’s why you should test this in a lab first!). The other challenge is having a valid SDDL string. You can change the SDDL string if you want to something gibberish, hence the typecast as a precaution.

Clone()

classACLDefClone

The same goes for cloning. In this case we need to test the target path. Alternatively, you could  also change the Folder to a new path… It works, you’d just have misleading ActionHistory entries… I wonder if there’s a way to make this read-only,  just thinking out loud here… (note to self)

ConvertSDDLToAccess()

This is just a lil’ something extra. Like I said in a previous blog SDDL really gives more information. For one, the SID will let you know which domain this object belongs to. One thing I ran into with ReACL is that SIDHistory will resolve to the current NTAccount. This had me puzzled for a while until I saw that the SIDs in SDDL where different.

Here’s what the ouput looks like:

convertSDDLToAccess

Now for those of you that are wondering just what is this AccessMask, wonder no more! 🙂

Remember the RoboCopy ExitCodes blog I did a while back? Well it’s the same principal 🙂 This is why classes & everything related should be on your radar…

enumAccessmask

Here’s how this works…

Say I wanted to evaluate the AccessMask of a SDDL entry

AccessMaskEnum

Here I have the SID & the NTAccount. This is the builtin administrators account but it also works for Domain accounts.classACLConvertSDDLToAccess

There’s a private function that will translate the SID accordingly.

To see what the account can actually do we can enumerate the AccessMask

AccessMaskEnum1

This is what we’d see using the advanced Security GUI tab of a folder.AdvancedPermissions

Not bad… Not bad at all…

I can’t state this enough, SDDL is NOT to be be trifled with. Yes you need admin rights and please try this is a testlab first.  SDDL is very potent, if used with caution, it could do a whole bit of good!

So finally, here’s the code…

Hope it’s worth something to you…

Ttyl,

Urv

AD Operation Validation class

‘Sup PSHomies,

I’m like a dog with a bone… 😛

2016 was all about operation validation for me. I did a series on Active Directory snapshot, report and validation that was well received by the community! Classes will definitely make the user experience more pleasant! I decided to refactor the code to a class 😉 Got a lot of ground to cover so let’s dive in!

Here’s a screenshot of the ADInfrastructure (was what popped in my mind at the time) class properties and methods:

classadinfra

The focus will be on Active Directory’s forest, domain, sites, sitelinks, subnets and domaincontrollers.

These are the methods I’ve worked out so far (work in progress):

GetCurrentConfig() will populate the necessary properties using the right activedirectory cmdletsgetcurrentconfig

ImportADSnapshot() will import a saved XML file to ADSnapshot property.importadsnapshot

ExportADSnapshot() saves the class object as a XML file. This will generate a new XML file using the current $exportDate valueexportadsnapshot

RunValidation() deserves it own section… 😉

RunValidation()

This is where the the operation validation will take place. This was a bit of a challenge getting the method right, but I think it worked out just fine… I should explain…

In my first attempt I ran the validation directly from the method. You can invoke the Describe block just like a function. That wasn’t the challenge, have a look at the It blockitblock

The test is hard coded to use $this as source and $this.ADSnapshot as target. I blogged about some possible validation gotchas a while back. To remedy this, I decided to use a scriptblock. You can also provide parameters to a scriptblock . In this case I provided ($Source, $Target) as parameters. This will make interchanging  input easier of which I’ll explain the advantages later on.

desribesourcetarget

Ok so the scriptblock was a good idea. One of the things I wanted to do was save the validation results. That’s where I ran into something interesting. The input has to be a *.tests.ps1 file(s). I tried using the scriptblock as input but that didn’t work. I visited the github page to see if scriptblock is supported as a feature, it isn’t. In order to save the results I would first have to save the test to a file. The scriptblock made that part a lot easier. As a workaround, this isn’t an issue.The file is generate every time RunValidation is executed, an inconvenience at most.  A scriptblock feature  would make for a cleaner approach.

Quick side step: There’s a poll on twitter for anyone interested in casting a vote 🙂 Only 5 more days left…

pester-poll

The test results are saved in ValidationResults. That’s RunValidation in a nutshell.

It’s quite a bit of code, so I’ll post that at the end of the blog. Here’s what you can expect if you try it out. First up, a simple verification of the current configuration against a saved snapshot

#region Verify Current Configuration against a snapshot
$snapShotDate = '12012017'
$ADInfra = [ADInfrastructure]::New()
$ADInfra.GetCurrentConfig()
$ADInfra.snapshotDate = $snapShotDate
$ADInfra.ExportADSnapshot()
$ADInfra.ImportADSnapshot()
$ADInfra.ADSnapshot
$ADInfra.RunValidation($ADInfra,$ADInfra.ADSnapshot,@('Forest','Domain'))
$ADInfra.RunValidation($ADInfra.ADSnapshot,$ADInfra,@('Forest','Domain'))
$ADInfra.ActionHistory | Select-Object -Property TimeGenerated,Tags,MessageData
#endregion

Before you get startetd, you need to instantiate the class. GetCurrentConfig() will save the information to the properties. ExportADSnapshot() will create a ADSnapshot-($exportDate).xml file. ImportADSnapshot will import any existing snapshot file of a given $snapshotDate formatted as ‘ddMMyyyy’.

Because I’m verifying the current configuration with a snapshot without any changes all the tests will pass.

currentconfigtest

No surprises.

For the next example, I wanted to validate against a manual configuration. This is where the scriptblock really made a difference. I added a non-existent DC to  $ADVerifyConfig for testing purposes.

 

The class is instantiated differently this time. The values are added externally. If you run GetCurrentConfig() at any point, it will rewrite the default values. Here are the results of the snapshot vs manual source first.

$ADVerifyInfra.RunValidation($ADVerifyInfra.ADSnapshot,$ADVerifyInfra,@('DomainControllers'))

mansnapshotvssource

The snapshot only has one DomainController, we never get to the second DomainController. Now if we switch parameters from positions…

$ADVerifyInfra.RunValidation($ADVerifyInfra,$ADVerifyInfra.ADSnapshot,@('DomainControllers'))

mansourcevssnapshot

Ah! DC-DSC-02 doesn’t exist in the snapshot so it will fail! There are always two sides to consider. RunValidation() makes it easier to test and verify both sides…

Bonus round

ActionHistory

I recently discovered the Information stream in PowerShell v5. I decided to make use of Write-Information to log activities as I go along. This makes for easier troubleshooting of actions and/or sequences of methods being executed, couldn’t hurt… 😉

actionhistory

ValidationResults

Saving the validation test result enables you to process the results in different ways.

For starters you can use Format-Pester by Erwan Quélin to generate documentation of the results. Now because it’s an object you can just as easily run a query:

$ADVerifyInfra.validationResults.Results.TestResult.Where{$_.Passed -eq $false}

You can even send a high-level overview to Slack (It’s on my to-do list).

Whew! I think I’ve covered all the essentials… Ok as promised the code:

Classes will definitely enhance your end-user’s experience…

Hope it’s worth something to you…

Ttyl,

Urv