Tag Archives: Export

Venturing into the world of PowerShell & DataTables

‘Sup PSHomies,

Been rethinking my Active Directory snapshot approach…

There are many ways to Rome, so which one to you choose? I’ve always been a fan of CSV and the Export-Csv cmdlet. Then I found out that Export-CliXML makes for a better experience when it comes to saving PS Objects (just don’t try figuring the tags out). Use the Import-CliXML cmdlet and you’ve got your PS Objects with Types back and in tact! Kevin Marguette has a great blog about this. While you’re at it, Jeff Hicks also has a great series on this topic, sharing is caring… 🙂

Gathering all you Active Directory data in one PS Object might not be the best approach when it comes to performance on a sizable AD. In my test lab, that wasn’t an issue. When I ran my script against a sizable AD, let’s just say I ran into some memory issues… 😉

My objective is to:

  • Gather AD data for offline reporting  purposes.
  • Keep data fragmentation to a minimum.

Let’s gather ADUser data for this exercise…

I want to collect the following data on AD users:

  • Disabled
  • Expired
  • NoExperationDate
  • Inactive
  • MustChangePassword
  • CannotChangepassword
  • All User & Properties

Approach #1

Pretty straightforward. Use the pipeline to keep resource usage to a minimum. When processing large collections you should always think pipeline!

Get-ADUser -Filter * -Properties * |
Export-Clixml .\export\dsa\dsADUsers-CliXML.xml -Encoding UTF8

Exporting directly using Export-CliXML will get the job done. I could do this for each AD User query:

  • Search-ADAccount -AccountDisabled
  • Search-ADAccount -AccountExpired
  • Get-ADUser -LDAPFilter ‘(|(accountExpires=0)(accountExpires=9223372036854775807))’
  • Search-ADAccount -AccountInactive
  • Get-ADUser -Filter {pwdLastSet -eq 0}
  • Get-ADUser -Filter * -Properties CannotChangePassword |Where-Object {$_.CannotChangePassword}
  • Get-ADUser -Filter * -Properties *

Nothing wrong with that, other than having a bunch of CliXML files. Collecting everything first isn’t an option performance wise.

Approach #2

Use a data Set/Table.

Full disclosure: This is my first attempt using data set/tables. I’m pretty sure that there’s a better approach and I’d love to here all about it!!!

I (re)found Chrissy LeMaire blog on basic .NET Datasets. Now why doesn’t that surprise me? 😛 Hehe…

 

Quick breakdown: First I created a Dataset (prefix ds just in case you were wondering…) and the data table. I wanted all available properties. I realize this might be more than you need so feel free to change that to your needs. To get the necessary column Names I’m using the Get-Member cmdlet with parameter MemberType Property. A quick select and expand and I have column names, beats hard coding column names…

Ok here’s where it gets a bit tricky, So I’m processing each object as it goes through the pipeline, but I do have a datatable object… Like I said first attempt…

Once the data table is ready just add it to the dataset! A dataset can hold multiple datatables which helps in solving my data gathering fragmentation.

Fun fact

Did you know that you can write/export your datatable to XML? Unlike CliXML output this one is legible…

dataTable ADuser

To write the dataTable to XML:

$dtADUsers.WriteXml('c:\scripts\export\dsa\dtADUsers.xml')

Same command applies for the dataSet 🙂 Added bonus is the file size, much smaller than CliXML, but then again not as rich…

dataTable ADuser size

Take away

I love the fact that there’s more than one way to work with data. I guess it comes down to preference. I was pleasantly surprised by the datatable XML formatting, nice and clean! If  export file size is an issue, then datasets can help, the trade off being you’ll loose rich data of the PS objects. The *CliXML cmdlets are sufficient for my needs, if I’m honest with myself, still glad I looked into data sets/tables…

Now If you have a full fledged SQL environment you can take advantage of, then, Go for it!!! Just ask Rob Sewell aka sqldbawithbeard  or Chrissy LeMaire, our PowerShell SQL experts to point you in the right direction!

The more you know! 😉

Hope it’s worth something to you…

Ttyl,

Urv

Advertisements

Revisiting Exporting Data

Sup PSHomies!

I’ve been playing with different formats lately. I’d like to share a few thoughts on the subject if I may… For demo purposes I’ll be using the following cmdlets: Export-Csv, Export-Clixml and ConvertTo-Json!

Export-Csv

I’ve talked about my love for exporting to csv in the past. Here’s thing, exporting to CSV treats everything as a string. For reporting purposes this might not be an issue. When it comes to nested objects… Yeah… Then you’re better off exporting to XML. Incidentally Jeff Hicks has a great blog on this topic, you should definitely check it out! CSV is still my goto format because of reporting in Excel, although, I’ve been using Doug Finke’s ImportExcel module more and more! Doug’s module cuts out the middle man and can export to Excel without having to export as a CSV first. It does a whole lot more! Worth looking into!

Export-Clixml

Exporting a nested object is pretty straightforward using Export-Clixml. The structure isn’t pretty though. That was the main reason I didn’t use the cmdlet. Export-Clixml is great when used in combination with Import-Clixml, it restores your nested object without a hitch! You can export your results, send the file and import elsewhere for further processing if needed. When I think of xml, I immediately conjure up ideas of html reporting. The xml tags are too cryptic for any css style, I wouldn’t even know where to begin. I recently discovered PScribo (Thanks to the PowerShell Summit in Stockholm), a module by Ian Brighton! This made html reporting a breeze! All I did was import my XML file back into PowerShell to retrieve my nested object and I did the rest in PowerShell! That was awesome!

ConvertTo-Json

The ConvertTo-Json cmdlet has been introduced in PowerShell version 3.0. Back then I was a stickler for XML so I briefly looked at it and forgot all about it… That is until Azure Resource Manager came along. If you’re doing anything with Azure Resource Manager then Json should be on your radar. If you’re not convinced just look at the ARM Templates out there. Json is a lot easier on the eyes for sure. Still not conviced? Just google Json vs XML.

Ok here’s some code you can play with to get a general idea of what the possibilities are when exporting to different formats. Have a look at the Json and Xml, which would you prefer? That was rhetorical… 😉

Bottom line

Export-Csv is best when you need to report  anything in Excel workbooks and you’re not worried about type. Everyone gets Excel.

Export-Clixml isn’t pretty but excellent when it comes to keeping the data metadata in tact. You can always import preserving the metadata and process further in PowerShell.

Use Json if you want to have a structured data set à la XML. Json is a lot friendlier than XML. I was also surprised that the cmdlet interpreted values to it’s best avail. False became Boolean as you would expect. Json is growing on me…

Hope it’s worth something to you…

Ttyl,

Urv