Monthly Archives: July 2015

 

Sup PSHomies?

So this regular expression thing has it’s advantages.  Like many things practice makes perfect…

In the past whenever I saw regex I’d flat line… There wasn’t really a need to use it, select-string with a dash of  -like, -contains, -split here and there, got me pretty far so why bother eh? Well for one thing regex is for the big boys, much like LDAPFilter on Active Directory cmdlets… Guess it’s time to put on my big boy pants and step my game up! 😛

Jeffrey Hicks recently blogged on converting Text to PSCustomObjects using regex. Anything Jeffrey publishes is golden…

Jeffrey’s pattern is easy enough to follow…

regex]$pattern = '(?<ID>\w+)\s+(?<Chassis>\d)\s+(?<Slot>\d)\s+(?<RAIDID>\w+)\s+(?<Status>\w+)\s+(?<Type>\w+)\s+(?<Media>\w+)'

Here’s a quick breakdown:

  • \w+ : Matches any word character as much as possible (that’s what the + is for)
  • \s+  : Matches any whitespace character as much as possible
  • \d    : Matches any decimal digit

Here a link to regex cheat sheet to explain what the character classes actually do…

Fun fact: I’ve been the proud owner of the Windows PowerShell Pocket reference (first edition, for quite some time I may add) and there is a whole chapter dedicated to regular expressions! Go figure…

The trick is to use capture names in your pattern to store the results. I saw this in the robocopy script by Joakim Svendsen. What I didn’t realize was that you can retrieve the captured names using GetGroupNames() method, nice! Just one thing, the first GroupName is always 0 it seems so just skip that one.

$pattern.GetGroupNames() | select -skip 1

This makes enumerating the names easy! Just head on over to Jeffrey’s blog to see how it’s done… 😉

In the mean time here’s a lil’ something to help you verify that the pattern works… 😉

@'
ID Chassis Slot RAIDID Status Type Media Spare SizeGB
=====================================================
c0d0  0     0    c0r0    Ok     sas  HDD   -    150
c0d0  0     0    c0r1    Ok     sas  HDD   -    150
c0d1  0     1    c0r0    FAILED sas  SDD   -    150
c0d1  0     1    c0r1    FAILED sas  SDD   -    150
'@ | Out-File "$env:TEMP\PatternFile.txt"
 
[regex]$pattern = '(?<ID>\w+)\s+(?<Chassis>\d)\s+(?<Slot>\d)\s+(?<RAIDID>\w+)\s+(?<Status>\w+)\s+(?<Type>\w+)\s+(?<Media>\w+)'
 
 $captureNames = $pattern.GetGroupNames() | Select-Object -skip 1

get-content "$env:TEMP\PatternFile.txt" |
ForEach-Object {
  if($_ -match $pattern){
    $captureResults = @{}
    foreach($captureName in $captureNames){
      $captureResults.Add($captureName,$Matches.$captureName)
    }
    [PSCustomObject]@{
      String = $_
      MatchFound = ($_ -match $pattern)
      regExResults = [PSCustomObject]$captureResults
    }
  }
  Else {
    [PSCustomObject]@{
      String = $_
      MatchFound = ($_ -match $pattern)
      regExResults = ''
    }
  }
} 

wpid-wp-1438336833636.jpg

Hehe… Hope it’s worth something to you

Ttyl,

Urv

Sup pshhomies? A colleague of mine Martijn Monshouwer was getting his PowerShell on when he ran into a little snag. A little birdie told him to run his script by me, which he did! “Dear PowerShell…” You had me at PowerShell… 🙂 Incidentally starting any email with PowerShell is a good way to get my attention.

The script is suppose to:

  • find user account(s) past expiration date
  • disable the user account(s)
  • move account(s) to a specific OU
  • clear the user group membership, all but three groups

Martijn did the grunt work. All that was missing was the remove group membership. I learned my lesson not to overwhelm but assist, so no rewriting entire scripts! At that time I was right in the middle of a migration, but hey, it’s PowerShell and Martijn was clever enough to appease my ego… Now how could I say no eh? 😉

I did a quick scan… “Read-Host… Yeah… let’s make that a parameter”. “Write-Host… Better change that to Write-Verbose lest we incur the wrath of Don Jones…” Ok Urv remember, no rewriting… (Well maybe just a little… old habits die hard…)

So here’s the end result:

<#

Author: M. Monshouwer / I. Strachan
Version: 1.0
Version History: N/A

Purpose:  Get expired AD accounts that are not disabled and move to disabled OU

          .\Move-ExpiredDisabledUserToDisableOU –Verbose

#>

[CmdletBinding()]
param(
    [string]$ChangeNumber='12345',

    [string]$Initials='I.S'
)

# Import Active Directory module
Import-Module ActiveDirectory -Verbose:$false

#region: Define variables
#Get expired AD accounts that are not disabled
Write-Verbose 'Searching for accounts that are enabled but expired...'
$ExpiredAccountsNotDisabled = Search-ADAccount -AccountExpired | Where-Object { $_.Enabled -eq $true }
Write-Verbose "Found $(@($ExpiredAccountsNotDisabled).count) that meet criteria"

#Define groups that will NOT be removed, Use the DistinguishedName
$ExcludedGroups = @(
'CN=GroupX,OU=Groups,OU=Resources,DC=pshirwin,DC=local'
'CN=GroupY,OU=Groups,OU=Resources,DC=pshirwin,DC=local'
)

#Define OU for Disabled Accounts. Change to reflect your OU
$DisabledAccountsOU = 'OU=Disabled,OU=Users,OU=IPI,DC=pshirwin,DC=local'

#Get current date
$Date = Get-Date -Format g
#endregion

#Loop through Expired Users
ForEach($User in $ExpiredAccountsNotDisabled) {
  Write-Verbose "Processing user $($user.SamAccountName) group membership"
  #Get User Group Membership
  $userMembership = Get-ADUser $user.SamAccountName -Properties MemberOf | Select-Object -ExpandProperty memberof
  Write-Verbose "User membership count: $(@($userMembership.Count))"

  foreach($member in $userMembership) {
    # Remove User Group membership
    if (!$ExcludedGroups.Contains($member)){
      try{
        Write-Verbose "Removing user $($User.SamAccountName) from $($member)"
        Get-ADGroup $member | Remove-ADGroupMember -Members $user.SamAccountName -Confirm:$false
      }
      catch{
        Write-Error "Something went wrong removing $($User.Name) from `'$member`'"
      }
    }
  }

  #Process user further
  try{
    # Write Description & Disable Account
    Write-Verbose "Setting user's Description to: `"Disabled $Date; $ChangeNumber; $Initials`""
    Set-ADUser -Identity $User.SamAccountName -Description "Disabled $Date; $ChangeNumber; $Initials" -Enabled $false
    # Move user to OU for Disabled Accounts
    Write-Verbose 'Moving user to Disabled OU'
    Get-ADUser -identity $User.SamAccountName | Move-ADobject -targetpath $DisabledAccountsOU
  }
  catch{
    Write-Error "Something went wrong processing `'$($User.DistinguishedName)`'"
  }
}

Martijn was quite happy with his script! Welcome to the club! 😉

While at the gym ( my other favorite past time) I had an epiphany (Sounds more dramatic eh?)
What if you needed to reinstate a user? The user would have to be moved back to the original OU, ReEnabled and be made member of all the groups removed from. Of Course this could be done manually but what’s the fun in that? Oh and some logging would be nice…

So Martijn…

Mission Impossible

Try making your script more resilient. Level up! Learning PowerShell is fun!!!

Ttyl,

Urv

Get-RCLogSummary take II

Trying to figure out this regular expression thing…. I just can’t stand the idea of not getting it!

Regular Expression meme

Joakim Svendsen Get-FolderSize uses regular expression, so I did some investigating. Why reinvent the wheel eh? 😉 Turns out there’s a lot going on under the hood here. First I had to figure out how the regular expression worked so I isolated just one…

[Regex]$regex_Dirs = 'Dirs\s:\s+(?<TotalDirs>\d+)\s+(?<CopiedDirs>\d+)(?:\s+\d+){2}\s+(?<FailedDirs>\d+)\s+\d+'

Turns out you can give your result a label if a match is found. Running the regular expression with -match operator will populate the $Matches variable with found label/value, which allows you to retrieve that value later on.

if ($_ -match $regex_Dirs){
  $rcLog.TotalDirs = [int]$Matches['TotalDirs']
  $rcLog.CopiedDirs = [int]$Matches['CopiedDirs']
  $rcLog.FailedDirs = [int]$Matches['FailedDirs']
} 

Quick side note: This script works with the /Bytes option. The plus side of using /Bytes is that you can do some fun stuff like calculating the size in GB or MB

TotalBytes  = $Matches['ByteCount']
TotalMBytes = ([int64] $Matches['ByteCount'] / 1MB).ToString('N')
TotalGBytes = ([int64] $Matches['ByteCount'] / 1GB).ToString('N')

Nice!

There’s yet another option, using ConvertFrom-String, an excellent blog by Bartek Bielawski brought to my attention by Dexter Dhami. I’m working on that as well. That one will only work with PowerShell version 5 so there’s that. Actually, I saw this feature in ISESteroids first at the PowerShell Summit. What I did notice is when using ConvertFrom-String performance took quite a hit…

I have mixed feelings when it comes to ConvertFrom-String. At times he just doesn’t get it! I love the template idea but you need to be specific, otherwise some data might fall through the cracks…You never know for sure… Like all tools you should be skilled at several. Think of regular expression as driving a stick and ConvertFrom-String as an automatic. Here in the Netherlands you can get your driver license on an automatic, but then you’re not allowed to drive stick ever! But if you pass driving a stick you’re allowed to drive automatic. Best bet, go with stick!

Here’s the code using regular expression, with a lil’ help from my friends… 😉

#region Hash with the Robocopy Log properties
$rcLogProperties = [Ordered]@{
  rcLogFile = ''
  Source = ''
  Target = ''
  TotalDirs = ''
  CopiedDirs = ''
  FailedDirs = ''
  TotalFiles = ''
  CopiedFiles = ''
  FailedFiles = ''
  TotalBytes = ''
  CopiedBytes = ''
  FailedBytes = ''
  StartTime = ''
  EndTime = ''
  Speed = ''
}
#endregion

#region Main
#Get Logfiles from folder
Get-ChildItem '.\temp\23-06-2015' -File |
ForEach-Object {
  #Get Lofile Header & Footer
  $arrSummary  = (Get-Content $_.FullName)[5..8] 
  $arrSummary += (Get-Content $_.FullName)[-10..-1]

  $rcLog = New-Object -TypeName psobject -Property $rcLogProperties
  $rcLog.rcLogFile = $_.Name

  Foreach($line in $arrSummary) {
    switch ($line){
      {$_ -like  '*Source :*'}
      {
        $rcLog.Source = ($_ -replace '(\s\w+)(\s.)','').Trim()
      }
      {$_ -like  '*Dest :*'}
      {
        $rcLog.Target = ($_ -replace '(\s\w+)(\s.)','').Trim()
      }
      {$_ -like  '*Dirs :*'}
      {
        [regex]$regex_Dirs = 'Dirs\s:\s+(?<TotalDirs>\d+)\s+(?<CopiedDirs>\d+)(?:\s+\d+){2}\s+(?<FailedDirs>\d+)\s+\d+'
        if ($_ -match $regex_Dirs){
          $rcLog.TotalDirs = [int]$Matches['TotalDirs']
          $rcLog.CopiedDirs = [int]$Matches['CopiedDirs']
          $rcLog.FailedDirs = [int]$Matches['FailedDirs']
        }    
      }
      {$_ -like  '*Files :*'}
      {
        [regex]$regex_Files = 'Files\s:\s+(?<TotalFiles>\d+)\s+(?<CopiedFiles>\d+)(?:\s+\d+){2}\s+(?<FailedFiles>\d+)\s+\d+'
        if ($_ -match $regex_Files){
          $rcLog.TotalFiles = [int]$Matches['TotalFiles']
          $rcLog.CopiedFiles = [int]$Matches['CopiedFiles']
          $rcLog.FailedFiles = [int]$Matches['FailedFiles']
        }    
      }
      {$_ -like  '*Bytes :*'}
      {
        [regex]$regex_Bytes = 'Bytes\s:\s+(?<TotalBytes>\d+)\s+(?<CopiedBytes>\d+)(?:\s+\d+){2}\s+(?<FailedBytes>\d+)\s+\d+'
        if ($_ -match $regex_Bytes){
          $rcLog.TotalBytes  = $Matches['TotalBytes']
          $rcLog.CopiedBytes = $Matches['CopiedBytes']
          $rcLog.FailedBytes = $Matches['FailedBytes']
        }    
      }
      {$_ -like  '*Ended :*'}
      {
        [regex]$regex_End = 'Ended\s:\s+(?<EndTime>.+)'
        if ($_ -match $regex_End){
          $rcLog.EndTime = $Matches['EndTime']
        }  
      }
      {$_ -like  '*Started :*'}
      {
        [regex]$regex_Start = 'Started\s:\s+(?<StartTime>.+)'
        if ($_ -match $regex_Start){
          $rcLog.StartTime = $Matches['StartTime']
        } 
      }
      {$_ -like  '*Speed :*'}
      {
        [regex]$regex_Speed = 'Speed\s:\s+(?<Speed>.+\/min)'
        if ($_ -match $regex_Speed){
          $rcLog.Speed = $Matches['Speed']
        } 
      }
    }
  } 
  $rclog
}|
Out-GridView
#endregion

Hope it’s worth something to you!

Ttyl,

Urv

‘Sup PSHomies?

Got a lil’ somethin’ for ya… Get-RCLogSummary! As you know I’m a big fan of RoboCopy! I thought I’d share one of the perks of using RoboCopy: the LogFile.

Here’s a list of RoboCopy Logging options, courtesy of ss64.com

   Logging options
                /L : List only - don’t copy, timestamp or delete any files.
               /NP : No Progress - don’t display % copied.
          /unicode : Display the status output as Unicode text.  ##
         /LOG:file : Output status to LOG file (overwrite existing log).
      /UNILOG:file : Output status to Unicode Log file (overwrite)
        /LOG+:file : Output status to LOG file (append to existing log).
     /UNILOG+:file : Output status to Unicode Log file (append)
               /TS : Include Source file Time Stamps in the output.
               /FP : Include Full Pathname of files in the output.
               /NS : No Size - don’t log file sizes.
               /NC : No Class - don’t log file classes.
              /NFL : No File List - don’t log file names.
              /NDL : No Directory List - don’t log directory names.
              /TEE : Output to console window, as well as the log file.
              /NJH : No Job Header.
              /NJS : No Job Summary.

My preference when it comes to logging is to have seperate logfiles instead of appending to one big file. The option /NP is a no brainer, displaying ‘%’ will give you an indication how long it took for that specific file/folder, but who wants that right?It will only increase your logfile size taking more time to parse it down the line. I recently used /NDL and I must say this will keep your logfile footprint small. I did include /FP to still have an idea where the file is being copied from. I’d go with /NDL in combination with /FP when doing a delta-sync. A delta-sync is a robocopy job that will copy the differences once a full-sync has taken place. If the file hasn’t changed robocopy will skip it. Only new and newer files will be copied… Ok enough background, let get scripting shall we? 😛

Function Get-RCLogSummary{
  param(
    [String]$LogFileName,
    [String[]]$LogSummary
  )

  $objLogSummary = @{
    rcLogFile = $LogFileName
    Speed = ''
  }

  Foreach($line in $logSummary) {
    switch ($line){
      #Header
      {$_ | select-string '   Source :'}
        {
          $_= $_.ToString()
          $objLogSummary.Add('Source',$_.Substring(11).Trim())
        }
      {$_ | select-string '     Dest :'}
        {
          $_= $_.ToString()
          $objLogSummary.Add('Target',$_.Substring(11).Trim())
        }
      {$_ | select-string '  Started :'}
        {
          $_= $_.ToString()
          $objLogSummary.Add('Start',$($_.Substring(11).Trim()))
        }
      #Footer
      {$_ | select-string '    Dirs :'}
        {
          $_= $_.ToString()
          $objLogSummary.Add('TotalDirs',$_.Substring(11,10).Trim())
          $objLogSummary.Add('FailedDirs',$_.Substring(51,10).Trim())
          $objLogSummary.Add('CopiedDirs',$_.Substring(21,10).Trim())
        }
      {$_ | select-string '   Files :'}
        {
          $_= $_.ToString()
          $objLogSummary.Add('TotalFiles',$_.Substring(11,10).Trim())
          $objLogSummary.Add('FailedFiles',$_.Substring(51,10).Trim())
          $objLogSummary.Add('CopiedFiles',$_.Substring(21,10).Trim())
        }
      {$_ | select-string '   Bytes :'}
        {
          $_= $_.ToString()
          $objLogSummary.Add('TotalBytes',$_.Substring(11,10).Trim())
          $objLogSummary.Add('FailedBytes',$_.Substring(51,10).Trim())
          $objLogSummary.Add('CopiedBytes',$_.Substring(21,10).Trim())
        }
      {$_ | select-string '   Ended :'}
        {
          $_= $_.ToString()
          $objLogSummary.Add('End',$($_.Substring(11).Trim()))
        }
      {$_ | select-string '   Speed :'}
        {
          $_= $_.ToString()
          $objLogSummary.Speed = $($_.Substring(11).Trim())
        }
      {$_ | select-string '   Times :'}
        {
          $_= $_.ToString()
          $objLogSummary.Add('Time Total',$($_.Substring(11,10).Trim()))
        }
      }
    }

  #return $objLogSummary
  [PSCustomObject]$objLogSummary
}

#region:array with all LogSummary Object Properties
$arrRCProperties = @(
  'rcLogFile',
  'Source',
  'Target',
  'TotalDirs',
  'TotalFiles',
  'TotalBytes',
  'FailedDirs',
  'FailedFiles',
  'FailedBytes',
  'CopiedDirs',
  'CopiedFiles',
  'CopiedBytes',
  'Start',
  'End',
  'Time Total',
  'Speed'
)
#endregion

#region: Get all robocopy LogFiles in specified folder and get Summary
get-childitem '.\log\rc\home\22-06-2015' -File |
ForEach-Object {
  #region: Get File Header & Footer
  $arrSummary  = (Get-Content $_.FullName)[5..8] #Header
  $arrSummary += (Get-Content $_.FullName)[-11..-1] #Footer
  #endregion

  Get-RCLogSummary -LogFileName $_.Name -LogSummary $arrSummary
}|
Select-Object $arrRCProperties |
Out-GridView
#endregion

First I’ll get a list of logFiles and retrieve the first 5-8 lines and the last 10 lines of each file for processing. The LogFileName & array Summary are then passed as parameters to Get-RCLogSummary. I did a select to get the parameters in a certain order. It was a toss up between using [Ordered] Hash or  defining a [PSCustomObject] beforehand. I figured you could minimize the Properties you want by tweaking the $arrRcProperties yourself. last but not least use Out-Gridview or Export-Csv to see the endresult.

I’m working on my pipeline skills, trust me my previous version was more ‘elaborate’, and by elaborate I mean over engineered…

So I guess you’ve noticed that regular expression is missing? Robocopy labels are fixed which is a good thing for me. I’m looking into it…

wpid-wp-1435926794192.jpg

This regular expression isn’t as easy as it seems… This works, just don’t include /Bytes in your robocopy parameter list. In that case you’ll definitely need regular expression. Version 2.0 I guess…

Hope it’s worth something to you

Ttyl,

Urv