Skip to main content

Using Text Based Logs with Windows PowerShell (3 of 8)


Day 3: Getting the Properties.
Today our focus will be on step 2: Identify a rule to separate property names from property values.
Yesterday we extracted the individual records from the log file.  Today we need to work on the properties.
As script writers, we can easily provide a static set of properties for our log objects.  This approach will cost us a lot of extra coding and make future modifications of this code more difficult.  The idea here is to dynamically extract the list of potential properties from the current log file.  That will help to maintain the maximum usability of the final output of our code.
    Function Get-PropertyList
    {
    Param ($Records)
    # To help build a more flexible object, we need to identify possible
    # property names.  This function will cycle through each record and use
    # a rule to identify the name of a property.  Our rule is any string
    # after the "]" in the time stamp and before the next ":". Any
    # object containing the property names is returned.
 
        # Array to hold all of the objects.
        $Object = @()
 
        <# Cycle through each record to get a list of properties. This is
           done to discover all the potential properties.
        #>
       
        ForEach ($R in $Records.record)
        {
            $Obj = New-Object -TypeName PSObject
            # Find the end of the Date time stamp and remove it along
            # with the leading space.
            $Start = $R.IndexOf("]")
            $Value = $R.Substring($Start+2)
 
            # Find the next colon which will mark the end of the
            # property.  Remove this colon and the rest of the string.
            $End = $Value.IndexOf(":")
            $value = $Value.Substring(0,$End)
           
            # Add the property to the list.
            $Splat = @{NotePropertyName = "Prop";
                        NotePropertyValue = "$($Value)"}
            $Obj | Add-Member @Splat
            $Object += $Obj              
        } # End: ForEach ($R in $Records.record)
 
        # Make sure that each property is unique.
 
        $ObjectList = $Object | Select-Object -Property Prop -Unique
 
        # Send the records to the calling statement.
        Write-Output $ObjectList
    } #End: Function Get-ADResourcePropertyList
 
 
Here we provide the Records parameter with the $Records objects created from yesterday’s lesson.  We also need to define a rule to extract the name of the property from each record.  The idea is to provide a consistent set of members to our final object so we can feed the final object into the pipeline without being concerned if an object is lacking a specific property. Here is the rule that we will create.
1.       Find the first “]” on each line and remove all characters to the left of this point + 1.   We use a method of the String object called IndexOf.  It will only return the place number in the string of the first instance of the “]” character. We then use the Substring method to return the string minus the date time stamp.
2.       We now have the property name to the left of the next “:”.  We utilize the same process above to first find the location of the “:”.  Next, we use the Substring method to remove everything but the property name.
At this point, you have used your rule to extract the property name.  Our next step is to add the property name to an object that will be returned to the calling statement.  We need to make sure that this list of objects have unique values so we pipe the list of objects to the Select-Object cmdlet and use the –Unique parameter to make sure that we do not have any duplicate values in any of the object’s Prop property.
Here is our code now:
Function Import-MyLog1
{
    # -----------------------------------------------------------------------
    Function Get-Log
    {
    # Reads the log file into memory.  Will display an error if the
    # file cannot be found.
 
        Try
        {
            Get-Content -path "\\Server1\Data.txt”  `
                        -ErrorAction Stop |
            Write-Output
        }
        Catch
        {
            Write-Error "The data file is not present"
            BREAK
        }
    } # End: Function Get-Log
    # -----------------------------------------------------------------------
    Function Get-Record
    {
    Param ($Log)
    # Determines what is a record.  This eliminates any header information.
    # For this to work properly, you need to utilize some type of rule
    # to determine what is a new record.  In this case, a pattern
    # is being utilized that we will send to a regular expression.
    # The pattern is:
    # -- ::
    # Example - 2013-05-30 16:06:40
    #
    # See About_Regular_Expressions for more information on how to use
    # Regular Expressions in the built in PowerShell help files.
 
 
        # Array to hold all of the objects.
        $Object = @()
 
        # Cycle through each line of the log and look for a line that
        # matches the pattern that we are using to denote a new record.
        ForEach ($L in $Log)
        {
            If ($L -match "\d*-\d*-\d* \d*:\d*:\d*")
            {
                $Obj = New-Object -TypeName PSObject
 
                $Splat = @{NotePropertyName = "Record";
                          NotePropertyValue = "$($L)"}
                $Obj | Add-Member @Splat
                $Object += $Obj 
            }
        } # End: ForEach ($L in $Log)
       
        # Send the records to the calling statement.
        Write-Output $Object
 
    } # End: Function Get-Record
    # -----------------------------------------------------------------------
    Function Get-PropertyList
    {
    Param ($Records)
    # To help build a more flexible object, we need to identify possible
    # property names.  This function will cycle through each record and use
    # a rule to identify the name of a property.  Our rule is any string
    # after the "]" in the time stamp and before the next ":". And
    # object containing the property names is returned.
 
        # Array to hold all of the objects.
        $Object = @()
 
        <# Cycle through each record to get a list of properties. This is
           done to discover all the potential properties.
        #>
       
        ForEach ($R in $Records.record)
        {
            $Obj = New-Object -TypeName PSObject
            # Find the end of the Date time stamp and remove it along
            # with the leading space.
            $Start = $R.IndexOf("]")
            $Value = $R.Substring($Start+2)
 
            # Find the next colon which will mark the end of the
            # property.  Remove this colon and the rest of the string.
            $End = $Value.IndexOf(":")
            $value = $Value.Substring(0,$End)
           
            # Add the property to the list.
            $Splat = @{NotePropertyName = "Prop";
                        NotePropertyValue = "$($Value)"}
            $Obj | Add-Member @Splat
            $Object += $Obj              
        } # End: ForEach ($R in $Records.record)
 
        # Make sure that each property is unique.
 
        $ObjectList = $Object | Select-Object -Property Prop -Unique
 
        # Send the records to the calling statement.
        Write-Output $ObjectList
    } #End: Function Get-ADResourcePropertyList
    # -----------------------------------------------------------------------
    # Load the log into memory
    $Log = Get-Log
 
    # Extract the records
    $Records = Get-Record -Log $Log
 
    # Get a list of all properties.
    $ObjProps = Get-PropertyList -Records $Records
 
    $ObjProps # Added only to see the current progress.
} #End: Function Import-MyLog1
 
Here is our output:
Prop                                                                                                              
----                                                                                                              
Info                                                                                                             
Warning                                                                                                           
Critical            
 
Even though our original log file had 4 records with the name Info as the property, we have filtered out all but unique instances of these property names.  Also, take the time now to verify that you do not need to make any exceptions to this rule or special variations.
Next week we will pull it all together and turn our log file into a PowerShell object.

Comments

Popular posts from this blog

Adding a Comment to a GPO with PowerShell

As I'm writing this article, I'm also writing a customization for a PowerShell course I'm teaching next week in Phoenix.  This customization deals with Group Policy and PowerShell.  For those of you who attend my classes may already know this, but I sit their and try to ask the questions to myself that others may ask as I present the material.  I finished up my customization a few hours ago and then I realized that I did not add in how to put a comment on a GPO.  This is a feature that many Group Policy Administrators may not be aware of. This past summer I attended a presentation at TechEd on Group Policy.  One organization in the crowd had over 5,000 Group Policies.  In an environment like that, the comment section can be priceless.  I always like to write in the comment section why I created the policy so I know its purpose next week after I've completed 50 other tasks and can't remember what I did 5 minutes ago. In the Group Policy module for PowerShell V3, th

Return duplicate values from a collection with PowerShell

If you have a collection of objects and you want to remove any duplicate items, it is fairly simple. # Create a collection with duplicate values $Set1 = 1 , 1 , 2 , 2 , 3 , 4 , 5 , 6 , 7 , 1 , 2   # Remove the duplicate values. $Set1 | Select-Object -Unique 1 2 3 4 5 6 7 What if you want only the duplicate values and nothing else? # Create a collection with duplicate values $Set1 = 1 , 1 , 2 , 2 , 3 , 4 , 5 , 6 , 7 , 1 , 2   #Create a second collection with duplicate values removed. $Set2 = $Set1 | Select-Object -Unique   # Return only the duplicate values. ( Compare-Object -ReferenceObject $Set2 -DifferenceObject $Set1 ) . InputObject | Select-Object – Unique 1 2 This works with objects as well as numbers.  The first command creates a collection with 2 duplicates of both 1 and 2.   The second command creates another collection with the duplicates filtered out.  The Compare-Object cmdlet will first find items that are diffe

How to list all the AD LDS instances on a server

AD LDS allows you to provide directory services to applications that are free of the confines of Active Directory.  To list all the AD LDS instances on a server, follow this procedure: Log into the server in question Open a command prompt. Type dsdbutil and press Enter Type List Instances and press Enter . You will receive a list of the instance name, both the LDAP and SSL port numbers, the location of the database, and its status.