Sort using LINQ to XML

The hobby for the last few days was to “play” with XML and LINQ syntax.

Something that I’ve tried doing was to sort an XML document not using XSL stylesheet but using LINQ. I’ve start by sorting the root elements from the XML documents.

        private static XElement Sort(XElement element)
        {
            return new XElement(element.Name,
                    element.Attributes(),
                    from child in element.Nodes()
                    where child.NodeType != XmlNodeType.Element
                    select child,
                    from child in element.Elements()
                    orderby child.Name.ToString()
                    select Sort(child));
        }

        private static XDocument Sort(XDocument file)
        {
            return new XDocument(
                    file.Declaration,
                    from child in file.Nodes()
                    where child.NodeType != XmlNodeType.Element
                    select child,
                    Sort(file.Root));
        }

Next step was to sort  deep XNodes. In order to this we need to identify the node:

                IEnumerable<XElement> currentNode =
                    (from el in xDocReport.Root.Elements("Incident")
                     where (string)el.Attribute("INumber") == refrence.ToString()
                     select el                  
                     );

… then based on the node we just do the sorting

                var xmlQuery = from item in currentNode.First().Descendants("Update")
                            orderby (Int32)item.Attribute("UNumber") descending
                            select item;

… then just replace the container with the one sorted

currentNode.First().Element("Updates").ReplaceWith(new XElement ("Updates", xmlQuery.Take(5)));

Take.(5) means that we are just taking the lsat 5 elemenst from the query.

All the above will be done in a foreach statement:

// Normal

foreach (var itemTest in xDocReport.Root.Elements("Incident").Attributes("INumber")) { }

// multi threading 

Parallel.ForEach(xDocReport.Root.Elements("Incident").Attributes("INumber"), itemTest => { });

The full schema based on which the XML was created and the LINQ query applied was looking as the one below:

<?xml version="1.0" encoding="utf-8"?>
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="Report">
    <xs:complexType>
      <xs:sequence>
        <xs:element maxOccurs="unbounded" name="Incident">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="Company" type="xs:string" />
              <xs:element name="Description" type="xs:string" />
              <xs:element name="Priority" type="xs:unsignedByte" />
              <xs:element name="RemainingSLA" type="xs:string" />
              <xs:element name="SLA-Stage" type="xs:string" />
              <xs:element name="Updates">
                <xs:complexType>
                  <xs:sequence minOccurs="0">
                    <xs:element maxOccurs="unbounded" name="Update">
                      <xs:complexType>
                        <xs:sequence>
                          <xs:element name="Text" type="xs:string" />
                          <xs:element name="UpdatedBy" type="xs:string" />
                          <xs:element name="UpdateTime" type="xs:string" />
                        </xs:sequence>
                        <xs:attribute name="UNumber" type="xs:unsignedShort" use="required" />
                      </xs:complexType>
                    </xs:element>
                  </xs:sequence>
                </xs:complexType>
              </xs:element>
            </xs:sequence>
            <xs:attribute name="INumber" type="xs:string" use="required" />
          </xs:complexType>
        </xs:element>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

 

 

Powershell & C# Notifyicon

Some time ago I’ve managed to create a tool to show me in real time when a new high priority incident is being logged and when one of the priority incidents was closed. In the background the tool was querying Service Now and extracting some XML file which then were stored in memory and displayed in a grid. To make some room on my desktop, the tool was configured to run in the background and display small notification balloons in the corned with what’s changed in the database.

Based on the above, I’ve tried replicating the same in Powershell. This may help when running time consuming scripts and  when we want periodic notification regarding the status of the script.

In C# the code is pretty simple:

notifyIcon1 = new NotifyIcon()


if (notifyIcon1.Visible != true)
{
   notifyIcon1.Visible = true;
}                                                                 // balloon made visibil
notifyIcon1.BalloonTipText = StringNotification;                  // set text
notifyIcon1.BalloonTipTitle = Convert.ToString("NEW: " + newIncidentList.Count + " || CLOSED: " + oldIncidentList.Count);
notifyIcon1.ShowBalloonTip(60000);                                 // How long

The result:

CsharpCapturePNG

 

In PowerShell in order to use a Notifyicon we will need to create an object type System.Windows.Forms.NotifyIcon. After that, the code is almost as the C# one.

The full function will be:

function Show-BalloonTip  
{
 
  [CmdletBinding(SupportsShouldProcess = $true)]
  param
  (
    [Parameter(Mandatory=$true)]
    $Text,
   
    [Parameter(Mandatory=$true)]
    $Title,
   
    [ValidateSet('None', 'Info', 'Warning', 'Error')]
    $Icon = 'Info',
    $Timeout = 10000
  )
 
  Add-Type -AssemblyName System.Windows.Forms

  if ($script:balloon -eq $null)
  {
    $script:balloon = New-Object System.Windows.Forms.NotifyIcon
  }

  $path                    = Get-Process -id $pid | Select-Object -ExpandProperty Path
  $balloon.Icon            = [System.Drawing.Icon]::ExtractAssociatedIcon($path)
  $balloon.BalloonTipIcon  = $Icon
  $balloon.BalloonTipText  = $Text
  $balloon.BalloonTipTitle = $Title
  $balloon.Visible         = $true

  $balloon.ShowBalloonTip($Timeout)
}

We can then use this function in a time consuming script and when we need something to be displayed we will append this function. The mandatory parameters are -text and -title

Show-BalloonTip -text "Some text here" -title "Powershell..... 64.... char"

The output:

notifyIcon

Monitoring virtual machines through scripts

How about monitoring VM’s using scripts instead of going to vCenter client? Is this posible?

As an administrator you can select a VM or an ESX and go to Performance tab and see the stats.

vc-perf-001

The stats provided by vCenter will look something similar with the ones below:

vc-perf

But how about creating your own charts based on some stats you are setting manually?

The first step will be to extract all the info you need by querying the virtual machines.

# import the necessary snapins

Add-PSSnapin VMware.VimAutomation.Core
Add-PSSnapin VMware.VimAutomation.Vds

# define the input file
# this will be the list of servers based on which the script will gather data
$inputFile = "C:\somepathhere\stats-vcenter.csv"

# define & create the output location
$start = Get-Date
$outputLocation = "C:\somepathhere\Get-VMPerfRealtime\{0}-{1:D2}-{2:D2} {3:D2}{4:D2}\" -f $start.Year, $start.Month, $start.Day, $start.Hour, $start.Minute
New-Item -ItemType Directory -Path $outputLocation | Out-Null

# connect to vCenter
Connect-VIServer My-vCenter-server -WarningAction SilentlyContinue | Out-Null

# get the VM objects
# get the list of servers either from an imput document - *.txt or *.csv or by using a query directly pointed to vCenter.
$vms = Get-VM -Name (import-csv $inputFile)


$vms | Foreach-Object {

	$serverStats = $_ | Get-Stat -Realtime -Stat (($_ | Get-StatType -Realtime) | Where { ($_ -like "cpu.entitlement*") -or ($_ -like "cpu.demand*") }) | Sort Timestamp,MetricId,Instance | Select Timestamp,MetricId,Value,Unit | Group Timestamp
	
	$results = @()
	
	$serverStats | Foreach-Object {

		$cpuDemand = $_.Group | Where { $_.MetricId -like "cpu.demand*" } | Select -ExpandProperty Value
		$cpuEntitlement = $_.Group | Where { $_.MetricId -like "cpu.entitlement*" } | Select -ExpandProperty Value

		$results += New-Object PSObject -Property @{
			'Timestamp' = ($_.Name -split " ")[1].Substring(0,5);
			'CPU Demand (MHz)' = $cpuDemand;
			'CPU Entitlement (MHz)' = $cpuEntitlement
		}
	}
	
	$filename = "{0}{1}.csv" -f $outputLocation, $_.Name
	
	$results | Export-Csv $filename -NoTypeInformation
}

Disconnect-VIServer My-vCenter-server -Confirm:$false

The above script is getting the CPU demand and CPU Entitlement (in Mhz) stats and store them in a *.csv format somewhere on disk. The *.csv file will contain some stats for each server we’ve queried.

"CPU Demand (MHz)","Timestamp","CPU Entitlement (MHz)"
"7095","8:55:","5845"
"7258","8:55:","5845"
"5289","8:56:","5845"
"4053","8:56:","5845"
"3029","8:56:","5845"
"4210","8:57:","5845"
"5019","8:57:","5845"
"4821","8:57:","5845"
"3599","8:58:","5845"
"2411","8:58:","5845"
"2736","8:58:","5845"
"2331","8:59:","5845"
"2723","8:59:","5845"
"2326","8:59:","5845"
"3221","9:00:","5845"

… then based on the stats we already captured we can create our own charts. This is not something very easy to do in Powershell, but as Powershell is very well integrated with all C# classes we can easily use the System.Windows.Forms namespace. The System.Windows.Forms namespace contains classes for creating Windows-based applications that take full advantage of the rich user interface features available in the Microsoft Windows operating system. (https://msdn.microsoft.com/en-us/library/system.windows.forms(v=vs.110).aspx)

We will need to also use “Reflection” to add the necessary assemblies in Powershell. You may refer Scripting Guy! Blog.

# load the appropriate assemblies 
[Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms") | Out-Null
[Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms.DataVisualization") | Out-Null

Now that we added the assemblies we will need to create a window for type chart, declare the chart area, create the chart legend and draw all necessarily lines.

We will end with a script like the below one:

########################################
# Plot the data
########################################



# load the appropriate assemblies 
[Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms") | Out-Null
[Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms.DataVisualization") | Out-Null

foreach ($file in (Get-ChildItem $outputLocation -Filter *.csv)) {

	$chart1 = New-object System.Windows.Forms.DataVisualization.Charting.Chart
	$chart1.Width = 1200
	$chart1.Height = 500
	$chart1.BackColor = [System.Drawing.Color]::White

	# title 
	$chart1.Titles.Add($file.BaseName) | Out-Null
	$chart1.Titles[0].Font = "Arial,13pt"
	$chart1.Titles[0].Alignment = "topLeft"

	# chart area 
	$chartarea = New-Object System.Windows.Forms.DataVisualization.Charting.ChartArea
	$chartarea.Name = "ChartArea1"
	$chartarea.AxisY.Title = "MHz"
	$chartarea.AxisX.Title = "Time"
	$chartarea.AxisY.Interval = 1000
	$chartarea.AxisX.Interval = 10
	$chartarea.AxisY.Maximum = 10000
	$chartarea.AxisY.IsStartedFromZero = $true
	$chartarea.AxisX.MinorGrid.Enabled = $false
	$chartarea.AxisX.MajorGrid.Enabled = $false

	$chart1.ChartAreas.Add($chartarea)

	# legend 
	$legend = New-Object system.Windows.Forms.DataVisualization.Charting.Legend
	$legend.name = "Legend1"
	$chart1.Legends.Add($legend)

	$datasource = Import-Csv $file.FullName

	[void]$chart1.Series.Add("CpuDemand")
	$chart1.Series["CpuDemand"].ChartType = "Line"
	$chart1.Series["CpuDemand"].BorderWidth  = 2
	$chart1.Series["CpuDemand"].IsVisibleInLegend = $true
	$chart1.Series["CpuDemand"].chartarea = "ChartArea1"
	$chart1.Series["CpuDemand"].Legend = "Legend1"
	$chart1.Series["CpuDemand"].color = "#62B5CC"
	$datasource | ForEach-Object {$chart1.Series["CpuDemand"].Points.addxy( $_.Timestamp , $_.'CPU Demand (MHz)') | Out-Null }

	[void]$chart1.Series.Add("CpuEntitlement")
	$chart1.Series["CpuEntitlement"].ChartType = "Line"
	$chart1.Series["CpuEntitlement"].BorderWidth  = 2
	$chart1.Series["CpuEntitlement"].IsVisibleInLegend = $true
	$chart1.Series["CpuEntitlement"].chartarea = "ChartArea1"
	$chart1.Series["CpuEntitlement"].Legend = "Legend1"
	$chart1.Series["CpuEntitlement"].color = "#FF0000"
	$datasource | ForEach-Object { $chart1.Series["CpuEntitlement"].Points.addxy( $_.Timestamp , $_.'CPU Entitlement (MHz)') | Out-Null }

	$chart1.SaveImage("$outputLocation$($file.BaseName).png","png")


	
	
}

The $outputLocation variable  will have the same path declared as in the 1st script where the *.csv’s are stored. Of course the *.csv files can be populated with any type of information. The only thing we need to take in consideration is that for each column in the *.csv  file we need to create a Chart series in the 2nd script like the one below.

[void]$chart1.Series.Add("CpuDemand")
	$chart1.Series["CpuDemand"].ChartType = "Line"
	$chart1.Series["CpuDemand"].BorderWidth  = 2
	$chart1.Series["CpuDemand"].IsVisibleInLegend = $true
	$chart1.Series["CpuDemand"].chartarea = "ChartArea1"
	$chart1.Series["CpuDemand"].Legend = "Legend1"
	$chart1.Series["CpuDemand"].color = "#62B5CC"
	$datasource | ForEach-Object {$chart1.Series["CpuDemand"].Points.addxy( $_.Timestamp , $_.'CPU Demand (MHz)') | Out-Null }

The System.Windows.Forms namespace is just an example of namespaces that can be added using Reflection into an Powershell script. Of course, not all namespaces from C# will work with Powershell, but most of them will.

In the end, the charts will look something like the below ones:

vc-perf-002

 

vc-perf-003

 

Threads in C#

There are few ways of starting a task thread in C#. The common one’s can be found below

The below namespace will need to be added in order to start a new thread.

using System.Threading.Tasks;

Using Task.factory

Task.Factory.StartNew(() => {Console.WriteLine("This is a task"); });

Using Action

Task task = new Task(new Action(PrintMessage));
task.Start();

….where PrintMessage is:

private void PrintMessage()
{
    Console.WriteLine("Hello Task library!");
}

Using delegate:

Task task = new Task(delegate { PrintMessage(); });
task.Start();

Using Lambda and names method:

Task task = new Task( () => PrintMessage() );
task.Start();

Using Lambda and anonymous method:

Task task = new Task( () => { PrintMessage(); } );
task.Start();

Using Task.Run in .NET 4.5

public async Task DoWork()
{
    await Task.Run(() => PrintMessage());
}

Using Task.FromResult in .NET 4.5

public async Task DoWork()
{
    int res = await Task.FromResult<int>(GetSum(4, 5));   
}
 
private int GetSum(int a, int b)
{
    return a + b;
}

System.Threading.Tasks .net class and Powershell

Last few days I was looking to see how I can add some performance to some large time consuming scripts. In C# I’ve used the System.Threading.Tasks class to create multi threading task but I need to find a way to also use this class with Powershell.

As we all know, Powershell is built on the .Net framework layer so we should be able to use the System.Threading.Task class to create parallel tasks and make out scripts running faster.

Windows PowerShell is a task automation and configuration management framework from Microsoft, consisting of a command-line shell and associated scripting language built on the .NET Framework. PowerShell provides full access to COM and WMI, enabling administrators to perform administrative tasks on both local and remote Windows systems as well as WS-Management and CIM enabling management of remote Linux systems and network devices. (https://en.wikipedia.org/wiki/Windows_PowerShell)

For those who don’t know what I’m referring to when speaking about parallel.foreach please the the below lines:

Foreach loop:

Iterations takes place sequentially one by one
foreach loop is run from a single Thread.
foreach loop is defined in every framework of .NET
Execution is slower (if single iterations are fast)

Parallel.ForEach:

Execution takes place in parallel way.
Parallel.ForEach uses multiple Threads.
Parallel.ForEach is defined in .Net 4.0 and above frameworks.
Execution is faster (if single iterations are slow)

You will need Powershell 4.0 to use this class as with .NET 4.0 new functionalities were added.

PS H:\> $delegate = [System.Action]{ Write-Host "Test" }
PS H:\> [System.Threading.Tasks.Task]::Run($delegate)


Id                     : 1
Exception              :
Status                 : Running
IsCanceled             : False
IsCompleted            : False
CreationOptions        : DenyChildAttach
AsyncState             :
IsFaulted              : False
AsyncWaitHandle        : System.Threading.ManualResetEvent
CompletedSynchronously : False



PS H:\> $psversiontable

Name                           Value
----                           -----
PSVersion                      4.0
WSManStackVersion              3.0
SerializationVersion           1.1.0.1
CLRVersion                     4.0.30319.34014
BuildVersion                   6.3.9600.17400
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0}
PSRemotingProtocolVersion      2.2


PS H:\>

You will need the .NET runtime to be loaded before using this class. Otherwise it will fail when you load it.

PowerShell (the engine) runs fine under .NET 4.0. PowerShell (the console host and the ISE) do not, simply because they were compiled against older versions of .NET. There’s a registry setting that will change the .NET framework loaded systemwide, which will in turn allow PowerShell to use .NET 4.0 classes:

reg add hklm\software\microsoft\.netframework /v OnlyUseLatestCLR /t REG_DWORD /d 1
reg add hklm\software\wow6432node\microsoft\.netframework /v OnlyUseLatestCLR /t REG_DWORD /d 1

To update just the ISE to use .NET 4.0, you can change the configuration ($psHome\powershell_ise.exe.config) file to have a chunk like this:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <startup>
      <supportedRuntime version="v4.0.30319" />
    </startup>
</configuration>

You can build .NET 4.0 applications that call PowerShell using the PowerShell API (System.Management.Automation.PowerShell) just fine, but these steps will help get the in-the-box PowerShell hosts to work under .NET 4.0.

Remove the registry keys when you don’t need them any more. These are machine-wide keys and forcibly migrate ALL applications to .NET 4.0, even applications using .net 2 and .net 3.5

WMISearcher vs Get-WmiObject

Is there a difference between Get-WmiObject and wmisearcher?

I honestly don’t know the answer but I’ve run some tests on different subnets and looks like using the .Net class System.Management.ManagementObjectSearcher it’s a little bit faster than using the traditional gwmi.

CODE:

$list = get-adcomputer -filter 'OperatingSystem -like "*server*2012*"' | select -first 50 | select -ExpandProperty name

measure-command {
    try {
        foreach ($hostname in $list){
            $searcher = [wmisearcher]"Select * from Win32_Service where Name='lanmanserver'"
            $searcher.scope.path = "\\$Hostname\root\cimv2"            
            $searcher.get()
        }
    }
    catch [exception] {
        Write-host ("Error: {0} on {1}" -f $_.exception.Message, $hostname) -ForegroundColor Red
    }
}
    
write-host "-------"

measure-command {
    
    foreach ($hostname in $list){
        try{
            gwmi -ComputerName $hostname -class Win32_Service | where {$_.Name -like "*lanmanserver*"}
        }
        catch [exception]{
            Write-host ("Error: {0} on {1}" -f $_.exception.Message, $hostname) -ForegroundColor Red
        }
    }
}

 

and the result:

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 37
Milliseconds      : 194
Ticks             : 371942651
TotalDays         : 0.000430489179398148
TotalHours        : 0.0103317403055556
TotalMinutes      : 0.619904418333333
TotalSeconds      : 37.1942651
TotalMilliseconds : 37194.2651

-------

 
Days              : 0
Hours             : 0
Minutes           : 2
Seconds           : 54
Milliseconds      : 729
Ticks             : 1747294155
TotalDays         : 0.00202233119791667
TotalHours        : 0.04853594875
TotalMinutes      : 2.912156925
TotalSeconds      : 174.7294155
TotalMilliseconds : 174729.4155

 

I’ve run the same query against the same 50 random servers but this time instead of searching for a particular service I’ve extracted all the services with all the attributes.

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 36
Milliseconds      : 807
Ticks             : 368071268
TotalDays         : 0.000426008412037037
TotalHours        : 0.0102242018888889
TotalMinutes      : 0.613452113333333
TotalSeconds      : 36.8071268
TotalMilliseconds : 36807.1268

-------

Days              : 0
Hours             : 0
Minutes           : 3
Seconds           : 49
Milliseconds      : 68
Ticks             : 2290687682
TotalDays         : 0.0026512588912037
TotalHours        : 0.0636302133888889
TotalMinutes      : 3.81781280333333
TotalSeconds      : 229.0687682
TotalMilliseconds : 229068.7682