PowerShell Tips – Multithreading a script

With the script template I normally use to adapt any scripts, the script will expect a computer parameter in string format or a computers parameters which will point to a list in text file.  It then process the list of computer(s) and output the result as PSObject.  To run through the script with thousands of computer in sequence, it takes hours sometimes.

Thanks to Mr Google, I have managed to research for a multithreaded solution for the script.

Param($ScriptFile = "get-registry.ps1",
$ComputerList = "servers.txt",
$MaxThreads = 20,
$SleepTimer = 1000,
$MaxWaitAtEnd = 1000,
$OutputType = "GridView")

$Computers = Get-Content $ComputerList
"Killing existing jobs . . ."
Get-Job | Remove-Job -Force

$i = 0

ForEach ($Computer in $Computers){
While ($(Get-Job -state running).count -ge $MaxThreads){
Write-Progress  -Activity "Creating Server List" -Status "Waiting for threads to close" -CurrentOperation "$i threads created - $($(Get-Job -state running).count) threads open" -PercentComplete ($i / $Computers.count * 100)
Start-Sleep -Milliseconds $SleepTimer

#"Starting job - $Computer"
Start-Job -FilePath $ScriptFile -ArgumentList $Computer -Name $Computer
Write-Progress  -Activity "Creating Server List" -Status "Starting Threads" -CurrentOperation "$i threads created - $($(Get-Job -state running).count) threads open" -PercentComplete ($i / $Computers.count * 100)


$Complete = Get-date

While ($(Get-Job -State Running).count -gt 0){
$ComputersStillRunning = ""
ForEach ($System  in $(Get-Job -state running)){$ComputersStillRunning += ", $($System.name)"}
$ComputersStillRunning = $ComputersStillRunning.Substring(2)
Write-Progress  -Activity "Creating Server List" -Status "$($(Get-Job -State Running).count) threads remaining" -CurrentOperation "$ComputersStillRunning" -PercentComplete ($(Get-Job -State Completed).count / $(Get-Job).count * 100)
If ($(New-TimeSpan $Complete $(Get-Date)).totalseconds -ge $MaxWaitAtEnd){"Killing all jobs still running . . .";Get-Job -State Running | Remove-Job -Force}
Start-Sleep -Milliseconds $SleepTimer

"Reading all jobs"

If ($OutputType -eq "Text"){
ForEach($Job in Get-Job){
Receive-Job $Job
" "
ElseIf($OutputType -eq "GridView"){
Get-Job | Receive-Job | Select-Object * -ExcludeProperty RunspaceId | out-gridview

<div class="line number1 index0 alt2">

The multithread script calls the original scripts, with the same list of the computers specified in the text file.  It will then generate a pre-defined threads (20 in my current usage), and loop through the computer list.  Technically, it should speed up the process time by 20 times.  The trade off, is the memory usage of this script.  Be prepared to have a performance impact on the host you ran the script from.  Depending on the complexity of the script, I have so far able to run on 50 concurent threads with 4GB RAM without MAJOR problem.  However, it is advised to run this multiplethread script off peak or at a dedicated management server.

Once the script completed, it will output the result to a gridview.  Which you can filter, or copy and paste to an Excel spreadsheet for further analysis.