Powershell: VMWare vCenter Standard, Move all VM from a host and place it in maintenance mode

I have VMWare vCenter standard edition, and there is a limitation with placing the server in maintenance mode, as it will never complete unless you move all the VM from that host to another host and then place the server in maintenance mode. something I really don't like, its a small basic regular thing any admin need to do, maybe updating the server or whatever.
When it comes to manual work (which I hate so much), you will need to distribute your VMs based on the host which has more free memory and also healthy. the challenge I have is I have about 30 servers all running the standard edition, and I need to update the hosts, so redistribute the load and check which server has more memory, and Bla Bla Bla, of boring things, so I got the idea to automate this.

The below script (can be downloaded from here too) will do the following:

• Import the VMWare module and connect to the VI Server.
• Get a list of all the VM in the host that should be placed in maintenance mode.
• for each virtual machine, find the proper server to place it (based on the most server which has the freest memory).
• Redistribute the load.
• if no server is available with enough memory to have the VM, the script will let you know and won't overcommit your servers, unless you want so.
• After all is done, will provide you with a basic report about each VM, the old host, and the new host.
• place the server in maintenance mode.

Parameter and Overcommit prevention

The script requires some parameter to be filled before it can do the magic:

• FromVMHostName: The name of the server you want to move the VM from and place it in maintenance mode, Please Note that the server name should be exactly as the one registered in your vCenter, so if you are using FQDN, pass it here too.
• MaxMemAllowed: The Percentage of the total free memory of the destination server before being excluded from the selection, the default value is 80, so no VM will be shipped to any server which has more than 80 percent of utilized memory.
• vCenter: your vCenter IP or name
• FromCluster: the cluster you want to search through

Usage and Execution

Set-AutoEMM.ps1 -FromVMHostName myserver.domain.local -MaxMemAllowed 80 -FromCluster Production

The above line will get all the VMs from myserver.domain.local and distribute them to all hosts in the same cluster which has less than 80 percent utilized memory.

What if and during the migration the servers get utilized more than 80 percent?

Well, the script will evaluate the server before and after each VM being moved and do the proper calculation, so before moving the VM the script will calculate and check if the destination server will be over 80 percent, it will be excluded from being a destination for the migration. If no more hosts are available with less than 80 percent memory utilization (or whatever the value you set in the MaxMemAllowed ), the script will fail and stop, surely it will display something on the screen telling that no more hosts available.

Hope you enjoy this script and find it useful, please let me know by commenting or dropping me a message farisnt@gmail.com

If you notice any bug or issue, let me know :)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136

<# .Description Function To Move VMs From 1 host to other It Depend on the host Memory availibilty .Example Set-AutoEMM.ps1 -FromVMHostName MyVMHostServer -MaxMemAllowed 80 -FromCluster Production -FromVMHostName: The Source VMHost Server -MaxMemAllowed: Percentage of memory limit, default is 80, if the server memory load is 80% or more VMs wont be moved to this server and the script will search for another server to move the VM to if there is no more available server with less than 80% or the required value, the script will stop and show an error indicating the failure. -FromCluster: Cluster name where the hosts are exist This Script is for free you can update and use it as you want, Please add your name under the contributor Created By: -Faris Malaeb Contributor #Requires -Modules VMware.VimAutomation.Core #> param ( [parameter(mandatory = $True)] $FromVMHostName , [parameter(mandatory = $False)] [int]$MaxMemAllowed = "80", [parameter(mandatory = $True)] $vCenter , [parameter(mandatory = $True)] $FromCluster ) ################# General Variables $Finalresult=@() [bool]$ErrorFlag=$False ########## ########## Required module Import-Module VMware.VimAutomation.Core ############### Function Get-MostFreeServer ($NeededMemorySizeinGB) { #Below I will get the list of VMHost which are applicable for migration which include the following filter #Excluding the Move From Server #Server with a proper state # Have enough memory that match the user input parameter $CurrentServersLoad=Get-VMHost | where{ ($_.name -notlike $FromVMHostName) -and ($_.ConnectionState -notlike "*main*") -and ($_.ConnectionState -notlike "NotResponding") -and ($_.ConnectionState -notlike "Unknown") -and ((($_.MemoryUsageGB + $NeededMemorySizeinGB)/$_.MemoryTotalGB * 100) -lt $MaxMemAllowed) } | Sort-Object MemoryUsageGB if (($CurrentServersLoad).count -lt 1){ Write-Host "I am sorry, but there is no space for any migration..." # No Server available to hold the migration return "PS_FAIL_No_Resource" } Else{ #Yes there are at least one server that is good to host the VM Migration. return $CurrentServersLoad[0].Name } } #Checking the MaxMemAllowed User input to make sure that the user type a proper value Write-Host "Validating the input" -ForegroundColor Yellow if (($MaxMemAllowed -le 5) -or ($MaxMemAllowed -gt 99)){ Write-Host "I am sorry, but it seems that the MaxMemAllowed is not correct" -ForegroundColor Red Write-Host "The MaxMemAllowed cannot be as "$MaxMemAllowed -ForegroundColor Red break } Try { if ($global:DefaultVIServer.count -eq 0) { #If No Connection to VC found then the script will start a new connection, otherwise the -vCenter Parameter is ignored write-host "Please Connect to vCenter using Connect-VIServer first" -ForegroundColor Yellow $VC = Get-Credential -Message "Please type the username and password for vCenter" -ea Stop -UserName "administrator@vsphere.local" if ($VC -eq $null) { Write-Host -ForegroundColor Red "User press Cancel, I am leaving ..."; exit } Connect-VIServer -Credential $VC -Server $vCenter -ErrorAction Stop } Write-Host "Getting a list of VMs in host "$FromVMHostName -ForegroundColor Yellow #Get a list of VM on the server that should be migrated. only powered on VM.. I dont care about powered off VM $VMsInHost = Get-VMHost $FromVMHostName -ErrorAction Stop | Get-VM | where{ $_.PowerState -like "*On" } Write-Host "Total Number of VM to move is:"$VMsInHost.Count -ForegroundColor Yellow Foreach ($SingleVM in $VMsInHost){ #Creating an Object to store the progress report $VMnewLocation=new-object PSObject $VMnewLocation | Add-Member -NotePropertyName "VM Name" -NotePropertyValue $SingleVM.Name $VMnewLocation | Add-Member -NotePropertyName "Old Host" -NotePropertyValue $SingleVM.VMHost Write-host "Parsing VM " $SingleVM.Name -ForegroundColor Yellow #Call the function to get a list of possible servers to move the VM to. $Migrate=Get-MostFreeServer -NeededMemorySizeinGB $SingleVM.MemoryGB #PS_FAIL_No_Resource is a custom message will return from the function incase there is no available server to host the VM Migration. if ($Migrate -notlike "PS_FAIL_No_Resource"){ Write-Host "Moving "$SingleVM "to "$Migrate ",Please wait a few seconds" -ForegroundColor Yellow #Move the VM Move-VM -VM $SingleVM -Destination $Migrate -ErrorAction Stop sleep -Seconds 1 $VMnewLocation | Add-Member -NotePropertyName "New Host" -NotePropertyValue ((get-vm $SingleVM).vmhost.Name) Write-Host $SingleVM "is now in the following host" $VMnewLocation.'New Host' -ForegroundColor Yellow $Finalresult+=$VMnewLocation } Else { #No available server, the following message will be displayed. Write-Host "It seems there is no more servers to move the load, you can change the value of the MaxMemAllowed to allow more load... Nothing more to do." -ForegroundColor Red Write-Host "I cannot move "$SingleVM Write-Host "final report is:" $Finalresult $ErrorFlag=$true return } } } Catch [exception]{ # Any unexpected error will return here and stop the script execution. Write-Host $_.Exception.message -ForegroundColor Red $ErrorFlag=$true break } Finally{ #Checking the server is any more VM on it, if there is no VM, then place the server in maintenance Mode #Else, there should be an error indicating what is going on. if ($ErrorFlag -notlike $true){ if ((Get-VM | where{ ($_.host -like $FromVMHostName) -and ($_.PowerState -like "*on")}).count -eq 0) { Write-Host "All Done, will place the server in maintenance mode" -ForegroundColor Yellow Set-VMHost $FromVMHostName -State Maintenance Get-VMHost $FromVMHostName $Finalresult | ft } if ((Get-VM | where{ ($_.host -like $FromVMHostName) -and ($_.PowerState -like "*on")}).count -gt 0) { Write-Host "It seems that VM Migration not completed. maybe some resource issue" $Finalresult | ft } } }

#Use it on your own risk.

Get All Windows Services account that are configured to use a domain account in the trusted network

Hi
This small script will get all the services that are configured with a RunAs account
The service will be using several possible accounts like LocalSystem , Network Services.. and also a domain account.
The common thing is the service that will be using a domain account should have the UPN (FQDN) or SamAccountName (NetBIOS)

$Netbios=(Get-ADDomain).NetBIOSName #Get the Domain NetBIOS Name
$fqdn=(Get-ADDomain).dnsroot #Get the Domain FQDN Name

#The WMI Query that will be used
$WMIQuery="select * from Win32_Service where startname like '$Netbios%' or startname like '%$fqdn'"

#Getting computer list from AD, you can use the filter that fit your criteria, in my case, I have used a computer name as my filter criteria, you can use the search base.

Then I am executing the gwmi Get-WMIObject on the computer I got from the pipeline 

Get-ADComputer -Filter {name -like "*MyServers*"} | foreach {gwmi -ea SilentlyContinue -ComputerName $_.DNSHostName -Query$WMIQuery}  |ft -AutoSize SystemName,caption,startname 

The result should be something like this.

SystemName                       caption                                         startname        
----------                               -------                                             ---------        

HQ-SRV-N1       SQL Server Reporting Services                  Domain\report

Microsoft Counters

Following are the counters the Microsoft Service Support engineers rely on for monitoring.

These data are very important for your server monitoring, you can use them with Operation Manager or Windows Performance Console

LogicalDisk\% Free Space This measures the percentage of free space on the selected logical disk drive. Take note if this falls below 15 percent, as you risk running out of free space for the OS to store critical files. One obvious solution here is to add more disk space.

PhysicalDisk\% Idle Time This measures the percentage of time the disk was idle during the sample interval. If this counter falls below 20 percent, the disk system is saturated. You may consider replacing the current disk system with a faster disk system.

PhysicalDisk\Avg. Disk Sec/Read This measures the average time, in seconds, to read data from the disk. If the number is larger than 25 milliseconds (ms), that means the disk system is experiencing latency when reading from the disk. For mission-critical servers hosting SQL Server® and Exchange Server, the acceptable threshold is much lower, approximately 10 ms. The most logical solution here is to replace the current disk system with a faster disk system.

PhysicalDisk\Avg. Disk Sec/Write This measures the average time, in seconds, it takes to write data to the disk. If the number is larger than 25 ms, the disk system experiences latency when writing to the disk. For mission-critical servers hosting SQL Server and Exchange Server, the acceptable threshold is much lower, approximately 10 ms. The likely solution here is to replace the disk system with a faster disk system.

PhysicalDisk\Avg. Disk Queue Length This indicates how many I/O operations are waiting for the hard drive to become available. If the value here is larger than the two times the number of spindles, that means the disk itself may be the bottleneck.

Memory\Cache Bytes This indicates the amount of memory being used for the file system cache. There may be a disk bottleneck if this value is greater than 300MB.

Memory Bottleneck

A memory shortage is typically due to insufficient RAM, a memory leak, or a memory switch placed inside the boot.ini. Before I get into memory counters, I should discuss the /3GB switch.

More memory reduces disk I/O activity and, in turn, improves application performance. The /3GB switch was introduced in Windows NT® as a way to provide more memory for the user-mode programs.

Windows uses a virtual address space of 4GB (independent of how much physical RAM the system has). By default, the lower 2GB are reserved for user-mode programs and the upper 2GB are reserved for kernel-mode programs. With the /3GB switch, 3GB are given to user-mode processes. This, of course, comes at the expense of the kernel memory, which will have only 1GB of virtual address space. This can cause problems because Pool Non-Paged Bytes, Pool Paged Bytes, Free System Page Tables Entries, and desktop heap are all squeezed together within this 1GB space. Therefore, the /3GB switch should only be used after thorough testing has been done in your environment.

This is a consideration if you suspect you are experiencing a memory-related bottleneck. If the /3GB switch is not the cause of the problems, you can use these counters for diagnosing a potential memory bottleneck.

Memory\% Committed Bytes in Use This measures the ratio of Committed Bytes to the Commit Limit—in other words, the amount of virtual memory in use. This indicates insufficient memory if the number is greater than 80 percent. The obvious solution for this is to add more memory.

Memory\Available Mbytes This measures the amount of physical memory, in megabytes, available for running processes. If this value is less than 5 percent of the total physical RAM, that means there is insufficient memory, and that can increase paging activity. To resolve this problem, you should simply add more memory.

Memory\Free System Page Table Entries This indicates the number of page table entries not currently in use by the system. If the number is less than 5,000, there may well be a memory leak.

Memory\Pool Non-Paged Bytes This measures the size, in bytes, of the non-paged pool. This is an area of system memory for objects that cannot be written to disk but instead must remain in physical memory as long as they are allocated. There is a possible memory leak if the value is greater than 175MB (or 100MB with the /3GB switch). A typical Event ID 2019 is recorded in the system event log.

Memory\Pool Paged Bytes This measures the size, in bytes, of the paged pool. This is an area of system memory used for objects that can be written to disk when they are not being used. There may be a memory leak if this value is greater than 250MB (or 170MB with the /3GB switch). A typical Event ID 2020 is recorded in the system event log.

Memory\Pages per Second This measures the rate at which pages are read from or written to disk to resolve hard page faults. If the value is greater than 1,000, as a result of excessive paging, there may be a memory leak.

Processor Bottleneck

An overwhelmed processor can be due to the processor itself not offering enough power or it can be due to an inefficient application. You must double-check whether the processor spends a lot of time in paging as a result of insufficient physical memory. When investigating a potential processor bottleneck, the Microsoft Service Support engineers use the following counters.

Processor\% Processor Time This measures the percentage of elapsed time the processor spends executing a non-idle thread. If the percentage is greater than 85 percent, the processor is overwhelmed and the server may require a faster processor.

Processor\% User Time This measures the percentage of elapsed time the processor spends in user mode. If this value is high, the server is busy with the application. One possible solution here is to optimize the application that is using up the processor resources.

Processor\% Interrupt Time This measures the time the processor spends receiving and servicing hardware interruptions during specific sample intervals. This counter indicates a possible hardware issue if the value is greater than 15 percent.

System\Processor Queue Length This indicates the number of threads in the processor queue. The server doesn’t have enough processor power if the value is more than two times the number of CPUs for an extended period of time.

Network Bottleneck

A network bottleneck, of course, affects the server’s ability to send and receive data across the network. It can be an issue with the network card on the server, or perhaps the network is saturated and needs to be segmented. You can use the following counters to diagnosis potential network bottlenecks.

Network Interface\Bytes Total/Sec This measures the rate at which bytes are sent and received over each network adapter, including framing characters. The network is saturated if you discover that more than 70 percent of the interface is consumed. For a 100-Mbps NIC, the interface consumed is 8.7MB/sec (100Mbps = 100000kbps = 12.5MB/sec* 70 percent). In a situation like this, you may want to add a faster network card or segment the network.

Network Interface\Output Queue Length This measures the length of the output packet queue, in packets. There is network saturation if the value is more than 2. You can address this problem by adding a faster network card or segmenting the network.

Process Bottleneck

Server performance will be significantly affected if you have a misbehaving process or non-optimized processes. Thread and handle leaks will eventually bring down a server, and excessive processor usage will bring a server to a crawl. The following counters are indispensable when diagnosing process-related bottlenecks.

Process\Handle Count This measures the total number of handles that are currently open by a process. This counter indicates a possible handle leak if the number is greater than 10,000.

Process\Thread Count This measures the number of threads currently active in a process. There may be a thread leak if this number is more than 500 between the minimum and maximum number of threads.

Process\Private Bytes This indicates the amount of memory that this process has allocated that cannot be shared with other processes. If the value is greater than 250 between the minimum and maximum number of threads, there may be a memory leak.

Should I defrag my Guest OS?

A very good post that answer this question,
http://blogs.vmware.com/vsphere/2011/09/should-i-defrag-my-guest-os.html#comment-203323

But in general, NO, you should not defrag the Guest OS:

1. Defragmentation also generates more I/O to the disk. This could be more of a concern to customers than any possible performance improvement that might be gained from the defrag.
2. There isn't any noticeable improvement in performance after a defragmentation of Guest OSes residing on SAN or NAS based datastores. (As VM Says)