Sunday, August 2, 2020

Powershell: VMWare vCenter Standard, Move all VM from a host and place it in maintenance mode

I have VMWare vCenter standard edition, and there is a limitation with placing the server in maintenance mode, as it will never complete unless you move all the VM from that host to another host and then place the server in maintenance mode. something I really don't like, its a small basic regular thing any admin need to do, maybe updating the server or whatever.
When it comes to manual work (which I hate so much), you will need to distribute your VMs based on the host which has more free memory and also healthy. the challenge I have is I have about 30 servers all running the standard edition, and I need to update the hosts, so redistribute the load and check which server has more memory, and Bla Bla Bla, of boring things, so I got the idea to automate this.
The below script (can be downloaded from here too) will do the following:
  • Import the VMWare module and connect to the VI Server.
  • Get a list of all the VM in the host that should be placed in maintenance mode.
  • for each virtual machine, find the proper server to place it (based on the most server which has the freest memory).
  • Redistribute the load.
    • if no server is available with enough memory to have the VM, the script will let you know and won't overcommit your servers, unless you want so.
  • After all is done, will provide you with a basic report about each VM, the old host, and the new host.
  • place the server in maintenance mode.
Parameter and Overcommit prevention
The script requires some parameter to be filled before it can do the magic:
  • FromVMHostName: The name of the server you want to move the VM from and place it in maintenance mode, Please Note that the server name should be exactly as the one registered in your vCenter, so if you are using FQDN, pass it here too.
  • MaxMemAllowed: The Percentage of the total free memory of the destination server before being excluded from the selection, the default value is 80, so no VM will be shipped to any server which has more than 80 percent of utilized memory.
  • vCenter: your vCenter IP or name
  • FromCluster: the cluster you want to search through
Usage and Execution

Set-AutoEMM.ps1 -FromVMHostName myserver.domain.local -MaxMemAllowed 80 -FromCluster Production

The above line will get all the VMs from myserver.domain.local and distribute them to all hosts in the same cluster which has less than 80 percent utilized memory.

What if and during the migration the servers get utilized more than 80 percent?

Well, the script will evaluate the server before and after each VM being moved and do the proper calculation, so before moving the VM the script will calculate and check if the destination server will be over 80 percent, it will be excluded from being a destination for the migration. If no more hosts are available with less than 80 percent memory utilization (or whatever the value you set in the MaxMemAllowed ), the script will fail and stop, surely it will display something on the screen telling that no more hosts available.

Hope you enjoy this script and find it useful, please let me know by commenting or dropping me a message farisnt@gmail.com

If you notice any bug or issue, let me know :)



  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
<# .Description
    Function To Move VMs From 1 host to other 
    It Depend on the host Memory availibilty 
    .Example
    Set-AutoEMM.ps1 -FromVMHostName MyVMHostServer -MaxMemAllowed 80 -FromCluster Production
    -FromVMHostName: The Source VMHost Server
    -MaxMemAllowed: Percentage of memory limit, default is 80, if the server memory load is 80% or more VMs wont be moved to this server
                    and the script will search for another server to move the VM to
                    if there is no more available server with less than 80% or the required value, the script will stop and show an error
                    indicating the failure.
    -FromCluster: Cluster name where the hosts are exist
    This Script is for free you can update and use it as you want, Please add your name under the contributor
    Created By: 
     -Faris Malaeb
    Contributor
    #Requires -Modules VMware.VimAutomation.Core
#>
param (
	[parameter(mandatory = $True)]
	$FromVMHostName ,
	[parameter(mandatory = $False)]
	[int]$MaxMemAllowed = "80",
	[parameter(mandatory = $True)]
	$vCenter ,
	[parameter(mandatory = $True)]
	$FromCluster 
)

################# General Variables
$Finalresult=@()
[bool]$ErrorFlag=$False
##########
########## Required module
Import-Module VMware.VimAutomation.Core
###############

Function Get-MostFreeServer ($NeededMemorySizeinGB) {

        #Below I will get the list of VMHost which are applicable for migration which include the following filter
          #Excluding the Move From Server
          #Server with a proper state
          # Have enough memory that match the user input parameter
    $CurrentServersLoad=Get-VMHost |  where{ ($_.name -notlike $FromVMHostName) -and ($_.ConnectionState -notlike "*main*") -and ($_.ConnectionState -notlike "NotResponding") -and ($_.ConnectionState -notlike "Unknown") -and ((($_.MemoryUsageGB + $NeededMemorySizeinGB)/$_.MemoryTotalGB * 100) -lt $MaxMemAllowed) } | Sort-Object MemoryUsageGB  
    if (($CurrentServersLoad).count -lt 1){ Write-Host "I am sorry, but there is no space for any migration..."
    # No Server available to hold the migration
    return "PS_FAIL_No_Resource"
    }
    Else{
    #Yes there are at least one server that is good to host the VM Migration.
    return $CurrentServersLoad[0].Name
    }   

}
#Checking the MaxMemAllowed User input to make sure that the user type a proper value
Write-Host "Validating the input" -ForegroundColor Yellow
if (($MaxMemAllowed -le 5) -or ($MaxMemAllowed -gt 99)){
Write-Host "I am sorry, but it seems that the MaxMemAllowed is not correct" -ForegroundColor Red
Write-Host "The MaxMemAllowed cannot be as "$MaxMemAllowed -ForegroundColor Red
break
}


Try
{
	
	if ($global:DefaultVIServer.count -eq 0)
	{
		#If No Connection to VC found then the script will start a new connection, otherwise the -vCenter Parameter is ignored
		write-host "Please Connect to vCenter using Connect-VIServer first" -ForegroundColor Yellow
		$VC = Get-Credential -Message "Please type the username and password for vCenter" -ea Stop -UserName "administrator@vsphere.local"
		if ($VC -eq $null) { Write-Host -ForegroundColor Red "User press Cancel, I am leaving ..."; exit }
		Connect-VIServer -Credential $VC -Server $vCenter -ErrorAction Stop
		
	}
	
	Write-Host "Getting a list of VMs in host "$FromVMHostName -ForegroundColor Yellow
	#Get a list of VM on the server that should be migrated. only powered on VM.. I dont care about powered off VM
	$VMsInHost = Get-VMHost $FromVMHostName -ErrorAction Stop | Get-VM | where{ $_.PowerState -like "*On" }
	Write-Host "Total Number of VM to move is:"$VMsInHost.Count -ForegroundColor Yellow
	Foreach ($SingleVM in $VMsInHost){
        #Creating an Object to store the progress report
	    $VMnewLocation=new-object PSObject
        $VMnewLocation | Add-Member -NotePropertyName "VM Name" -NotePropertyValue $SingleVM.Name
        $VMnewLocation | Add-Member -NotePropertyName "Old Host" -NotePropertyValue $SingleVM.VMHost
        
		Write-host "Parsing VM " $SingleVM.Name -ForegroundColor Yellow
        #Call the function to get a list of possible servers to move the VM to.
        $Migrate=Get-MostFreeServer -NeededMemorySizeinGB $SingleVM.MemoryGB
        #PS_FAIL_No_Resource is a custom message will return from the function incase there is no available server to host the VM Migration.
            if ($Migrate -notlike "PS_FAIL_No_Resource"){
                Write-Host "Moving "$SingleVM "to "$Migrate ",Please wait a few seconds" -ForegroundColor Yellow
                #Move the VM
                Move-VM -VM $SingleVM -Destination $Migrate -ErrorAction Stop
                sleep -Seconds 1
                $VMnewLocation | Add-Member -NotePropertyName "New Host" -NotePropertyValue ((get-vm $SingleVM).vmhost.Name)
                Write-Host $SingleVM "is now in the following host" $VMnewLocation.'New Host' -ForegroundColor Yellow
                $Finalresult+=$VMnewLocation 
            }
		    Else {
                    #No available server, the following message will be displayed.
                Write-Host "It seems there is no more servers to move the load, you can change the value of the MaxMemAllowed to allow more load... Nothing more to do." -ForegroundColor Red
                Write-Host "I cannot move "$SingleVM
                Write-Host "final report is:"
                $Finalresult
                $ErrorFlag=$true
                return
                }
    }
}



Catch [exception]{
# Any unexpected error will return here and stop the script execution.
	Write-Host $_.Exception.message -ForegroundColor Red
    $ErrorFlag=$true
    break

}

Finally{
        #Checking the server is any more VM on it, if there is no VM, then place the server in maintenance Mode
        #Else, there should be an error indicating what is going on.
        if ($ErrorFlag -notlike $true){
            if ((Get-VM | where{ ($_.host -like $FromVMHostName) -and ($_.PowerState -like "*on")}).count -eq 0) { 
                Write-Host "All Done, will place the server in maintenance mode" -ForegroundColor Yellow
                Set-VMHost $FromVMHostName -State Maintenance
                Get-VMHost $FromVMHostName
                $Finalresult | ft
            }
            if ((Get-VM | where{ ($_.host -like $FromVMHostName) -and ($_.PowerState -like "*on")}).count -gt 0) { 
                 Write-Host "It seems that VM Migration not completed. maybe some resource issue"
                 $Finalresult | ft
            }
        }
}

#Use it on your own risk.

Tuesday, July 21, 2020

Powershell: Dump the console output to a file (Output redirection)


Let's think about the following case, I want to copy files from the following directory C:\MySource to D:\MyDestination plus adding a log file about the operation using Powershell. 
The easy answer is using the following command 

copy-Item -Path C:\MySource -Destination D:\MyDestination -Recurse | Out-File -FilePath D:\MyDestination\CopyLog.txt

The line above will copy the files from C:\MySource to D:\MyDestination and use the Out-File to redirect the console output to a file (output redirection), but when opening the file... the file is totally empty, so where is the output.

Actually, if we remove the out-file and checked the console output from the copy command, the result will be empty too... so how can we write the log ?!





There is two way we can use to write a log, one will be a bit complex by creating a function that will do the copy and the log writing, but I won't use this approach as there is a much easier way to do it.

Output redirection can be achieved by using the asterisk and the greater than sign *> 
But before trying it check this wonderful table about the things which can replace the asterisk [Microsoft.com]

Stream #DescriptionIntroduced in
1Success StreamPowerShell 2.0
2Error StreamPowerShell 2.0
3Warning StreamPowerShell 3.0
4Verbose StreamPowerShell 3.0
5Debug StreamPowerShell 3.0
6Information StreamPowerShell 5.0
*All StreamsPowerShell 3.0
So by replacing the asterisk (*) with the appropriate number, we will get only the required stream. For example, if you want to get only the errors that occurred during the copy operation, the code will be like this
Copy-Item -Path C:\MySource -Destination D:\MyDestination -Recurse 2> D:\MyDestination\CopyLog.txt

The line above and by replacing the asterisk with the number 2, it will only redirect the output of errors to the file, which in my case, something similar to this:

Copy-Item : An item with the specified name D:\MyDestination\MySource already exists.
At line:1 char:1
+ Copy-Item -Path C:\MySource -Destination D:\MyDestination -Recurse 2> ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ResourceExists: (D:\MyDestination\MySource:String) [Copy-Item], IOException
    + FullyQualifiedErrorId : DirectoryExist,Microsoft.PowerShell.Commands.CopyItemCommand


Hope you find this informative.
If you have any comment or question, leave it in the comment section below 
Have a good day.
 

Monday, June 1, 2020

Convert to GPT from MBR without losing any data (work on Exchange DAG)

Hi,
Recently my exchange server 2016 DAG was running out of space, my database reaches 2 TB and the MBR cannot be extended anymore, so I need to convert to GPT. I tried Diskpart, which is a good builtin tool using the following command, but it fails and returns the following error

Virtual Disk Service error:
The specified disk is not convertible. CDROMs and DVDs
are examples of disks that are not convertable.

Based on some googling and reading it seems that this command will work fine if I clear the disk (format it) which cause a full data lost on the disk, and this is not applicable on my case, I have 2 exchange server 2016 with 2TB of data on one of the disk, resyncing the data will take several days.

The Fix
I find this super tools
https://sourceforge.net/projects/gptgen/files/gptgen/v1.1/gptgen-1.1.zip/download
Just download it and place it on drive C: or whatever drive other than you want to convert and run the following command gptgen.exe, the output will be

where device_path is the full path to the device file,
e.g.\\.\physicaldrive0.

Available arguments (no "-wm"-style argument combining support):
-b , --backup : write a backup of the original MBR to
-c nnn, --count nnn: build a GPT containing nnn entries (default=128)
-h, --help, --usage: display this help message
-k, --keep-going: don't ask user if a boot partition is found
-m, --keepmbr: keep the existing MBR, don't write a protective MBR
-w, --write: write directly to the disk, not to separate files

One of the required parameters is the disk number and we can get it from Diskpart, using the following command
> Diskpart
>> List disk


You need to know the disk number you want to convert. Let's assume that the disk you want to convert is disk 1, just remember the number, now exit Diskpart and navigate to GPTGen and execute the following command
gptgen.exe -w \\.\physicaldrive1

The -w parameter is used to write GPT table and is required in the converting process, just make sure to write the correct disk number, in my case, it's 1.
The output will be similar to


That easy, GPT without losing any data
No need to reboot, even though I recommend to reboot and make sure that everything is working fine. Also, I recommend before doing, try to minimize IO on the disk, place the server in maintenance mode, turn the services off, and also backup your data if you can. Gptgen have the option to backup the current MBR before converting it.

Enjoy :D



Monday, January 27, 2020

PSscript to get list of share and perform test on them


The Problem
In my environment, I got about 200 VM and some of the servers physical servers and others in a remote site. One of the critical security issues is the open share, where the users can Read/Write/Delete data, usually, these shares are created due to a misconfigured share, or shares that are intended to be temporary and then suddenly these shares became critical and placed for production.
Power users who have the ability to create share and manage permission may forget to set proper permission, sometimes they follow the easy way of Grant Everyone everything in both NTFS and share, which expose all the files to all the users in some cases these files are critical files.
There are several scripts on the Internet to search for share folder in the servers, but usually the result will be slow, incompatible, or with a lot of noise, such as Admin share. Also, another challenge is even after getting the share list, how can the admin confirm if a user can Access/Read/Write/Delete files in the shared folder without going though the long list of ACL for both share and NTFS?!

Powershell Script
To address this issue, I created a Powershell script that will do the following
  1. Get a list of all AD Computer using filter operatingsystem -like "*server*" so only servers are retrieved.
  2. Test the computer reachability and ensure that it's responding
  3. For each server matching the filter, get the share list using WMI query. The share list only include the Folders and Admin Share such as Admin$, C$, IPC$ ..., excluding printers
  4. The Script can read cluster share
  5. If Impersonation parameter is set to $true then the script will request to use another user credentials to perform Access,Write, Read, Delete on each share, usually these credential are for a limited user:
    • Create a PSDrive and assign any available letter to the drive, this will use any available letter. At least make sure that you have 1 available letter :)
    • If the Map was success, then the users can Access and the Access is set to Yes, else, all the other test will stop as I assume that if the user don't have the right to access a folder so this user wont have write, read or delete access permissions.
    • If the Access was success, then the script proceeds in Writing a file to the destination as AdcciTestPSWrite.txt
    • If The Write was success, the script will proceed in Reading it.
    • The last step is to remove the created file, and the result will be updated
      • The script will not remove random files, it will only try to remove the file it wrote as part of the test.
Limitations:

  1. The script may report that the user has full access (Access, Read, Write, Delete) in a folder where Creator Owner has full control, such as Folder Redirection, Home Folder. I will fix this issue in the coming version. But for now its good :).
  2. There might be a duplicated result when the file server cluster have muti rules. also will work on fixing this issue on the next version. 
Output
The output is a CSV file, you can format it in the way you want that show the folder name and a table of the permission the impersonated user can do


Download the scrip, try it and let me know

https://drive.google.com/drive/folders/1NL8YqCI1jBWzhtcWNcMCRihjp2Tn3nJa?usp=sharing


You can reach me farisnt@gmail.com




Friday, March 15, 2019

Could not find this item

Recently I noticed some wired behavior of my test file server.
Whenever I remove a folder, the folder icon remains there, even though it deleted, but it's still showing.
if I double click on it, the folder open and then automatically exit "Go back one level up".
If I try to remove the folder again, I got the message below
Using Sysinternal Procmon.exe, I got the following message when trying to remove the folder. NAME NOT FOUND.
I noticed also using the CMD the folder name is ending with white space.
My folder name is VDI1, using the CMD with auto-complete I got this:
rd "vdi1 "

FIX:
To remove the file use the following command to remove the folder
rd "\\?\d:\Shares\Home\vdi1 "
use the TAB while typing the path for auto-complete, as the end of the folder contains white space.
using this way. I was able to remove the folder.





Friday, March 1, 2019

Get MyDocument and Desktop Folder size for all computer in the network


This Script will read the computer list from a CSV file and scan all the computer in the list to get the current size for MyDocumet and Desktop folder in the remote PC.
This script is very useful to size your VDI environment and get an idea about the expected user profile, especially when all the users are using regularly PC and storing all their content on the desktop and MyDocuments
I chose to get the information from the CSV file as I request from the network team for the most recent computers list which is connected to the network. the active directory can contain some old computer object or test pc, which will increase the percentage of failed scan. Anyway, if you prefer to get the list from AD you still can do it by using 
Get-ADComputer -filter * | Export-Csv -NoTypeInformation -Path c:\temp.csv




#############################################################
# This Script is written by Faris Malaeb
# Feel free and contact me farisnt@yahoo.com, farisnt@gmail.com
# http://farisnt.blogspot.com/
#
#This script will scan the network and get the users desktop and mydocument profile size
#This Script will help you in giving you an idea about the users data
# I wrote this script as I needed to know how much users have in their desktop and MyDocument folder
#These data should be moved to the servers as a part of VDI Project implementation.
#
#
# Get the computer objects
# Tested on Windows 10
#
#To use the Script .\Get-DesktopnDocs.ps1 -FullPathForSoruce C:\PCList.csv -ResultDestination C:\MyFinzalResult.csv
#You can use the -Verbose parameter for more information during the runtime
#############################################################


param(
[parameter(Mandatory=$False)]$FullPathForSoruce="C:\PCs.csv",
[parameter(Mandatory=$False)]$ResultDestination="C:\MyFinzalResult.csv"
)
$Collection=Get-Content -Path $FullPathForSoruce
#The final result will be added to this array
$FullResult=@()

try{
    Write-Host "Validating the destination path $ResultDestination" -ForegroundColor Green
    Add-Content -Path $ResultDestination -Value "temp"
    Remove-Item $ResultDestination
    }
catch{
    Write-Host "Failed to write to path $ResultDestination... Exiting " $_.exception.message
    break
    }


#Looping through all the computer object
foreach($SinglePC in $Collection){


#Checking if the computer is alive or dead
    try{ (Test-Connection -ComputerName $SinglePC -Count 1 -ErrorAction Stop |Out-Null)

        #I need to have an object which contain ComputerName, Deskop, MyDocument and Username properties
        #I will name this object as TotalResult
        $totalresult =New-Object psobject
        $totalresult | Add-Member -NotePropertyName ComputerName -NotePropertyValue ""
        $totalresult | Add-Member -NotePropertyName Desktop -NotePropertyValue ""
        $totalresult | Add-Member -NotePropertyName MyDocument -NotePropertyValue ""
        $totalresult | Add-Member -NotePropertyName username -NotePropertyValue ""
        $totalresult.ComputerName= $SinglePC

        #Each properties of the newly created object will contain the output from a command invokation
        #Getting the username of the active user from the remote machine.
        #I am getting the username through reading the running process which run under the user context.
        #My Domain name is Domain-local, and as the client OS only support one login, so all the process should have the current active user
        #I used the $env:USERDOMAIN System Variable to get the domain name, lets assume that my domain is named as Domain-Local
        #After getting the list of processes running under the username Domain-local\..., I select the first Item in the array and removing the domain name
        #This will only return the username, and then store the result in the $TotalResult.Username Properties
        $ScriptToExecute={
        ((Get-Process -IncludeUserName |where {$_.username -like "$env:USERDOMAIN*"})[0]).username.substring(([int]$env:USERDOMAIN.Length+1))
        }
        $totalresult.username=invoke-command -ComputerName $SinglePC -ScriptBlock $ScriptToExecute

        #The command which will be invoked on the remote computer
        $ScriptToExecute= {
        $UserNViaProcess =$args[0] #read the argument that will be passed to the script block through the invoke-command -ArgumentList property
        #The Lines below will get the Desktop items which have the extension below.
        #the Measure-Object will calculate the value a group based on numeric property, and in our case the value I want to measure is the Length
        #After the calculation I devide the value by 1GB to get the result in GB
        ((Get-ChildItem -Path "C:\users\$UserNViaProcess\Desktop" -Recurse -Include @("*.docx","*.xlsx","*.pptx","*.txt","*.mmp","*.jpg","*.png","*.pdf","*.vsdx")|Measure-Object -Sum Length).Sum /1GB)
        }
        #Executing the remote command invoke and passing the arguament list, which is the username
        $totalresult.Desktop=invoke-command -ComputerName $SinglePC -ScriptBlock $ScriptToExecute -ArgumentList $totalresult.username

                                                                           
        $ScriptToExecute={
        $UserNViaProcess =$args[0]
        ((Get-ChildItem -Path "C:\users\$UserNViaProcess\Documents" -Recurse -Include @("*.docx","*.xlsx","*.pptx","*.txt","*.mmp","*.jpg","*.png","*.pdf","*.vsdx") |Measure-Object -Sum Length).Sum /1GB)
        }
        $totalresult.MyDocument=invoke-command -ComputerName $SinglePC -ScriptBlock $ScriptToExecute -ArgumentList $totalresult.username
 
        #Display the result in the console
        $totalresult
        #Adding the Result to the array
        $FullResult+=$totalresult

        }


    Catch {

            Write-Host "I failed on " $SinglePC " Error is " $_.exception.message
         }

    Finally{
    $FullResult | Export-Csv $ResultDestination -NoTypeInformation

        }
}