Nutanix: fatal mounting installer media
Last week , we were doing the foundation on NX-1365-G6 block. The foundation process hangs at 26% with error fatal mounting installer media. When this happens, the Nutanix nodes are being powered off. I have attached 2 screenshots below that depicts the problem we faced.
You could see it in the images , that the IPMITool is trying to restart the server and failing to do so.
Therefore , to overcome this situation , we logged in to IPMI on each node and did a Uni Reset & Factory Default via the Maintenance Menu. Thereafter we restarted the foundation from scratch and it got completed successfully.
AsBuilt Report for VSphere
Hi Folks
Until recent years , I was struggling to build a proper AsBuilt Document for VSphere environments. As the manual process requires capturing screenshots and time consuming word document preparations.
Last week , I came across 2 blogs talking about this AsBuilt tool for VMware which turned out to be very handy and must have tool for VMware installations .
For those who want to read more about this tool, could visit the 2 blogs that are listed at the bottom of this page.
You need Windows PowerShell. Once you are ready with the PowerShell run the below commands to build your AsBuilt document .
1) Install the PSCribo Module
#Install-Module PSCribo
2)Download the AsBuilt PowerShell Scripts via https://github.com/tpcarman/As-Built-Report
2.1)Extract it to a Folder
#Import-Module C:\As-Built-Report-dev\AsBuiltReport.psd1
3)Install PowerCLI Module
#Find-Module -Name VMware.PowerCLI
#Install-Module -Name VMware.PowerCLI
3.1)Run the below command to bypass SSL warning for VCenter/ESXi
#Set-PowerCLIConfiguration -InvalidCertificateAction Ignore -Confirm:$false
4) Below command will create the Report
New-AsBuiltReport -Target vcenterip -Credential (get-credential) -Type vSphere -Format HTML,Word -TimeStamp imeStamp -Healthchecks -AsBuiltConfigPath C:\As-Built-Report-dev\Src\Public\Reports\vSphere\vsphere.json
Source:
https://www.timcarman.net/as-built-report/
As Built Report – working with it in my lab
Nutanix AOS Upgrade Tips
Recently we were upgrading our Nutanix Cluster which was running an AOS version 4.5.2.3 to the latest 5.9. The process was seamless and non interruptive. I have listed the commands we have used along with Nutanix Engineer during the process for future reference.
Initial Check prior to AOS Upgrade
- ncli cluster info
- ncli host ls
- ncli ru ls
- ncli ms ls
- ncc –version
- cluster status | grep -v UP
- nodetool -h 0 ring | grep -i normal | wc -l
- svmips | wc -w
Once the output of above checks are fine Use the Software Upgrade feature from PRISM to upgrade the AOS.
To check the upgrade / pre-upgrade status and on which node is being picked up.& Confirm the versions after upgrade
- allssh ls -ltra ~/data/logs | grep -i preupgrade
- tail -F ~/data/logs/preupgrade.out
- use upgrade_status (to verify the status , less verbose mode)
- ncli –version
- stargate –version
- watch -d genesis status (to check the services status after the CVM reboot)
Optional: To delete the previously uploaded ISO
- cd ~/software_downloads/nos ( use it with allssh to run it on all the CVM’s)
Finally after AOS upgrade sometime the curator replication process and it takes some time to complete . Until it completes we cannot proceed with the other update , thus you could check via the below command;
- curator_cli get_under_replication_info
How to Capture & Analyze Network Traffic on ESXi
Being an ESXI Implementer or an Administrator , you may come across some situations where you need to make your hands dirty 🙂 , with deep network troubleshooting. I had a similar situation few months ago , which I would like to share it in this post.
We deployed the Horizon View (for VDI) in one of our customer’s ESXi Cluster ( 8 Nodes) environment, The Desktop users were complaining about they were not able to specific network .
Thus to further investigate we swapped the Physical Adapter to the on-board BroadCom cards (1Gps). Then we were able to re-establish the network. We thought to engage the VMware Support with the intention to find out the root cause and get a permanent fix. The VMware support was pretty awesome and they were able to nail it very quickly.
First they used the two built-in commands on ESXI , which are
- pktcap-uw (To capture the Network Packets)
- tcpdump-uw ( To read the captured Packets)
They ran the below commands on both the NIC cards to initially capture the traffic.
- pktcap-uw –uplink vmnic0 –dir 0 –mac 00:00:00:00:00:00 —vlan 18 -o /tmp/f.pcap
uplink – Name of the VMnic
dir – 0 means RX Traffic
mac – MAC address of the machine which you are troubleshooting
vlan – The VLAN ID
Thereafter we read the output of the above command using
- tcpdump-uw -ner /tmp/f.pcap
By comparing the output from both NIC’s we were able to narrow down the problem to the Mellanox cards. when tagged traffic passed by on a Mellanox Network Card (10 Gbps), the reply packet was not being tagged with the proper VLAN ID causing disruption to the network traffic.
Good Luck
Muralee
Veeam Backup Repository Sizing
Folks who wants to size the Backup Repository for their Veeam Deployment , could make use of the online sizing tool below for their calculations.
It is very user friendly and output is much descriptive.
Good Luck.
How to create an O365 Mailbox when there is no On-Prem Exchange Servers.
In some cases the IT department decomission the On-Prem Exchange Server after migrating the mailboxes to O365. (For detailed steps for uninstallation of Exchange Please refer my previous article : https://vands.pro/2018/11/06/how-to-decomission-on-prem-exchange-server-after-migrating-the-mailboxes-to-o365/.
OK, Coming back to mailbox creation ; All these steps need to be done in the ADUC
+ Create an AD User.
+ Type the email address in the email field.
+ Go to the Account Tab and select the correct domain name.
+ In the Attributes Editor modify the two parameter’s as below
proxyAddresses: SMTP: myemail@email.com
targetAddress: SMTP: myemail@companyname.onmicrosoft.com
+ Either perform a manual sync or wait for the next schedule.
+ After the Sync is completed you will be able to see the user in O365 Portal and need to assign the Exchange License to complete
the mailbox creation.
Credits: https://c7solutions.com/2014/07/creating-mailboxes-in-office-365-when-using-dirsync
How to Decomission On-Prem Exchange Server after migrating the Mailboxes to O365
Few weeks ago, we had project to migrate the Exchange 2010 Mailboxes to O365. After the migration completed we had to remove the On-Prem Exchange Servers(Even though it is not a supported scenario from Microsoft , still you could do this.) This will lead you to to adopt non standard methods when creating new mailboxes(Refer my post https://vands.pro/2018/11/06/how-to-create-an-o365-mailbox-when-there-is-no-on-prem-exchange-servers/ for detailed steps). Also if you still needed an Exchange Server in your premises to avoid complication you could contact MS O365 Support to obtain a free Exchange Hybrid License and install an Exchange Server.
Coming back to the original goal of this article you need to follow the below steps if you have decided to remove the Exchange Servers; Also note that uninstallation of Exchange will not impact the O365 mailboxes in any manner.
+ By this time you should have changed the MX and related DNS records on
the on-prem DNS and Public DNS Servers.
+ Set the AD Internal URL to null
Get-ClientAccessServer | Set-ClientAccessServer
-AutoDiscoverServiceInternalUri $Null
+ Then disable the AD Sync(Temporarily)
Set-MsolDirSyncEnabled -EnableDirSync $false
(Get-MsolCompanyInformation).DirectorySynchronizationEnabled – To
view the current status.
+ Open Exchange Shell in one of the Exchange Server(uninstallation should
be started in the order of MBOX,CAS,HUB & Edge)
+ Remove default Public folders
Get-PublicFolder “\” -Recurse -ResultSize:Unlimited |
Remove-PublicFolder -Recurse -ErrorAction:SilentlyContinue
+ Remove system Public folders
Get-PublicFolder “\Non_Ipm_Subtree” -Recurse -ResultSize:Unlimited |
Remove-PublicFolder -Recurse -ErrorAction:SilentlyContinue
+ Remove Offline Address Book
Get-OfflineAddressBook | Remove-OfflineAddressBook
+ Remove send connectors
Get-SendConnector | Remove-SendConnector
+ Remove Public Folder Database
Get-PublicFolderDatabase | Remove-PublicFolderDatabase
+ + Remove Arbitration Mailbox
Get-Mailbox -Arbitration | Disable-Mailbox -Arbitration -DisableLastArbitrationMailboxAllowed
+ Disable / Delete All non migrated Mailboxes
Get-Mailbox | Disable-Mailbox
+ Open CMD as Administrator and Go to the bin folder in the Exchange Installation Folder in your C: drive(Depend on your environment)
setup.com /m:unistall.
+ Once all the Exchange Roles are uninstalled disjoin the servers from the AD.
+ Re-run the AD Sync Tool and remove the Tick on Exchange Hybrid Configuration.
+ Re-enable the AD Sync
Set-MsolDirSyncEnabled -EnableDirSync $false
Credits: https://www.itpromentor.com/remove-hybrid-keep-sync/
Good Luck Guys.
