Backup and AIR replication is slow after upgrading Netbackup Enterprise Server and Appliance to 8.1 /3.1

it is a very common behavior that after upgrading NetBackup appliance to 3.1 backup and replication will slow, well basically there is not an issue it is just because of MSDP security enhancement with the new version.

MSDP uses the rolling data conversion mechanism to convert Blowfish encrypted
data to AES-256 encrypted data, and MD5-like fingerprints to SHA-512/256
fingerprints, in parallel. There are two modes of data conversion: normal mode and
fast mode.

Normal Mode:- 

By default, the data conversion process starts in a normal mode for an upgraded system. Similar to compaction, the data conversion runs only when no backup, restore, or Content Router Queue Processing (CRQP) jobs are active. In the normal mode, the time for data conversion depends on the following factors:
■ Total size of the storage
■ Power of the CPU
■ The workload on the system
Data conversion in the normal mode may take a longer time. Veritas tests in a controlled environment showed that for a single 1-TB mount point, the conversion speed is about 50MB/s in normal mode

That why appliance 5220 is not recommended to upgrade to 3.1

Fast Mode:- 

In the fast mode, the data conversion disables cyclic redundancy checks and compaction. The rolling data conversion runs while backup, restore, duplication or CRQP jobs are active. Veritas tests in a controlled environment showed that for a single 1-TB mount point, the conversion speed is about 105MB/s in fast mode.


To start the data conversion process ( by default it starts after upgrade)

install_path\Veritas\pdde\Crcontrol.exe –dataconverton

(applinance) UNIX: /usr/openv/pdde/pdcr/bin/crcontrol –dataconverton

To determine the mode of data conversion and the conversion progress :- 

install_path\Veritas\pdde\Crcontrol.exe –dataconvertstate
(Appliance) UNIX: /usr/openv/pdde/pdcr/bin/crcontrol –dataconvertstate


compare to the fast mode normal mode is very slow hence if your backup loads depend to change conversation mode, run the below command on appliance, changing mode can be done on fly, no need to stop NetBackup services

install_path\Veritas\pdde\Crcontrol.exe –dataconvertmode 1


after the upgrade, I would also suggest to contact Veritas support and get NBAPP_EEB_ET3935219- , it has may fixes related to MSDP converstion and replication.


Netbackup Upgrade Stuck with error System_Call_Fastpath+0x16/0x1b

Netbackup Appliance Upgrade from 7X to 8.1 stuck with the error “System_Call_Fastpath+0x16/0x1b”. in my case Appliance upgrade was stuck after doing self-test and Netbackup software and appliance software upgrade. what I observed is that upgrade is completed successfully, however, there is some issue with operating system kernel (kernel panic)


In my case, I had waited for some time but that doesn’t help me, I have rebooted the appliance and able to load the operating system and all the NetBackup services were started successfully . in order to make sure issue resolved, I had rebooted appliance two times and there is no error

It is just one time error, after rebooting issue is resolved.

Since the Netbackup 8.1 is still latest there are only a few articles is available for the same error.

Nutanix Prism Central One Click Deployment (AOS 5.5)

I must say when it comes to Make the things easily and provide an amazing combination of the technology Nutanix team outstanding!

Nutanix is not a just hyper-converged system which simply converges the local storage to the distributed storage fabric (NDFS)but it provides the amazing platform where we can manage the complete infrastructure like a private cloud,

Prism Elements is signal web-based screen where we manage and monitor the single NUTANIX cluster, PE(Prism Elements) Runs on each C.V.M (Controller VM)


With the Prism Central (PC) ,

Prism Central is adding one  extra layer to the Prism Element , with the PC(Prism Central) we can manage + Monitor + also can plan future resource requirement for Mutiple Nutanix Cluster


Prism Central deployment available with the one-click deployment feature, below are the steps

  1. Download the installation binaries and Metadata file from the Nutanix Portal


  1. Access the Prism Element (HTTPS://Cluster,CVMIP:9440)
  2. Under the Prism Center Select Register or create new
  3. Select deploy New Prism Central Instance


5. Upload the downloaded files as below


6. Once uploaded , select install


7. Assign hostname and ip address to the prism central VMs

8. Wait the process to complete , installation process may take 30 min to 1 hours

9. nce the process complete ,register the prism central from Prism element 

10 To access the Prism Central lick the Lunch on prism element or acess the PC ip address with port 9440


Note:- Prism Central Default credentials

User Name :- admin

Password :- Nutanix/4u

Migrate Enterprise Vault Database to New SQL Server

Before beginning of database migration note down below important things

  1. SQL Server Name : –  SQL Server can be confirmed from the EV admin console and registry as below




Fingerprint database  :-  Run the “select *from FingerprintCatalogueDatabase” to check finger print database name


Vault store Database :-  expand directory database –> table –>  Right Click  dbo.vaultstoreGroup –> Select top 1000 Rows


Permission requirement on New sql Server

1. Confirm the Vault Service Account is a member of the local administrators group.

2. Confirm the Vault Service Account is assigned the dbcreator role.

            a. Open SQL Server Management Studio

            b. Expand Security Logins > right-click the Vault Service account and select Properties.

            c. Click Server Roles and confirm dbcreator is checked.

3. Grant the Vault Service account “Select” permissions on the following msdb tables:

  • sysjobs
  • sysjobschedules
  • sysjobservers
  • sysjobsteps.

Note: Use the following script to grant select permissions to the Vault Service Account over the MSDB tables. Replace DOMAIN\VSA by the Vault Service Account:



GRANT SELECT ON dbo.sysjobschedules TO “DOMAIN\VSA”

GRANT SELECT ON dbo.sysjobservers TO “DOMAIN\VSA”


4. Assign the database role “SQLAgentUserRole” to the Vault Service account.

5. Confirm TCP/IP and/or Named Pipes are enabled.

            a. Open SQL Server Configuration Manager

            b. Expand SQL Server Network Configuration and click Protocols for <instance name>.

            c. Confirm Named Pipes and/or TCP/IP is Enabled under the Status

  1. Check all other required permission from Enterprise vault installation and configuration guide

Backup and restore Database to new Server

Stop all the Enterprise Vault Service from all the EV Servers

Engage DBA (Database administrator) and backup and restore database to new sql Server

Now it’s time to update Entry for New SQL Server

Update the Fingerprint Database

Run attached script as mention in following article ,

Update the Monitoring Database

1. Open SQL Server Management Studio

2. Run the following query, substituting the New_SQL_Server value with the hostname of the new SQL server:

USE EnterpriseVaultDirectory

UPDATE MonitoringSettings

SET SQLServer = ‘New_SQL_Server’

Update the Directory Database in the registry

his step is performed on the Enterprise Vault Server.

1. Click Start > Run , type regedit and click OK .

For 32bit systems, HKEY_LOCAL_MACHINE\Software\KVS\Enterprise Vault\Directory\DirectoryService.

For 64bit systems, HKEY_LOCAL_MACHINE\Software\Wow6432Node\KVS\Enterprise Vault\Directory\DirectoryService

2. Double-click the SQLServer Name string value.

3. Enter the name of the new SQL server in the value data field and click OK.

4. Start the Enterprise Vault Directory Service, this will also start the Admin Service.

Update the Vault Store Database

This step is performed on the Enterprise Vault Server.

1. Open the Vault Administration Console (VAC), right-click the name of the Vault Store and select Properties.

2. Select the Database tab.

3. Next to SQL Server, enter the name of the new SQL server and click OK.

4. Repeat steps 1-3 for any remaining Vault Stores.

5. Start the remaining Enterprise Vault services

Update the Audit Database

This step is performed on the Enterprise Vault Server.

Note: No Enterprise Vault services need to be stopped for this step.

The EVAudit System DSN must be updated with the new location of the audit database.

1. For 32bit systems, Open Data Sources (ODBC) from the Administrative Tools panel on the Enterprise Vault Server. For 64bit systems, browse to %systemroot%\syswow64\odbcad32.exe

2. Select the System DSN tab.

3. Select the EVAudit entry.

4. Click the Configure button.

5. On the first page of the wizard, under Which SQL Server do you want to connect to?, enter the name of the new SQL server.

6. Continue through the wizard and click Finish to apply the changes.

Resume EV operations using the New SQL Server

  1. Start all EV Services on check event viewer for any error connecting to database

Base Source :-


Enterprise Vault Installation and configuration guide

Nutanix Cluster Health Check

Nutanix is now become more and more popular as an hyperconverged solution and in my blogs I am trying to cover steps to conduct the health check for Nutanix Cluster

Basically for health check “ncc health_checks run_all” is sufficient to provide all the required information however  you can gather specific information for below mention components with the help of below commands

+Cluster Info

– Cluster name

– Uptime

– NOS Version

– Cluster ID

– Block serial number

– HW model

 Commands to Run

 ncli cluster info

ncli host ls


+ Storage pool list

– Name

– Capacity (logical used and total)

– IOPS and latency

+ Container Info

– Name

– Capacity (logical used and total)

– IOPS and latency

– Replication factor

 Commands to Run

 ncli sp ls

ncli ctr ls

IOPS and Perf metrics can be viewed from prism page


 + CVM

– Status of service for each CVM

– CVM memory and VCPU usage

– Uptime

– Network stats

– IP addresses of CVMs

– NIC errors

 Commands to Run

 cluster status

allssh uptime




+ Disk status

– Perf stats and usage

 All Perf IO metrics are available on prism


+ Currently set gflags

ncc health_checks system_checks gflags_diff_check


+ Hypervisor

– Hypervisor software and version

– Uptime

– Installed VMs

– Memory Usage

– Attached Datastore

This is independent to the hypervisor ,


+ Datastore Info

– Usage

– Capacity

– Name

information available in prism


+ Disk list

+ Domain fault tolerance states

ncli cluster get-domain-fault-tolerance-status type=node

+ Default gateway

+ SSH key list

+ SMTP config

+ NTP config

information available in prism


Top things to know about vSAN — vErmirHoxhaj

As we know so far VMware vSAN is a distributed layer of software that runs natively as a part of the ESXi hypervisor. vSAN aggregates local or direct-attached capacity devices of a host cluster and creates a single storage pool shared across all hosts in the vSAN cluster. While supporting VMware features that require shared storage, such as HA, vMotion, and DRS, vSAN eliminates the need […]

via Top things to know about vSAN — vErmirHoxhaj