vCenter 6 – Create new licenses operation failed for the entity with the following error messages: storage error

As discussed in previous post, I have one SSO domain that is rather large (multiple vcenters, multiple external PSC servers, thousands of VMs). I have been adding my vSphere 6 licenses to prep for our hosts upgrades when I ran into an error. I tried to add another license and I got a Web Client error:

Create new licenses operation failed for the entity with the following error messages: storage error

In the thick client I would just get a generic error popup. Removing a license had a similar error, I also could not license hosts.

I opened a SR and it looks like I hit a maximum size for the VMDIR as described in this kb

My logs had entries similar to the examples in the KB:

• In the  C:\programdata\VMWare\vCenterServer\logs\sso\vmware-sts-idmd.log file, you see entries similar to:[2016-05-25T10:33:38.102-04:00 vsphere.local 8f828994-5c0a-4233-9bea-4cef7743c3de ERROR] [ServerUtils] Exception ‘com.vmware.identity.interop.ldap.OperationsErrorLdapException: Operations Error  LDAP error [code: 1]’
• In the C:\ProgramData\VMware\vCenterServer\logs\vmdird\vmdir.log file, you see entries similar to:2016-05-25T14:22:15.912Z:t@74550624:ERROR: VmDirSendLdapResult: Request (74), Error (1), Message (txn commit (9700)(MDB_MAP_FULL: Environment mapsize limit reached)), (0) socket ([2056] [::1]:389<-[::1]:60946)

Update: 12-6-2016 additional symptoms:

Can not assign a license to Host <hostname>. Make sure the License Service is available.

I’ve had a look in the logs under cis-license on the PSC and get the error:

com.vmware.cis.license.vpx.server.core.oldmanagement.LicenseAssignmentManagerException$StorageException: Unable to store license entity

…

Caused by: com.vmware.cis.license.vpx.server.core.oldmanagement.dao.licstorage.LicenseStorageException: com.vmware.cis.license.embedded.server.core.management.exception.StorageException: Unable to add asset: host-10202-516f80b0-a3dd-11e5-971a-0050568e7aae

A server error occurred.

[500] SSO error: Failed to import TenantConfiguration for [vsphere.local]: Failed to remove existing service provider registration.

 

The resolution was to add a registry key that set the VMDIR size to something greater than 1024 (I chose 2048). UPDATE: 12-6-2016 I had the error again and changed the value to 4196.

You can run the following from an administrator cmd prompt to add the key (replace servername with your server, this can update remote servers as well)

 

Assuming you use a default install, here is how to restart directory services (this one I logged in and ran on each PSC, though it could have probably been done using psexec or powercli)

Another design consideration for a large SSO domain! Increase the vmdir size to accommodate increased number of items in the directory.