vmware / vmware/pyvmomi

Is there a better way to get thousans of VirtualMachine information in a single vCenter?

Open
#851 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
2.3k
Forks
763
PR merge metrics
No merged PRs in 30d

Description

object_type = []
properties = {
    'VirtualMachine': ['config', 'guest', 'name', 'parent', 'runtime'],
     'HostSystem': ['name']
}
with vCenterPropertyCollector(vcenter, object_type, properties) as result:
class vCenterPropertyCollector(vCenterBase):
    """collect designation properties of vcenter object

    using Managed-object 'ProertyCollector' to retrieve properties
    """
    def __init__(self, vcenter, object_type, properties):
        super(vCenterPropertyCollector, self).__init__(vcenter)
        self._object_type = object_type
        self._properties = properties

    def __enter__(self):
        self.connect()
        result = self._collect(self._object_type, self._properties)
        return result

    def __exit__(self, exc_type, exc_val, exc_tb):
        self.disconnect()

    def _collect(self, object_type, properties):
        si = self.si
        if si is None:
            return {}

        result = dict(content=si.content)
        container = si.content.rootFolder
        objs = self.get_container_view(container, object_type)
        props = parse_propspec(properties)
        try:
            objects = self.collect_object_property(objs, props)
        except vmodl.query.InvalidProperty:
            raise
        for obj in objects:
            key = obj.obj
            value = dict()
            for prop in obj.propSet:
                value.update({prop.name: prop.val})
            result.update({key: value})
        return result
    def collect_object_property(self, objs, props, maxObjects=None):
        """ Retrieve properties of objs using PropertyCollector managed object

        :param objs: The objects will be query
        :param props: The properties of objects need to be query
        :param maxObjects: The maximum number of ObjectContent data objects that should be
        returned in a single result from RetrievePropertiesEx
        """
        if self.si is None:
            return

        pc = self.si.content.propertyCollector
        filterSpec = self.create_filter_spec(objs, props)
        options = vmodl.query.PropertyCollector.RetrieveOptions(maxObjects=maxObjects)
        result = pc.RetrievePropertiesEx([filterSpec], options)

        # because the maximum number of objects retrieved by RetrievePropertiesEx and
        # ContinueRetrievePropertiesEx is limit to 100, so, we need to continue retrieve
        # properties using token
        def _continue(token=None):
            _result = pc.ContinueRetrievePropertiesEx(token)
            _token = _result.token
            _objects = _result.objects
            if _token is not None:
                _objects_ex = _continue(_token)
                _objects.extend(_objects_ex)
            return _objects

        if result is None:
            return {}

        token = result.token
        objects = result.objects
        if token is not None:
            _objects = _continue(token)
            objects.extend(_objects)

        return objects

This is my current code to obtain virtual machine information in a single, but when the number of virtual machine becomes large, it will become very slow.

How can i solve the problem? Or do you have better way?
Thanks!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by examining vCenterPropertyCollector._collect and collect_object_property, especially the RetrievePropertiesEx and ContinueRetrievePropertiesEx flow shown in the issue. Profile retrieval with large numbers of VirtualMachine objects and compare an improved approach against the current behavior; done means obtaining the requested VM information with measurably better performance.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.