vmware / vmware/pyvmomi

Performance Manager Property Collector Not Returning IOPS for 1 minute intervals.

Open
#195 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

needs verification question
Dominant language
Python
Stars
2.3k
Forks
763
PR merge metrics
No merged PRs in 30d

Description

I'm trying to fix my negative iops so I wrote a script that returns different IOPS metrics for each vm. I used a property collector and performance manager to pull the data. The datastore.datastoreWriteIops.latest, and the datastore.datastoreReadIops.latest doesn't return anything ever. If I take the start_time and end_time out of the query it will return a value for the other four metrics but they don't align with the real time value in vCenter. I need to get the realtime value for these metrics can you please help me.

So to summarize, when I leave the start and end time in the query for the perfManager nothing is returned for any of the metrics.

When I take the start and end time out of the query it returns values that don't align with the realtime values in vCenter which is what I'm trying to get.

the start_time and end_time are 1 minute apart because I'm trying to get the realtime value. So I've tried with and without it.

This is the query it builds now I'm using the server time because I read in the documentation start_time is based on server time.

 start_time = vcenter_connection.CurrentTime() - datetime.timedelta(minutes=1)

    (vim.PerformanceManager.QuerySpec) {
       dynamicType = <unset>,
       dynamicProperty = (vmodl.DynamicProperty) [],
       entity = 'vim.VirtualMachine:vm-759',
       startTime = 2014-12-03T19:51:24.363823Z,
       endTime = <unset>,
       maxSample = <unset>,
       metricId = (vim.PerformanceManager.MetricId) [
          (vim.PerformanceManager.MetricId) {
         dynamicType = <unset>,
         dynamicProperty = (vmodl.DynamicProperty) [],
         counterId = 129,
         instance = '*'
          }
       ],
       intervalId = 20,
       format = <unset>
    }


  rundate = datetime.datetime.now()
  start_time = rundate - datetime.timedelta(minutes=(vm_sample_interval+1))
  end_time = rundate - datetime.timedelta(minutes=1)

  for k in xrange(0, len(config_values['username'])):      # Loop through each vcenter server
     password = config_values['password'][k]
     hostname = config_values['vcenter-ip'][k]
     username = config_values['username'][k]
     vcenter_connection = SmartConnect(host=hostname,user=username,pwd=password)
     vcenter_id = vcenters[uuid]

     if not vcenter_connection:
        logging.error("Could not connect to vcenter-id %d, vcenter-ip %s" % (vcenter_id, str(hostname)))
        continue

     atexit.register(Disconnect, vcenter_connection)             
     content = vcenter_connection.RetrieveContent()       
     perf_dict = {} 
     perfList = content.perfManager.perfCounter

     for counter in perfList: #build the vcenter counters for the objects
        counter_full = "{}.{}.{}".format(counter.groupInfo.key,counter.nameInfo.key,counter.rollupType)
        perf_dict[counter_full] = counter.key

     viewType = [vim.VirtualMachine]
     props = ['name','runtime.powerState', 'datastore']
     specType = vim.VirtualMachine
     objView = content.viewManager.CreateContainerView(content.rootFolder,viewType,True)  
     tSpec = vim.PropertyCollector.TraversalSpec(name='tSpecName', path='view', skip=False, type=vim.view.ContainerView)    
     pSpec = vim.PropertyCollector.PropertySpec(all=False, pathSet=props,type=specType)   
     oSpec = vim.PropertyCollector.ObjectSpec(obj=objView,selectSet=[tSpec],skip=False)   
     pfSpec = vim.PropertyCollector.FilterSpec(objectSet=[oSpec], propSet=[pSpec], reportMissingObjectsInResults=False)   
     vm_properties = content.propertyCollector.RetrieveProperties(specSet=[pfSpec])  
     objView.Destroy()     

     for vm_property in vm_properties: #loop through the list built from vcenter and build dictonaries.
        property_dic = {}
        for prop in vm_property.propSet:
           property_dic[prop.name] = prop.val 

        vm = vm_property.obj 
        vm_mor = vm._moId
        storage_perf_dict = {}
        if "poweredOn" in vm.runtime.powerState:
           try:

              counters = ['disk.numberWrite.summation','disk.numberRead.summation','datastore.datastoreWriteIops.latest','datastore.datastoreReadIops.latest','datastore.numberReadAveraged.average', 'datastore.numberWriteAveraged.average']
              for counter_name in counters: 
                 perfManager = content.perfManager  
                 counterId = perf_dict[counter_name]
                 metricId = vim.PerformanceManager.MetricId(counterId=counterId, instance="*") 
                 query = vim.PerformanceManager.QuerySpec(intervalId=20, entity=vm, metricId=[metricId], startTime=start_time, endTime=end_time) 
                 statDatastoreIo = perfManager.QueryPerf(querySpec=[query])
                 storage_perf_dict[counter_name] = statDatastoreIo

              for counter in counters:
                 stat = storage_perf_dict[counter]
                 if len(stat) > 0:
                    print vm_mor, counter, sum(stat[0].value[0].value)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the inline PerformanceManager.QuerySpec and QueryPerf calls, especially intervalId=20 and the startTime/endTime handling. Compare the returned datastore and disk metrics with the vCenter real-time values and the referenced server-time behavior. Done means identifying why the one-minute query returns no usable IOPS data and documenting or correcting the behavior with reproducible validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.