vmware / vmware/pyvmomi

multithreaded access to properties is slower than serial access

Open
#1,084 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
2.3k
Forks
763
PR merge metrics
No merged PRs in 30d

Description

Describe the bug

I noticed that accessing host properties from multiple threads is slower than doing so serialy.
I wrote a script to reproduce the issue:

# ruff: noqa

import ssl
from threading import Thread
import time

from pyVim.connect import SmartConnect
from pyVmomi import vim

NUM_THREADS = 8
HOST = ""
PASSWORD = ""

context = ssl._create_unverified_context()
con = SmartConnect(host=HOST, pwd=PASSWORD, sslContext=context)

host = con.content.viewManager.CreateContainerView(con.content.rootFolder, [vim.HostSystem], True).view[0]


threads = []
for i in range(NUM_THREADS):

    def print_driver(i):
        print(host.config.network.pnic[i].driver)

    t = Thread(target=print_driver, args=(i,))
    threads.append(t)

start = time.time()
for t in threads:
    t.start()
for t in threads:
    t.join()
end = time.time()
print(f"multi threaded: {end - start}")


start = time.time()
for i in range(NUM_THREADS):
    print(host.config.network.pnic[i].driver)
end = time.time()
print(f"single threaded: {end - start}")

On my host with 8 vmnics I get:

multi threaded: 11.908450603485107
single threaded: 3.76969313621521

The single threaded performance is stable around 4 seconds but the multithreaded performance jumps around between 6-12 seconds each run.
The script can be changed to always access pnic[0] with the same result.
The more threads run at the same time, the slower it gets.

Reproduction steps
  1. set NUM_THREADS, HOST, PASSWORD
  2. run the repro script
Expected behavior

I expect multithreaded performance to be better or equal to serial performance.

Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided Python reproduction using SmartConnect and concurrent access to host.config.network.pnic properties. Inspect the property-access and threading paths involved, then verify that concurrent access no longer performs worse than serial access across repeated benchmark runs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.