mlcommons / mlcommons/power-dev

ptd is asked to stop measurement twice

Open
#287 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
28
Forks
30
No merged PRs in 30d

Description

I'm running into an error with both the server and the client on 1a16663 (i.e. the very last change before updating the supported PTD versions).

Server

After the ranging run, everything is fine - we change the Amp & Volt ranges accordingly. 
ptd-server 2023-02-10 06:31:52,246 [INFO] Got command from the client 'session,2023-02-10_06-18-14,start,testing'
ptd-server 2023-02-10 06:31:52,246 [INFO] Sending to ptd: 'SR,V,120.190000'
02-10-2023 12:31:52.350: Completed test
02-10-2023 12:31:52.350: Avg watts 583.590709, min watts 172.960000, max watts 713.140000, samples 790, errors 0, valid 790
02-10-2023 12:31:52.350: Volt range set to 150.000000V
02-10-2023 12:31:53.851: Response to client sent: Range V changed
ptd-server 2023-02-10 06:31:53,852 [INFO] Reply from ptd: 'Range V changed'
ptd-server 2023-02-10 06:31:53,852 [INFO] Sending to ptd: 'SR,A,6.065300'
02-10-2023 12:31:53.852: Amp range set to 10.000000A
02-10-2023 12:31:55.354: Response to client sent: Range A changed
ptd-server 2023-02-10 06:31:55,354 [INFO] Reply from ptd: 'Range A changed'
Then we start testing. 
ptd-server 2023-02-10 06:32:05,365 [INFO] Starting testing mode
ptd-server 2023-02-10 06:32:05,365 [INFO] Sending to ptd: 'Go,1000,0,2023-02-10_06-18-14_testing'
02-10-2023 12:32:05.365: Go with mark '2023-02-10_06-18-14_testing'
02-10-2023 12:32:05.365: Response to client sent: Starting untimed measurement, maximum 500000 samples at 1000ms with 0 rampup samples
ptd-server 2023-02-10 06:32:05,365 [INFO] Reply from ptd: 'Starting untimed measurement, maximum 500000 samples at 1000ms with 0 rampup samples'
ptd-server 2023-02-10 06:32:05,366 [INFO] Sending reply to client 'OK'

After 13 minutes, a 'Stop' command is sent to PTD. 
ptd-server 2023-02-10 06:45:05,436 [INFO] Got command from the client 'session,2023-02-10_06-18-14,stop,testing'
ptd-server 2023-02-10 06:45:15,446 [INFO] Sending to ptd: 'Stop'
02-10-2023 12:45:15.447: Response to client sent: Stopping untimed measurement
ptd-server 2023-02-10 06:45:15,447 [INFO] Reply from ptd: 'Stopping untimed measurement'
ptd-server 2023-02-10 06:45:15,488 [INFO] Sending reply to client 'OK'
ptd-server 2023-02-10 06:45:15,493 [INFO] Got command from the client 'session,2023-02-10_06-18-14,done'
And then another 'Stop' command is sent to PTD, whose response 'Error: no measurement to stop' is sent to the client. 
ptd-server 2023-02-10 06:45:15,493 [INFO] Sending to ptd: 'Stop'
02-10-2023 12:45:16.365: Completed test
02-10-2023 12:45:16.365: Avg watts 595.281669, min watts 188.000000, max watts 714.600000, samples 791, errors 0, valid 791
02-10-2023 12:45:16.365: Response to client sent: Error: no measurement to stop
ptd-server 2023-02-10 06:45:16,366 [INFO] Reply from ptd: 'Error: no measurement to stop'
I believe it is the above error that the client receives, while it expects an integer (see below).
We set the Volt & Amp ranges back to Auto. 
ptd-server 2023-02-10 06:45:16,366 [INFO] Sending to ptd: 'SR,V,Auto'
02-10-2023 12:45:16.366: Volt range set to Auto
02-10-2023 12:45:16.869: Response to client sent: Range V changed
ptd-server 2023-02-10 06:45:16,869 [INFO] Reply from ptd: 'Range V changed'
ptd-server 2023-02-10 06:45:16,869 [INFO] Sending to ptd: 'SR,A,Auto'
02-10-2023 12:45:16.870: Ampere range set to Auto
02-10-2023 12:45:17.372: Response to client sent: Range A changed
ptd-server 2023-02-10 06:45:17,372 [INFO] Reply from ptd: 'Range A changed'
ptd-server 2023-02-10 06:45:17,373 [INFO] Set initial values for Amps Auto and Volts Auto
Why stopping PTD again and shutting connection down? 
ptd-server 2023-02-10 06:45:17,373 [INFO] Stopping ptd...
02-10-2023 12:45:17.373: No data returned by socket read.
02-10-2023 12:45:17.373: Shutting connection down...
02-10-2023 12:45:17.373: Connection is down.
02-10-2023 12:45:17.373: Waiting for a connection...

Client

ValueError: invalid literal for int() with base 10: 'Error' 
client 2023-02-10 06:45:15,488 [INFO] Sending command to the server: 'session,2023-02-10_06-18-14,done'
client 2023-02-10 06:45:17,375 [INFO] Got response: 'OK'
client 2023-02-10 06:45:17,375 [INFO] Fetching file '/home/krai/CK_REPOS/ck-mlperf/program/mlperf-power-client/tmp/2023-02-10_06-18-14/power/ptd_logs.txt'
client 2023-02-10 06:45:17,379 [INFO] Received '/home/krai/CK_REPOS/ck-mlperf/program/mlperf-power-client/tmp/2023-02-10_06-18-14/power/ptd_logs.txt'
client 2023-02-10 06:45:17,379 [INFO] Fetching file '/home/krai/CK_REPOS/ck-mlperf/program/mlperf-power-client/tmp/2023-02-10_06-18-14/ranging/spl.txt'
client 2023-02-10 06:45:17,381 [INFO] Received '/home/krai/CK_REPOS/ck-mlperf/program/mlperf-power-client/tmp/2023-02-10_06-18-14/ranging/spl.txt'
client 2023-02-10 06:45:17,381 [INFO] Fetching file '/home/krai/CK_REPOS/ck-mlperf/program/mlperf-power-client/tmp/2023-02-10_06-18-14/power/server.json'
client 2023-02-10 06:45:17,382 [INFO] Received '/home/krai/CK_REPOS/ck-mlperf/program/mlperf-power-client/tmp/2023-02-10_06-18-14/power/server.json'
client 2023-02-10 06:45:17,382 [INFO] Fetching file '/home/krai/CK_REPOS/ck-mlperf/program/mlperf-power-client/tmp/2023-02-10_06-18-14/power/server.log'
client 2023-02-10 06:45:17,384 [INFO] Received '/home/krai/CK_REPOS/ck-mlperf/program/mlperf-power-client/tmp/2023-02-10_06-18-14/power/server.log'
client 2023-02-10 06:45:17,384 [INFO] Fetching file '/home/krai/CK_REPOS/ck-mlperf/program/mlperf-power-client/tmp/2023-02-10_06-18-14/run_1/spl.txt'
client 2023-02-10 06:45:17,386 [INFO] Received '/home/krai/CK_REPOS/ck-mlperf/program/mlperf-power-client/tmp/2023-02-10_06-18-14/run_1/spl.txt'
client 2023-02-10 06:45:17,386 [INFO] Fetching file '/home/krai/CK_REPOS/ck-mlperf/program/mlperf-power-client/tmp/2023-02-10_06-18-14/run_1/ptd_out.txt'
Traceback (most recent call last):
  File "/home/krai/CK_TOOLS/mlperf-power-r3.0/power/ptd_client_server/client.py", line 27, in 
    client.main()
  File "/home/krai/CK_TOOLS/mlperf-power-r3.0/power/ptd_client_server/../ptd_client_server/lib/client.py", line 357, in main
    command.download(f"download,{session},{fname}", os.path.join(out_dir, fname))
  File "/home/krai/CK_TOOLS/mlperf-power-r3.0/power/ptd_client_server/../ptd_client_server/lib/client.py", line 63, in download
    self._server.recv_file(fname)
  File "/home/krai/CK_TOOLS/mlperf-power-r3.0/power/ptd_client_server/../ptd_client_server/lib/common.py", line 110, in recv_file
    chunk_len = int(line, 10)
ValueError: invalid literal for int() with base 10: 'Error'

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the stop and download flow in ptd_client_server/lib/client.py, then trace recv_file in ptd_client_server/lib/common.py against the server log showing two Stop commands. Reproduce the session from the issue and follow the client/server response protocol. Done means the measurement is stopped once and the client no longer parses the server's error response as an integer during file download.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.