oxidecomputer / oxidecomputer/propolis

phd live migration tests fail on atrium

Open
#1,099 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

testing
Dominant language
Rust
Stars
270
Forks
42
Avg merge
4d 5h
Merged PRs (30d)
6

Description

atrium is a shared system in the lab that does not have a VMM reservoir configured. I tried to run the PHD tests locally (via cargo xtask phd run) against what's currently on propolis master (445814849d97db485ae3f693b3236baeef22f965) and the migration tests fail:

                                                                                                                                                                                                                                                                                           
failures:                                                                                                                                                                                                                                                                                  
    phd_tests::migrate::running_process::export_failure                                                                                                                                                                                                                                    
    phd_tests::migrate::from_base::can_migrate_from_base                                                                                                                                                                                                                                   
    phd_tests::migrate::from_base::serial_history                                                                                                                                                                                                                                          
    phd_tests::migrate::from_base::migration_from_base_and_back                                                                                                                                                                                                                            
                                                                                                                                                                                                                                                                                           
test result: FAILED. 42 passed; 4 failed; 3 skipped; 0 not run; finished in 488.42s 

The tests for from_base all fail like this:

00:48:36.994Z ERRO propolis-server (vm_state_driver): failed to activate new VM
    error = creating VM objects for new instance: failed to join VM object creation task: failed to add low memory region
00:48:36.995Z INFO propolis-server: request completed
    error_message_external = Internal Server Error
    error_message_internal = VM initialization failed: failed to join VM object creation task: failed to add low memory region
    latency_us = 28320
    local_addr = 127.0.0.1:9000
    method = PUT
    remote_addr = 127.0.0.1:58585
    req_id = 9a03ed6d-0dea-4436-bad7-cc387620267b
    response_code = 500
    uri = /instance

It was suggested in hypervisor channel chatter that this might be because there is no reservoir configured on this system. It would be good to figure that out for sure and perhaps improve things a bit here.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with cargo xtask phd run on atrium and reproduce the four failures under phd_tests::migrate, comparing the from_base and running_process::export_failure cases. Investigate whether the missing VMM reservoir produces the low-memory-region error, then make the relevant test or environment behavior clearly handle that condition and verify the PHD suite no longer fails unexpectedly.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
operating-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.