Add SDV 1-VM and 2-VM presubmit E2E tests - #3153
Conversation
ab016d7 to
f06083f
Compare
8ff79cf to
e2157fa
Compare
Integrate Software Defined Vehicle (SDV) test coverage into android-cuttlefish presubmit, validating lifecycle operations and multi-instance isolation based on Android platform testing: - Single-VM lifecycle operations (status, stop, start, restart). - Two-VM environment creation and secondary instance lifecycle with primary instance isolation checks. Summary of changes: - Add //cvd/sdv_tests:sdv_tests Go E2E test target under e2etests/cvd. - Implement 1-VM SDV test covering image fetch, boot, status check, stop/start lifecycle, and restart lifecycle with sysprop validation. - Implement 2-VM SDV test (SDV Core ins-1 on 127.0.0.1:6520 and SDV Media ins-2 on 127.0.0.1:6521) covering multi-VM creation via load config, status verification, and ins-2 stop/start and restart lifecycle with ins-1 isolation checks. - Exclude sdv_tests from containerized kokoro_podcvd via podcvd_excluded tag. - Add common helpers in e2etests/cvd/common/common.go: - RunAdbWaitForDeviceSerial, WaitForDeviceOnline, WaitForDeviceOffline - CVDInstanceStop, CVDInstanceStart, CVDInstanceRestart, CVDInstanceStatus - ParseCVDStatusJSON supporting array, group, and single-object JSON formats - GetSyspropStringForDevice for explicit serial-targeted sysprop reads Test: - bazel build //cvd/... - bazel test //cvd/sdv_tests:sdv_tests (in Kokoro presubmit environment) BUG=507906785 TAG=agy CONV=e15765ae-4a0e-4b45-a7f2-07dad2ac1459
cd2fa8a to
5bb066b
Compare
SuperStrongDinosaur
changed the title
|
Instead of treating SDV CF as an outlier, a better approach is to generalize the new test cases so they can optionally support non-SDV CF variants. So I recommend the following changes:
@Databean please let me know your thoughts. |
Databean
left a comment
Databean
left a comment
There was a problem hiding this comment.
a better approach is to generalize the new test cases so they can optionally support non-SDV CF variants.
Most of the tests do look pretty general, except for the GetSyspropStringForDevice parts.
|
|
||
| // Checks if adb shell is reachable for a given device serial. | ||
| func (tc *TestContext) IsAdbShellReachable(serial string) bool { | ||
| res, err := tc.RunCmd("timeout", "5s", "adb", "-s", serial, "shell", "echo", "ping") |
There was a problem hiding this comment.
Sleep is not recommended in tests (go/tott/303) whether using timeout as a shell command or time.Sleep below.
In the case of trying to connect with adb, the biggest source of issues I usually see is that the Cuttlefish host tools internally attempt to adb connect the device at regular intervals, but only if the adb server is already running. That means that the first adb shell may fail if it starts the server, and a later one succeeds because the host tools have had a chance to adb connect.
If the problem is specifically that issue, it should be possible to run adb devices first, then wait for cvd create / cvd start to return, then the device should be there.
| if i%5 == 0 { | ||
| tc.RunCmd("adb", "connect", serial) | ||
| } |
There was a problem hiding this comment.
The cuttlefish host tools will already attempt to adb connect in adb_connector.
| // Performs `cvd stop <args>`. | ||
| func (tc *TestContext) CVDStop(args ...string) error { | ||
| stopCmd := append([]string{"stop"}, args...) | ||
| if _, err := tc.RunCVD(stopCmd...); err != nil { | ||
| log.Printf("Failed to stop instance(s): %v", err) | ||
| return err | ||
| } | ||
| return nil | ||
| } |
There was a problem hiding this comment.
Can all the CVD* methods be moved out to free functions in a new file that accept a TestContext as the first argument? This file is getting kind of large and they all only rely on tc.RunCVD and not any other TestContext implementation details.
| type CVDInstanceStatusEntry struct { | ||
| AdbPort int `json:"adb_port"` | ||
| AdbSerial string `json:"adb_serial"` | ||
| AssemblyDir string `json:"assembly_dir"` | ||
| InstanceDir string `json:"instance_dir"` | ||
| InstanceName string `json:"instance_name"` | ||
| Status string `json:"status"` | ||
| WebAccess string `json:"web_access"` | ||
| WebRtcDeviceID string `json:"webrtc_device_id"` | ||
| } | ||
|
|
||
| // ParseCVDStatusJSON parses the JSON output returned by `cvd status --print`. | ||
| // It handles both single-instance array `[...]` and group object `{"instances": [...]}` formats. | ||
| func ParseCVDStatusJSON(output string) ([]CVDInstanceStatusEntry, error) { |
There was a problem hiding this comment.
Since the only way tests get this json output is to call cvd status --print and since no test calls cvd status without --print, can this be called internally from CVDInstanceStatus and CVDStatus?
4 participants
This PR introduces end-to-end presubmit test coverage for SDV targets under
e2etests/cvd/sdv_tests.The test suite validates SDV device lifecycle operations (status querying, graceful stop, start, and restart) and multi-VM isolation based on the device-host interaction baseline tests:
- Single-VM lifecycle operations (
sdv_fw_baseline_dhi_1vm_test.py)- Two-VM multi-instance lifecycle and isolation (
sdv_fw_baseline_dhi_2vm_test.py)BUG: b/559633154