Skip to content

DAOS-19635 test: dfuse/daos_build_vm.py:DaosBuildVM - scons cmd timeout - #19077

Open
dinghwah wants to merge 13 commits into
masterfrom
dinghwah/daos-19635-ddtest
Open

DAOS-19635 test: dfuse/daos_build_vm.py:DaosBuildVM - scons cmd timeout#19077
dinghwah wants to merge 13 commits into
masterfrom
dinghwah/daos-19635-ddtest

Conversation

@dinghwah

@dinghwah dinghwah commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Steps for the author:

  • Commit message follows the guidelines.
  • Appropriate Features or Test-tag pragmas were used.
  • Appropriate Functional Test Stages were run.
  • At least two positive code reviews including at least one code owner from each category referenced in the PR.
  • Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

  • Gatekeeper requested (daos-gatekeeper added as a reviewer).

@dinghwah
DAOS-18659 test: launch test with 10 repeated loops
65df405 
Test-tag: test_snapshot_aggregation
Skip-unit-tests: true
Test-repeat: 10

Signed-off-by: Ding-Hwa Ho <ding-hwa.ho@hpe.com>
@dinghwah
restore rf.py
a92bb46 
Signed-off-by: Ding-Hwa Ho <ding-hwa.ho@hpe.com>
@dinghwah
restore pool/rf.py to vm
7a03588 
Signed-off-by: Ding-Hwa Ho <ding-hwa.ho@hpe.com>
@dinghwah
DAOS-19637 test: rebuild/pool_destroy_race.py - test timeout in tearDown
d857518 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: test_pool_destroy_with_io
Required-githooks: true

Signed-off-by: Ding-Hwa Ho <ding-hwa.ho@hpe.com>
@dinghwah
DAOS-19635 test: dfuse/daos_build_vm.py:DaosBuildVM.test_dfuse_daos_b…
49bbe6f 
…uild_wt_pil4dfs - scons command timeout

Skip-unit-tests: true
Skip-fault-injection-test: true
Required-githooks: true

Signed-off-by: Ding-Hwa Ho <ding-hwa.ho@hpe.com>
@dinghwah
Restore rebuild/pool_destroy_race.yaml
e090f49 
Skip-unit-tests: true
Skip-fault-injection-test: true
Required-githooks: true

Signed-off-by: Ding-Hwa Ho <ding-hwa.ho@hpe.com>
@github-actions

github-actions Bot commented Sep 15, 2026 •
edited
Loading

Copy link
Copy Markdown

Ticket title is 'dfuse/daos_build_vm.py:DaosBuildVM.test_dfuse_daos_build_wt_pil4dfs - scons command timeout'
Status is 'In Review'
Labels: 'ci_master_weekly,weekly_test'
https://daosio.atlassian.net/browse/DAOS-19635

@dinghwah dinghwah changed the title DAOS-19635 test: dfuse/daos_build_vm.py:DaosBuildVM.test_dfuse_daos_build_wt_pil4dfs - scons command timeout DAOS-19635 test: dfuse/daos_build_vm.py:DaosBuildVM - scons command timeout Sep 16, 2026
@dinghwah
dinghwah marked this pull request as ready for review September 16, 2026 15:08
@dinghwah
dinghwah requested review from a team as code owners September 16, 2026 15:08
mchaarawi
mchaarawi requested changes Sep 16, 2026
View reviewed changes

@mchaarawi mchaarawi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as i mentioned in the ticket, this should not be increased when VMs is used, but when the il is set

daltonbohning
daltonbohning reviewed Sep 16, 2026
View reviewed changes
Comment thread src/tests/ftest/dfuse/daos_build.py Outdated
Comment on lines +89 to +90
if run_on_vms:
build_time += 60

@daltonbohning daltonbohning Sep 16, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if run_on_vms:
build_time += 60
if il_lib is not None::
build_time += 60

@dinghwah dinghwah Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please see the new commit

@dinghwah
Update per Dalton and Mohamad's comment
e465212 
Skip-unit-tests: true
Skip-fault-injection-test: true
Required-githooks: true

Signed-off-by: Ding-Hwa Ho <ding-hwa.ho@hpe.com>
daltonbohning
daltonbohning previously approved these changes Sep 16, 2026
View reviewed changes
shimizukko
shimizukko previously approved these changes Sep 16, 2026
View reviewed changes
@dinghwah
dinghwah requested review from a team and mchaarawi September 17, 2026 01:28
@dinghwah
dinghwah requested a review from a team September 17, 2026 21:33
phender
phender requested changes Sep 17, 2026
View reviewed changes

@phender phender left a comment •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR needs to run the test_dfuse_daos_build_wt_pil4dfs test, and based upon the edit so it should be run with:

Test-tag: DaosBuild DaosBuildVM

Comment thread src/tests/ftest/dfuse/daos_build.py Outdated
dfuse.disable_wb_cache.value = True
elif cache_mode == 'nocache':
build_time *= 4
if il_lib is not None:

@phender phender Sep 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test failing in DAOS-19635 is test_dfuse_daos_build_wt_pil4dfs is which calls run_build_test(self, "nocache", il_lib='libpil4dfs.so', run_on_vms=True), so to only bump the timeout on this test don't we want:

Suggested change
if il_lib is not None:
if il_lib is 'libpil4dfs.so' and run_on_vms:

@mchaarawi mchaarawi Sep 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't think this bump should be associated with vm or not.
i think we should bump the timeout by 1hours regardless of VM or HW, cache or no cache.
When il is used (ioil for now is disabled and will stay disabled, so just pil4dfs is fine), we want to bump the timeout.

@mchaarawi mchaarawi Sep 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since the dfuse build tests are moving to CB, maybe we should just collapse all those tests back into a single .py file and just make them all hw,cb ?

@dinghwah
Update per Phil's comment with repeat 10 loops
5708f38 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-repeat: 10
Required-githooks: true

Signed-off-by: Ding-Hwa Ho <ding-hwa.ho@hpe.com>
@dinghwah
dinghwah dismissed stale reviews from shimizukko and daltonbohning via 5708f38 September 17, 2026 21:52
@dinghwah dinghwah changed the title DAOS-19635 test: dfuse/daos_build_vm.py:DaosBuildVM - scons command timeout DAOS-19635 test: dfuse/daos_build_vm.py:DaosBuildVM - scons cmd timeout Sep 17, 2026
@dinghwah
DAOS-18659 test: launch test with 10 repeated loops
597134a 
Test-tag: test_snapshot_aggregation
Skip-unit-tests: true
Test-repeat: 10

Signed-off-by: Ding-Hwa Ho <ding-hwa.ho@hpe.com>
@dinghwah
restore rf.py
1282b05 
Signed-off-by: Ding-Hwa Ho <ding-hwa.ho@hpe.com>
@dinghwah
restore pool/rf.py to vm
e516e23 
Signed-off-by: Ding-Hwa Ho <ding-hwa.ho@hpe.com>
@dinghwah
Merge branch 'master' into dinghwah/daos-19635-ddtest
4402592 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-repeat: 10
Required-githooks: true

Signed-off-by: Ding-Hwa Ho <ding-hwa.ho@hpe.com>
@dinghwah
Remove run_on_vms
6dc7d7f 
Skip-unit-tests: true
Skip-fault-injection-test: true
Test-tag: DaosBuild DaosBuildVM
Test-repeat: 10
Required-githooks: true

Signed-off-by: Ding-Hwa Ho <ding-hwa.ho@hpe.com>
@daosbuild3

daosbuild3 commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

Test stage Functional Cluster Box Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-19077/5/execution/node/1058/log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@phender phender phender requested changes
@shimizukko shimizukko shimizukko left review comments
@daltonbohning daltonbohning daltonbohning left review comments
@mchaarawi mchaarawi Awaiting requested review from mchaarawi

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@dinghwah @daosbuild3 @shimizukko @mchaarawi @daltonbohning @phender