Skip to content

Fixed the parsing of the test name in utility_functions::basic_checks(). - #174

Open
bieryAtFnal wants to merge 1 commit into
developfrom
kbiery/test_name_fix_for_banner
Open

bieryAtFnal wants to merge 1 commit into
developfrom
kbiery/test_name_fix_for_banner

Conversation

@bieryAtFnal

Copy link
Copy Markdown
Collaborator

Description

I recently realized that the parsing of the PYTEST_CURRENT_TEST value to produce a concise test name in our integration tests was not correctly handling cases in which the "run_dunerc" fixture has been named by the integtest developer.

Different contents of PYTEST_CURRENT_TEST are illustrated by these two values:

integtest/minimal_system_quick_test.py::test_dunerc_success[MinimalSystem-StandAloneSSH_PM-run_dunerc0]

.venv/lib/python3.12/site-packages/drunc/integtest/pm_us_test.py::test_dunerc_success[SmallFootprint-StandAloneSSH_PM-MultiRCAppSession] 

In both cases, we want to focus on the string inside the square brackets. In the first case, we also want to strip off the "-run_dunerc0" since it doesn't provide much useful information.

The changes in this PR fixes the parsing of these strings.

Here are suggested instructions for testing these changes and showing examples of the desired concise test names: (WIBEth_TPG_System-StandAloneSSH_PM and SmallFootprint-StandAloneSSH_PM-MultiRCAppSession:

DATE_PREFIX=`date '+%d%b'`
TIME_SUFFIX=`date '+%H%M'`

source /cvmfs/dunedaq.opensciencegrid.org/setup_dunedaq.sh
setup_dbt latest
dbt-create -n NFD_DEV_261017_A9 ${DATE_PREFIX}FDDevTest_${TIME_SUFFIX}
cd ${DATE_PREFIX}FDDevTest_${TIME_SUFFIX}/sourcecode

git clone https://github.com/DUNE-DAQ/daqsystemtest.git -b develop

cd ..

cd pythoncode
git clone https://github.com/DUNE-DAQ/drunc.git -b kbiery/integtest_infr_updates
git clone https://github.com/DUNE-DAQ/integrationtest.git -b kbiery/test_name_fix_for_banner
cd ..

sed -i 's/utility_functions.basic_checks(run_dunerc, caplog, print_test_name=False)/utility_functions.basic_checks(run_dunerc, caplog, print_test_name=True)/' pythoncode/drunc/src/drunc/integtest/basic_multiapp_test.py

dbt-workarea-env
dbt-build -j 12
dbt-workarea-env

dunedaq_integtest_bundle.sh -r local --verb 4 -k '3ru_3df|basic' --pytest-options "-k 'TPG or Small'"

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Testing checklist

  • Full set of integration tests pass (dunedaq_integtest_bundle.sh)

@PawelPlesniak

Copy link
Copy Markdown

To investigate this PR, I ran the above test, generating logfile /tmp/pytest-of-pplesnia/dunedaq_integtest_bundle_20261021184244.log
I also ran the same without the branch specifiers or sed command, generating logfile /tmp/pytest-of-pplesnia/dunedaq_integtest_bundle_20261021184922.log

In comparing these logfiles, I did not see an instance of -run_dunerc0 in either logfile.

I ran MSQT in both cases. In the area with the branches I saw

pytest /nfs/home/pplesnia/nightlyDev/21OctFDDevTest_1839/install/daqsystemtest/share/integtest/small_footprint_quick_test.py 
============================================ test session starts =============================================
platform linux -- Python 3.12.12, pytest-9.0.2, pluggy-1.6.0
rootdir: /nfs/home/pplesnia/nightlyDev/21OctFDDevTest_1839
plugins: anyio-4.12.1, integrationtest-4.1.0
collected 3 items                                                                                            

install/daqsystemtest/share/integtest/small_footprint_quick_test.py ...                                [100%]

======================================== 3 passed in 60.08s (0:01:00) ========================================

In the area without the branches I saw

pytest /nfs/home/pplesnia/nightlyDev/21OctFDDevTest_1839/install/daqsystemtest/share/integtest/small_footprint_quick_test.py 
============================================ test session starts =============================================
platform linux -- Python 3.12.12, pytest-9.0.2, pluggy-1.6.0
rootdir: /nfs/home/pplesnia/nightlyDev
plugins: anyio-4.12.1, integrationtest-4.1.0
collected 3 items                                                                                            

../21OctFDDevTest_1839/install/daqsystemtest/share/integtest/small_footprint_quick_test.py ...         [100%]

======================================== 3 passed in 60.45s (0:01:00) ========================================

Could you remind me of how to run the tests to see the difference?

@bieryAtFnal

Copy link
Copy Markdown
Collaborator Author

Apologies, here are hopefully better instructions for demonstrating this fix:

DATE_PREFIX=`date '+%d%b'`
TIME_SUFFIX=`date '+%H%M'`

source /cvmfs/dunedaq.opensciencegrid.org/setup_dunedaq.sh
setup_dbt latest
dbt-create -n NFD_DEV_261021_A9 ${DATE_PREFIX}FDDevTest_${TIME_SUFFIX}
cd ${DATE_PREFIX}FDDevTest_${TIME_SUFFIX}/sourcecode

git clone https://github.com/DUNE-DAQ/daqsystemtest.git -b develop

cd ..

cd pythoncode
git clone https://github.com/DUNE-DAQ/drunc.git -b develop
git clone https://github.com/DUNE-DAQ/integrationtest.git -b develop

cd ..
sed -i 's/utility_functions.basic_checks(run_dunerc, caplog, print_test_name=False)/utility_functions.basic_checks(run_dunerc, caplog, print_test_name=True)/' pythoncode/drunc/src/drunc/integtest/basic_multiapp_test.py

. ./env.sh
dbt-build -j 12
dbt-workarea-env

dunedaq_integtest_bundle.sh -r drunc -k basic --verb 4

cd pythoncode/integrationtest
git checkout kbiery/test_name_fix_for_banner
cd ../../

dbt-build -j 12
dbt-workarea-env

dunedaq_integtest_bundle.sh -r drunc -k basic --verb 4

In the banner that is printed out from the first running of the integtest, the test name that is shown is

.venv/lib/python3.12/site-packages/drunc/integtest/basic_multiapp_test.py::test_dunerc_success[SmallFootprint-StandAloneSSH_PM-MultiRCAppSession] (call)

In the second test, the following is printed out:

SmallFootprint-StandAloneSSH_PM-MultiRCAppSession

IMO, the second one is preferable, hence this fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants