Exercise 4.3 - ROS Integration Test - Test fails at output

girishkumar.kannan · November 30, 2022, 8:44am

Hi The Construct team,

I just finished the chapter “ROS Integration Tests” of this course Unit Testing with ROS.

When I execute exercise 4.3, the result / output shows that the test failed.
Should the test fail or pass?

This is my output:

user:~/catkin_ws$ rostest robot_control rotate_robot_integration_test.test
... logging to /home/user/.ros/log/rostest-1_xterm-12497.log
[ROSUNIT] Outputting test results to /home/user/.ros/test_results/robot_control/rostest-test_rotate_robot_integration_test.xml
Clockwise
Traceback (most recent call last):
  File "/home/user/catkin_ws/src/robot_control/src/robot_control/rotate_robot_srv_client.py", line 20, in <module>
    result = rotate_robot_service(rotate_robot_object)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 435, in __call__
    return self.call(*args, **kwds)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 523, in call
    raise rospy.exceptions.ROSInterruptException("node shutdown interrupted service call")
rospy.exceptions.ROSInterruptException: node shutdown interrupted service call
Traceback (most recent call last):
  File "/home/user/catkin_ws/src/robot_control/src/robot_control/reset_world_client.py", line 10, in <module>
    rospy.wait_for_service('/gazebo/reset_world')
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 159, in wait_for_service
    raise ROSInterruptException("rospy shutdown")
rospy.exceptions.ROSInterruptException: rospy shutdown
[Testcase: testrotate_robot_integration_test_ex] ... ok

[ROSTEST]-----------------------------------------------------------------------

[robot_control.rosunit-rotate_robot_integration_test_ex/test_correct_rotation][FAILURE]
Integration error. Rotation was not between the expected values.
  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/home/user/catkin_ws/src/robot_control/test/rotate_robot_integration_test_ex.py", line 44, in test_correct_rotation
    "Integration error. Rotation was not between the expected values.")
  File "/usr/lib/python2.7/unittest/case.py", line 422, in assertTrue
    raise self.failureException(msg)
--------------------------------------------------------------------------------


SUMMARY
 * RESULT: FAIL
 * TESTS: 1
 * ERRORS: 0
 * FAILURES: 1

rostest log file is in /home/user/.ros/log/rostest-1_xterm-12497.log

Here is my rotate_robot_integration_test.test file:

<launch>
    <node pkg="robot_control"
          type="reset_world_client.py"
          name="reset_world_service_client"
          output="screen"/>
    <node pkg="robot_control"
          type="rotate_robot_srv.py"
          name="robot_control_node"
          output="screen"/>
    <node pkg="robot_control"
          type="rotate_robot_srv_client.py"
          name="rotate_robot_service_client"
          output="screen"/>
    <test pkg="robot_control"
          type="rotate_robot_integration_test_ex.py"
          test-name="rotate_robot_integration_test_ex"/>
</launch>

Thanks,
Girish

ralves · November 30, 2022, 11:03pm

Hi girishkumar.kannan

If the test passes when you run it alone (unit test), then it should pass also if running with Integration Test (rostest)

I would recommend you just try to run the test individually to see if it passes.

If for any reason you have timeout-related errors, just try modifying the .test file to increase the timeout for the test.

The timeout is 60 seconds by default. In case you want to increase it to 2 minutes, for example, you can just add time-limit=“120.0” to the rotate_robot_integration_test.test test file. Example:

<test pkg="robot_control"
          type="rotate_robot_integration_test_ex.py"
          time-limit="120.0"
          test-name="rotate_robot_integration_test_ex"/>

Please let us know if you are still having problems to make this test pass.

girishkumar.kannan · December 1, 2022, 8:57am

Hi @ralves ,

Thanks for the reply.

Yes, I have tested the codes individually. They execute fine - without any problems.
I get this error only when I run rostest with the <...>.test file.

I also tried including the timeout as you mentioned, but it had no effect in my opinion.

To check what was happening, I added time.time() to all the files to know in which order they start during the test. I was surprised that even though they started in the right order (after changing the .test file order), the test still failed.
Is it probably because I am using Python 3.x on a course that currently runs on Kinetic and Python 2.7?

My modified rotate_robot_integration_test.test file:

<launch>
    <test pkg="robot_control"
          type="rotate_robot_integration_test_ex.py"
          test-name="rotate_robot_integration_test_ex"
          time-limit="120.0"/>
    <node pkg="robot_control"
          type="rotate_robot_srv.py"
          name="robot_control_node"
          output="screen"/>
    <node pkg="robot_control"
          type="reset_world_client.py"
          name="reset_world_service_client"
          output="screen"/>
    <node pkg="robot_control"
          type="rotate_robot_srv_client.py"
          name="rotate_robot_service_client"
          output="screen"/>
</launch>

My output of rostest robot_control rotate_robot_integration_test.test :

user:~/catkin_ws$ rostest robot_control rotate_robot_integration_test.test
... logging to /home/user/.ros/log/rostest-1_xterm-16664.log
[ROSUNIT] Outputting test results to /home/user/.ros/test_results/robot_control/rostest-test_rotate_robot_integration_test.xml
('Server', 1669884785.70351)
('Reset', 1669884786.01766)
('SrvClient', 1669884786.335594)
30.0
90.0
Clockwise
Traceback (most recent call last):
  File "/home/user/catkin_ws/src/robot_control/src/robot_control/reset_world_client.py", line 12, in <module>
    rospy.wait_for_service('/gazebo/reset_world')
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 159, in wait_for_service
    raise ROSInterruptException("rospy shutdown")
rospy.exceptions.ROSInterruptException: rospy shutdown
Traceback (most recent call last):
  File "/home/user/catkin_ws/src/robot_control/src/robot_control/rotate_robot_srv_client.py", line 22, in <module>
    result = rotate_robot_service(rotate_robot_object)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 435, in __call__
    return self.call(*args, **kwds)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 523, in call
    raise rospy.exceptions.ROSInterruptException("node shutdown interrupted service call")
rospy.exceptions.ROSInterruptException: node shutdown interrupted service call
[Testcase: testrotate_robot_integration_test_ex] ... ok

[ROSTEST]-----------------------------------------------------------------------

[robot_control.rosunit-rotate_robot_integration_test_ex/test_correct_rotation][FAILURE]
Integration error. Rotation was not between the expected values.
  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/home/user/catkin_ws/src/robot_control/test/rotate_robot_integration_test_ex.py", line 45, in test_correct_rotation
    "Integration error. Rotation was not between the expected values.")
  File "/usr/lib/python2.7/unittest/case.py", line 422, in assertTrue
    raise self.failureException(msg)
--------------------------------------------------------------------------------


SUMMARY
 * RESULT: FAIL
 * TESTS: 1
 * ERRORS: 0
 * FAILURES: 1

rostest log file is in /home/user/.ros/log/rostest-1_xterm-16664.log

So, according to the output, Server starts first, followed by Reset World service call, which is then followed by Service Client call.
But the actual problem seems to be related to rospy shutdown. I am assuming that one of the nodes has shutdown during the test.

I am not sure how to fix this. Any advice would be good!

Thanks,
Girish