Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

suggest a build option for slow tests #266

Closed
edwardhartnett opened this issue Dec 18, 2024 · 1 comment · Fixed by #271
Closed

suggest a build option for slow tests #266

edwardhartnett opened this issue Dec 18, 2024 · 1 comment · Fixed by #271
Assignees
@AlexanderRichert-NOAA
Labels
build Building software is complicated!

Comments

@edwardhartnett
Copy link
Contributor

There are some slow tests in ip. I suggest you add an option (RUN_SLOW_TESTS or something like that), and only run the tests when that option is ON, and have it OFF by default.

Then make sure the slow tests are turned on in at least one CI workflow.

That will make build+test faster for spack-based builds. Also long test times discourage users from running the tests after building.
@edwardhartnett edwardhartnett added the build Building software is complicated! label Dec 18, 2024
@edwardhartnett
Copy link
Contributor Author 

(env) ed@Pooh-Bah:~/Downloads/NCEPLIBS-ip-5.1.0/b$ make -j test
Running tests...
Test project /home/ed/Downloads/NCEPLIBS-ip-5.1.0/b
        Start   1: test_ipxwafs_4
  1/110 Test   #1: test_ipxwafs_4 ...........................................   Passed    0.00 sec
        Start   2: test_earth_radius_4
  2/110 Test   #2: test_earth_radius_4 ......................................   Passed    0.00 sec
        Start   3: tst_gdswzd_c_grib2_4
  3/110 Test   #3: tst_gdswzd_c_grib2_4 .....................................   Passed    0.04 sec
        Start   4: test_lambert_bilinear_scalar_grib2_4
  4/110 Test   #4: test_lambert_bilinear_scalar_grib2_4 .....................   Passed    0.13 sec
        Start   5: test_gaussian_neighbor_scalar_grib2_4
  5/110 Test   #5: test_gaussian_neighbor_scalar_grib2_4 ....................   Passed    0.13 sec
        Start   6: test_latlon_bilinear_scalar_grib2_4
  6/110 Test   #6: test_latlon_bilinear_scalar_grib2_4 ......................   Passed    0.08 sec
        Start   7: test_mercator_bicubic_scalar_grib2_4
  7/110 Test   #7: test_mercator_bicubic_scalar_grib2_4 .....................   Passed    0.01 sec
        Start   8: test_polar_stereo_neighbor_budget_scalar_grib2_4
  8/110 Test   #8: test_polar_stereo_neighbor_budget_scalar_grib2_4 .........   Passed    0.98 sec
        Start   9: test_rotatedB_spectral_scalar_grib2_4
  9/110 Test   #9: test_rotatedB_spectral_scalar_grib2_4 ....................   Passed   73.38 sec
        Start  10: test_rotatedE_budget_scalar_grib2_4
 10/110 Test  #10: test_rotatedE_budget_scalar_grib2_4 ......................   Passed    5.64 sec
        Start  11: test_station_points_bilinear_scalar_grib2_4
 11/110 Test  #11: test_station_points_bilinear_scalar_grib2_4 ..............   Passed    0.01 sec
        Start  12: test_station_points_bicubic_scalar_grib2_4
 12/110 Test  #12: test_station_points_bicubic_scalar_grib2_4 ...............   Passed    0.00 sec
        Start  13: test_station_points_neighbor_scalar_grib2_4
 13/110 Test  #13: test_station_points_neighbor_scalar_grib2_4 ..............   Passed    0.00 sec
        Start  14: test_station_points_budget_scalar_grib2_4
 14/110 Test  #14: test_station_points_budget_scalar_grib2_4 ................   Passed    0.00 sec
        Start  15: test_station_points_spectral_scalar_grib2_4
 15/110 Test  #15: test_station_points_spectral_scalar_grib2_4 ..............   Passed    0.05 sec
        Start  16: test_station_points_neighbor_budget_scalar_grib2_4
 16/110 Test  #16: test_station_points_neighbor_budget_scalar_grib2_4 .......   Passed    0.00 sec
        Start  17: test_lambert_bilinear_vector_grib2_4
 17/110 Test  #17: test_lambert_bilinear_vector_grib2_4 .....................   Passed    0.29 sec
        Start  18: test_gaussian_neighbor_vector_grib2_4
 18/110 Test  #18: test_gaussian_neighbor_vector_grib2_4 ....................   Passed    0.14 sec
        Start  19: test_latlon_bilinear_vector_grib2_4
 19/110 Test  #19: test_latlon_bilinear_vector_grib2_4 ......................   Passed    0.13 sec
        Start  20: test_mercator_bicubic_vector_grib2_4
 20/110 Test  #20: test_mercator_bicubic_vector_grib2_4 .....................   Passed    0.06 sec
        Start  21: test_polar_stereo_neighbor_budget_vector_grib2_4
 21/110 Test  #21: test_polar_stereo_neighbor_budget_vector_grib2_4 .........   Passed    1.83 sec
        Start  22: test_rotatedB_spectral_vector_grib2_4
 22/110 Test  #22: test_rotatedB_spectral_vector_grib2_4 ....................   Passed  114.13 sec
        Start  23: test_rotatedE_budget_vector_grib2_4
 23/110 Test  #23: test_rotatedE_budget_vector_grib2_4 ......................   Passed   15.08 sec
        Start  24: test_rotatedB_direct_spectral_vector_grib2_4
 24/110 Test  #24: test_rotatedB_direct_spectral_vector_grib2_4 .............   Passed  114.31 sec
        Start  25: test_rotatedB_direct_ncep_post_spectral_vector_grib2_4
 25/110 Test  #25: test_rotatedB_direct_ncep_post_spectral_vector_grib2_4 ...   Passed  108.63 sec
        Start  26: test_rotatedE_direct_budget_vector_grib2_4
 26/110 Test  #26: test_rotatedE_direct_budget_vector_grib2_4 ...............   Passed   15.29 sec
        Start  27: test_station_points_bilinear_vector_grib2_4
 27/110 Test  #27: test_station_points_bilinear_vector_grib2_4 ..............   Passed    0.01 sec
        Start  28: test_station_points_bicubic_vector_grib2_4
 28/110 Test  #28: test_station_points_bicubic_vector_grib2_4 ...............   Passed    0.01 sec
        Start  29: test_station_points_neighbor_vector_grib2_4
 29/110 Test  #29: test_station_points_neighbor_vector_grib2_4 ..............   Passed    0.01 sec
        Start  30: test_station_points_budget_vector_grib2_4
 30/110 Test  #30: test_station_points_budget_vector_grib2_4 ................   Passed    0.01 sec
        Start  31: test_station_points_spectral_vector_grib2_4
 31/110 Test  #31: test_station_points_spectral_vector_grib2_4 ..............   Passed    0.04 sec
        Start  32: test_station_points_neighbor_budget_vector_grib2_4
 32/110 Test  #32: test_station_points_neighbor_budget_vector_grib2_4 .......   Passed    0.01 sec
        Start  33: tst_gdswzd_c_grib1_4
 33/110 Test  #33: tst_gdswzd_c_grib1_4 .....................................   Passed    0.02 sec
        Start  34: test_lambert_bilinear_scalar_grib1_4
 34/110 Test  #34: test_lambert_bilinear_scalar_grib1_4 .....................   Passed    0.16 sec
        Start  35: test_gaussian_neighbor_scalar_grib1_4
 35/110 Test  #35: test_gaussian_neighbor_scalar_grib1_4 ....................   Passed    0.13 sec
        Start  36: test_latlon_bilinear_scalar_grib1_4
 36/110 Test  #36: test_latlon_bilinear_scalar_grib1_4 ......................   Passed    0.08 sec
        Start  37: test_mercator_bicubic_scalar_grib1_4
 37/110 Test  #37: test_mercator_bicubic_scalar_grib1_4 .....................   Passed    0.03 sec
        Start  38: test_polar_stereo_neighbor_budget_scalar_grib1_4
 38/110 Test  #38: test_polar_stereo_neighbor_budget_scalar_grib1_4 .........   Passed    0.96 sec
        Start  39: test_rotatedB_spectral_scalar_grib1_4
 39/110 Test  #39: test_rotatedB_spectral_scalar_grib1_4 ....................   Passed   73.65 sec
        Start  40: test_rotatedE_budget_scalar_grib1_4
 40/110 Test  #40: test_rotatedE_budget_scalar_grib1_4 ......................   Passed    5.56 sec
        Start  41: test_lambert_bilinear_vector_grib1_4
 41/110 Test  #41: test_lambert_bilinear_vector_grib1_4 .....................   Passed    0.31 sec
        Start  42: test_gaussian_neighbor_vector_grib1_4
 42/110 Test  #42: test_gaussian_neighbor_vector_grib1_4 ....................   Passed    0.16 sec
        Start  43: test_latlon_bilinear_vector_grib1_4
 43/110 Test  #43: test_latlon_bilinear_vector_grib1_4 ......................   Passed    0.10 sec
        Start  44: test_mercator_bicubic_vector_grib1_4
 44/110 Test  #44: test_mercator_bicubic_vector_grib1_4 .....................   Passed    0.03 sec
        Start  45: test_polar_stereo_neighbor_budget_vector_grib1_4
 45/110 Test  #45: test_polar_stereo_neighbor_budget_vector_grib1_4 .........   Passed    1.80 sec
        Start  46: test_rotatedB_spectral_vector_grib1_4
 46/110 Test  #46: test_rotatedB_spectral_vector_grib1_4 ....................   Passed  113.70 sec
        Start  47: test_rotatedE_budget_vector_grib1_4
 47/110 Test  #47: test_rotatedE_budget_vector_grib1_4 ......................   Passed   15.30 sec
        Start  48: test_ncpus_4
 48/110 Test  #48: test_ncpus_4 .............................................   Passed    0.00 sec
        Start  49: test_splaplac_4
 49/110 Test  #49: test_splaplac_4 ..........................................   Passed    0.00 sec
        Start  50: test_splat_4
 50/110 Test  #50: test_splat_4 .............................................   Passed    0.01 sec
        Start  51: test_sppad_4
 51/110 Test  #51: test_sppad_4 .............................................   Passed    0.02 sec
        Start  52: test_sptezv_4
 52/110 Test  #52: test_sptezv_4 ............................................   Passed    0.00 sec
        Start  53: test_fft_4
 53/110 Test  #53: test_fft_4 ...............................................   Passed    0.00 sec
        Start  54: test_sptrung_4
 54/110 Test  #54: test_sptrung_4 ...........................................   Passed    0.04 sec
        Start  55: test_sptrungv_4
 55/110 Test  #55: test_sptrungv_4 ..........................................   Passed    1.96 sec
        Start  56: test_ipxwafs_d
 56/110 Test  #56: test_ipxwafs_d ...........................................   Passed    0.00 sec
        Start  57: test_earth_radius_d
 57/110 Test  #57: test_earth_radius_d ......................................   Passed    0.00 sec
        Start  58: tst_gdswzd_c_grib2_d
 58/110 Test  #58: tst_gdswzd_c_grib2_d .....................................   Passed    0.02 sec
        Start  59: test_lambert_bilinear_scalar_grib2_d
 59/110 Test  #59: test_lambert_bilinear_scalar_grib2_d .....................   Passed    0.13 sec
        Start  60: test_gaussian_neighbor_scalar_grib2_d
 60/110 Test  #60: test_gaussian_neighbor_scalar_grib2_d ....................   Passed    0.10 sec
        Start  61: test_latlon_bilinear_scalar_grib2_d
 61/110 Test  #61: test_latlon_bilinear_scalar_grib2_d ......................   Passed    0.04 sec
        Start  62: test_mercator_bicubic_scalar_grib2_d
 62/110 Test  #62: test_mercator_bicubic_scalar_grib2_d .....................   Passed    0.01 sec
        Start  63: test_polar_stereo_neighbor_budget_scalar_grib2_d
 63/110 Test  #63: test_polar_stereo_neighbor_budget_scalar_grib2_d .........   Passed    0.99 sec
        Start  64: test_rotatedB_spectral_scalar_grib2_d
 64/110 Test  #64: test_rotatedB_spectral_scalar_grib2_d ....................   Passed   69.97 sec
        Start  65: test_rotatedE_budget_scalar_grib2_d
 65/110 Test  #65: test_rotatedE_budget_scalar_grib2_d ......................   Passed    5.53 sec
        Start  66: test_station_points_bilinear_scalar_grib2_d
 66/110 Test  #66: test_station_points_bilinear_scalar_grib2_d ..............   Passed    0.01 sec
        Start  67: test_station_points_bicubic_scalar_grib2_d
 67/110 Test  #67: test_station_points_bicubic_scalar_grib2_d ...............   Passed    0.00 sec
        Start  68: test_station_points_neighbor_scalar_grib2_d
 68/110 Test  #68: test_station_points_neighbor_scalar_grib2_d ..............   Passed    0.00 sec
        Start  69: test_station_points_budget_scalar_grib2_d
 69/110 Test  #69: test_station_points_budget_scalar_grib2_d ................   Passed    0.01 sec
        Start  70: test_station_points_spectral_scalar_grib2_d
 70/110 Test  #70: test_station_points_spectral_scalar_grib2_d ..............   Passed    0.04 sec
        Start  71: test_station_points_neighbor_budget_scalar_grib2_d
 71/110 Test  #71: test_station_points_neighbor_budget_scalar_grib2_d .......   Passed    0.01 sec
        Start  72: test_lambert_bilinear_vector_grib2_d
 72/110 Test  #72: test_lambert_bilinear_vector_grib2_d .....................   Passed    0.33 sec
        Start  73: test_gaussian_neighbor_vector_grib2_d
 73/110 Test  #73: test_gaussian_neighbor_vector_grib2_d ....................   Passed    0.17 sec
        Start  74: test_latlon_bilinear_vector_grib2_d
 74/110 Test  #74: test_latlon_bilinear_vector_grib2_d ......................   Passed    0.08 sec
        Start  75: test_mercator_bicubic_vector_grib2_d
 75/110 Test  #75: test_mercator_bicubic_vector_grib2_d .....................   Passed    0.04 sec
        Start  76: test_polar_stereo_neighbor_budget_vector_grib2_d
 76/110 Test  #76: test_polar_stereo_neighbor_budget_vector_grib2_d .........   Passed    1.80 sec
        Start  77: test_rotatedB_spectral_vector_grib2_d
 77/110 Test  #77: test_rotatedB_spectral_vector_grib2_d ....................   Passed  112.25 sec
        Start  78: test_rotatedE_budget_vector_grib2_d
 78/110 Test  #78: test_rotatedE_budget_vector_grib2_d ......................   Passed   15.89 sec
        Start  79: test_rotatedB_direct_spectral_vector_grib2_d
 79/110 Test  #79: test_rotatedB_direct_spectral_vector_grib2_d .............   Passed  113.18 sec
        Start  80: test_rotatedB_direct_ncep_post_spectral_vector_grib2_d
  C-c C-cmake: *** [Makefile:71: test] Interrupt

I suggest the ones that take longer than a minute be categorized a slow tests...

Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@AlexanderRichert-NOAA AlexanderRichert-NOAA

Labels
build Building software is complicated!
Projects
NCEPLIBS-ip-5.2.0
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

add label for slow-running (rotatedB) code tests AlexanderRichert-NOAA/NCEPLIBS-ip
2 participants
@edwardhartnett @AlexanderRichert-NOAA