Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create tutorial about adding product types #1434

Open
wants to merge 8 commits into
base: develop
Choose a base branch
from
Open

Create tutorial about adding product types #1434

wants to merge 8 commits into from

Conversation

amarandon
Copy link
Collaborator

@amarandon amarandon commented Dec 9, 2024 •
edited
Loading

Ref #1432

@amarandon amarandon linked an issue Dec 9, 2024 that may be closed by this pull request
Add documentation about adding a new product to an existing provider #1432
Open
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Dec 9, 2024 •
edited
Loading

Test Results

    4 files  ± 0      4 suites  ±0   6m 11s ⏱️ +3s
  585 tests + 3    582 ✅ + 3   3 💤 ±0  0 ❌ ±0 
2 340 runs  +12  2 242 ✅ +12  98 💤 ±0  0 ❌ ±0 

Results for commit 3e56dd1. ± Comparison against base commit d709297.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Dec 9, 2024 •
edited
Loading

badge

Code Coverage (Ubuntu)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                      8       0  100.00%
cli.py                                         323      64  80.19%   62-63, 87, 674-735, 837-888, 892
config.py                                      429      27  93.71%   83-85, 94, 102, 106-108, 179, 190, 693-695, 808-811, 854-855, 864-865, 970, 1033-1038, 1040
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/core.py                                    782      74  90.54%   317-318, 368, 658, 702-705, 743, 787, 821, 866-871, 897, 988, 1056, 1194, 1279-1291, 1327, 1329, 1357, 1361-1372, 1385-1391, 1474-1477, 1510-1530, 1582, 1599-1603, 1615-1618, 1954, 1978-1984, 2235, 2239-2243, 2252-2254, 2298-2299, 2328-2329
api/search_result.py                            58       4  93.10%   92, 101, 108, 122
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          48       5  89.58%   75, 147, 155, 158-162
api/product/_product.py                        187      20  89.30%   70-72, 237-238, 313, 342, 399, 413-416, 429, 453-456, 499-505
api/product/metadata_mapping.py                700      76  89.14%   131-133, 230, 262-263, 309-310, 320-332, 334, 345, 351-363, 410-411, 448, 469-472, 495, 503-504, 590-591, 615-616, 622-625, 640-641, 790, 836, 989, 998-1002, 1019-1024, 1151, 1165-1185, 1205, 1210, 1339, 1353, 1424, 1476, 1499-1500, 1516-1520, 1536, 1544, 1563
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      6       1  83.33%   38
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 21       2  90.48%   48, 55
plugins/manager.py                             172      15  91.28%   116-121, 171, 209, 231, 235, 259, 281, 399-402, 414-415
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           97      10  89.69%   170-172, 219-220, 246-248, 301-302
plugins/apis/usgs.py                           180      31  82.78%   155, 257, 291, 326-328, 333, 359-360, 365, 395-402, 413-418, 440-446, 448-454, 477
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py              19       0  100.00%
plugins/authentication/base.py                  17       2  88.24%   43, 56
plugins/authentication/generic.py               14       2  85.71%   51, 61
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       7  84.78%   151-154, 175-180
plugins/authentication/oauth.py                 13       7  46.15%   42-44, 48-51
plugins/authentication/openid_connect.py       206      27  86.89%   80-81, 93-111, 149, 155-183, 191, 323-326, 352
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              47       1  97.87%   76
plugins/authentication/token.py                100      16  84.00%   141, 170, 172, 207-220, 276-280
plugins/authentication/token_exchange.py        36      14  61.11%   75, 92-120
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          10       1  90.00%   43
plugins/crunch/filter_date.py                   59      14  76.27%   52-57, 69, 78, 87, 90, 102-104, 113-115, 122
plugins/crunch/filter_latest_intersect.py       47       8  82.98%   52-53, 69, 78-81, 83, 90-93
plugins/crunch/filter_latest_tpl_name.py        31       1  96.77%   83
plugins/crunch/filter_overlap.py                66      18  72.73%   28-30, 66-69, 76-79, 85, 93, 104-120
plugins/crunch/filter_property.py               30       7  76.67%   55-60, 63-64, 80-84
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        491     163  66.80%   282, 295, 362-365, 379-383, 425-427, 431, 463-464, 470-474, 503, 535, 539, 546, 576-584, 588, 620-628, 639-641, 672-746, 764-824, 835-840, 852-865, 890, 905-907, 910, 920-928, 936-949, 959-990, 997-1009, 1047, 1073, 1118-1120, 1340
plugins/download/base.py                       258      49  81.01%   136, 164, 300-301, 355-359, 365-366, 408, 411-425, 437, 441, 505-509, 539-540, 548-565, 572-580, 582-586, 629, 651, 673, 681
plugins/download/creodias_s3.py                 17       9  47.06%   53-67
plugins/download/http.py                       545     106  80.55%   237, 333-336, 339, 346-351, 369-384, 401, 413, 461, 468-474, 492, 506, 520, 528-530, 546-551, 562, 580, 623-626, 648, 688, 733, 747-753, 782-846, 864, 894-903, 939, 964-965, 984-989, 995, 998, 1015, 1032-1033, 1063-1064, 1071, 1133, 1148, 1207-1208, 1214, 1224, 1260, 1296, 1316, 1351-1353
plugins/download/s3rest.py                     116      24  79.31%   118, 152, 159, 194, 221-228, 231-233, 237, 248-254, 262-263, 266-270, 293, 314-317
plugins/search/__init__.py                      22       0  100.00%
plugins/search/base.py                         146      11  92.47%   100, 104, 128-134, 274, 294, 433
plugins/search/build_search_result.py          341      49  85.63%   249, 277-278, 314, 317, 386-389, 478-495, 523, 558, 560, 585-592, 622-624, 628, 650, 685, 736, 765-766, 781-796, 839, 864, 867, 870, 878, 997-998, 1004, 1015, 1073, 1124
plugins/search/cop_marine.py                   240      47  80.42%   56, 64-66, 76-77, 82, 87-88, 104, 106, 109, 175-176, 218, 224, 228, 232, 245, 256-257, 265, 293, 297, 312, 316, 320, 324, 328-332, 338-341, 344-358, 375, 424-428, 433, 445
plugins/search/creodias_s3.py                   55       3  94.55%   59, 77, 111
plugins/search/csw.py                          105      81  22.86%   98-99, 103-104, 112-160, 166-179, 187-219, 237-278
plugins/search/data_request_search.py          202      69  65.84%   188-191, 207, 218, 222-223, 234, 239, 244, 251, 264-267, 321-322, 326, 336-342, 347, 373-376, 384-395, 412, 414, 421-424, 426-427, 445-449, 482, 492, 503, 516, 522-537, 542
plugins/search/qssearch.py                     732      89  87.84%   450, 508, 522, 526-532, 560-564, 677-689, 734-737, 808-809, 857, 876, 883, 895, 952, 973, 976-977, 986-987, 996-997, 1006-1007, 1034, 1105-1110, 1114-1123, 1157, 1179, 1256, 1346, 1431-1432, 1442, 1519-1523, 1585, 1588, 1594-1595, 1616, 1644-1656, 1663, 1695-1697, 1707-1713, 1743, 1766, 1771-1772, 1787, 1803, 1888-1891, 1896-1899, 1908, 1938-1942, 1948
plugins/search/static_stac_search.py            75      13  82.67%   98-124, 140, 153
rest/__init__.py                                 4       2  50.00%   21-22
rest/cache.py                                   33       7  78.79%   35-37, 53-55, 59, 68
rest/config.py                                  25       0  100.00%
rest/constants.py                                6       0  100.00%
rest/core.py                                   260      66  74.62%   258, 266, 284-301, 316-352, 446, 483-522, 705, 712-765
rest/errors.py                                  69       5  92.75%   106, 116, 127, 143-144
rest/server.py                                 192      24  87.50%   94, 117-119, 293-298, 326, 522-524, 541-546, 575, 577, 581-582, 586-587
rest/stac.py                                   319      63  80.25%   306, 328, 380-383, 410-437, 468-470, 493, 525-526, 608-648, 670-686, 778-782, 789, 843-844, 905, 995-997
rest/types/__init__.py                           0       0  100.00%
rest/types/collections_search.py                13      13  0.00%    18-44
rest/types/eodag_search.py                     180       6  96.67%   225-229, 282, 285, 353, 385
rest/types/queryables.py                        57       5  91.23%   93-98, 162
rest/types/stac_search.py                      125       7  94.40%   138, 184, 199-201, 209, 213
rest/utils/__init__.py                          94      12  87.23%   111-112, 131-133, 185, 195-209
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           23       3  86.96%   48, 60, 62
types/__init__.py                              128      42  67.19%   66, 79-83, 94-106, 134-136, 143-148, 189, 208, 230, 240-256, 261, 263, 285, 290, 298, 308
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                            108       1  99.07%   63
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
types/whoosh.py                                 82      16  80.49%   131-134, 138-145, 157-163, 176-178
utils/__init__.py                              520      47  90.96%   85, 201-202, 211-238, 241, 256, 336-340, 413-417, 436-438, 451-465, 544, 559, 599-600, 964-967, 975-976, 1014-1015, 1062-1063, 1197
utils/exceptions.py                             46       0  100.00%
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                28       1  96.43%   41
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   38       8  78.95%   51, 53, 57, 98, 122-129
utils/requests.py                               55      29  47.27%   51-52, 64, 85-96, 107-124, 128
utils/rest.py                                   36       1  97.22%   55
utils/stac_reader.py                           111      45  59.46%   56-57, 63-85, 95-97, 101, 137, 153-156, 203-212, 222-252
TOTAL                                         9971    1661  83.34%

Diff against develop

Filename      Stmts    Miss  Cover
----------  -------  ------  --------
TOTAL             0       0  +100.00%

Results for commit: 3e56dd1

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Dec 9, 2024 •
edited
Loading

badge

Code Coverage (Windows)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                      8       0  100.00%
cli.py                                         323      64  80.19%   62-63, 87, 674-735, 837-888, 892
config.py                                      429      28  93.47%   83-85, 94, 102, 106-108, 179, 190, 693-695, 808-811, 854-855, 864-865, 970, 1001, 1033-1038, 1040
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/core.py                                    782      74  90.54%   317-318, 368, 658, 702-705, 743, 787, 821, 866-871, 897, 988, 1056, 1194, 1279-1291, 1327, 1329, 1357, 1361-1372, 1385-1391, 1474-1477, 1510-1530, 1582, 1599-1603, 1615-1618, 1954, 1978-1984, 2235, 2239-2243, 2252-2254, 2298-2299, 2328-2329
api/search_result.py                            58       4  93.10%   92, 101, 108, 122
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          48       5  89.58%   75, 147, 155, 158-162
api/product/_product.py                        187      20  89.30%   70-72, 237-238, 313, 342, 399, 413-416, 429, 453-456, 499-505
api/product/metadata_mapping.py                700      77  89.00%   131-133, 230, 262-263, 309-310, 320-332, 334, 345, 351-363, 410-411, 448, 469-472, 495, 503-504, 590-591, 615-616, 622-625, 640-641, 790, 836, 989, 998-1002, 1019-1024, 1151, 1165-1185, 1205, 1210, 1339, 1353, 1424, 1476, 1499-1500, 1503, 1516-1520, 1536, 1544, 1563
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      6       1  83.33%   38
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 21       3  85.71%   48, 55, 68
plugins/manager.py                             172      15  91.28%   116-121, 171, 209, 231, 235, 259, 281, 399-402, 414-415
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           97      10  89.69%   170-172, 219-220, 246-248, 301-302
plugins/apis/usgs.py                           180      31  82.78%   155, 257, 291, 326-328, 333, 359-360, 365, 395-402, 413-418, 440-446, 448-454, 477
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py              19       0  100.00%
plugins/authentication/base.py                  17       2  88.24%   43, 56
plugins/authentication/generic.py               14       2  85.71%   51, 61
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       7  84.78%   151-154, 175-180
plugins/authentication/oauth.py                 13       7  46.15%   42-44, 48-51
plugins/authentication/openid_connect.py       206      27  86.89%   80-81, 93-111, 149, 155-183, 191, 323-326, 352
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              47       1  97.87%   76
plugins/authentication/token.py                100      16  84.00%   141, 170, 172, 207-220, 276-280
plugins/authentication/token_exchange.py        36      14  61.11%   75, 92-120
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          10       1  90.00%   43
plugins/crunch/filter_date.py                   59      14  76.27%   52-57, 69, 78, 87, 90, 102-104, 113-115, 122
plugins/crunch/filter_latest_intersect.py       47      33  29.79%   49-54, 67-112
plugins/crunch/filter_latest_tpl_name.py        31       1  96.77%   83
plugins/crunch/filter_overlap.py                66      18  72.73%   28-30, 66-69, 76-79, 85, 93, 104-120
plugins/crunch/filter_property.py               30       7  76.67%   55-60, 63-64, 80-84
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        491     163  66.80%   282, 295, 362-365, 379-383, 425-427, 431, 463-464, 470-474, 503, 535, 539, 546, 576-584, 588, 620-628, 639-641, 672-746, 764-824, 835-840, 852-865, 890, 905-907, 910, 920-928, 936-949, 959-990, 997-1009, 1047, 1073, 1118-1120, 1340
plugins/download/base.py                       258      51  80.23%   136, 164, 231-233, 300-301, 355-359, 365-366, 408, 411-425, 437, 441, 505-509, 539-540, 548-565, 572-580, 582-586, 629, 651, 673, 681
plugins/download/creodias_s3.py                 17       9  47.06%   53-67
plugins/download/http.py                       545     107  80.37%   237, 333-336, 339, 346-351, 369-384, 401, 413, 461, 468-474, 492, 506, 520, 528-530, 546-551, 562, 580, 623-626, 648, 688, 733, 747-753, 782-846, 864, 894-903, 939, 964-965, 984-989, 995, 998, 1015, 1032-1033, 1046, 1063-1064, 1071, 1133, 1148, 1207-1208, 1214, 1224, 1260, 1296, 1316, 1351-1353
plugins/download/s3rest.py                     116      24  79.31%   118, 152, 159, 194, 221-228, 231-233, 237, 248-254, 262-263, 266-270, 293, 314-317
plugins/search/__init__.py                      22       0  100.00%
plugins/search/base.py                         146      11  92.47%   100, 104, 128-134, 274, 294, 433
plugins/search/build_search_result.py          341      50  85.34%   249, 277-278, 314, 317, 386-389, 478-495, 523, 558, 560, 585-592, 622-624, 628, 650, 685, 736, 750, 765-766, 781-796, 839, 864, 867, 870, 878, 997-998, 1004, 1015, 1073, 1124
plugins/search/cop_marine.py                   240      47  80.42%   56, 64-66, 76-77, 82, 87-88, 104, 106, 109, 175-176, 218, 224, 228, 232, 245, 256-257, 265, 293, 297, 312, 316, 320, 324, 328-332, 338-341, 344-358, 375, 424-428, 433, 445
plugins/search/creodias_s3.py                   55       3  94.55%   59, 77, 111
plugins/search/csw.py                          105      81  22.86%   98-99, 103-104, 112-160, 166-179, 187-219, 237-278
plugins/search/data_request_search.py          202      69  65.84%   188-191, 207, 218, 222-223, 234, 239, 244, 251, 264-267, 321-322, 326, 336-342, 347, 373-376, 384-395, 412, 414, 421-424, 426-427, 445-449, 482, 492, 503, 516, 522-537, 542
plugins/search/qssearch.py                     732     125  82.92%   450, 508, 522, 526-532, 560-564, 677-689, 734-737, 808-809, 857, 876, 883, 895, 952, 973, 976-977, 986-987, 996-997, 1006-1007, 1034, 1105-1110, 1114-1123, 1157, 1179, 1256, 1346, 1431-1432, 1442, 1519-1523, 1585, 1588, 1594-1595, 1616, 1644-1656, 1663, 1695-1697, 1707-1713, 1743, 1766, 1771-1772, 1787, 1803, 1871-1981
plugins/search/static_stac_search.py            75      13  82.67%   98-124, 140, 153
rest/__init__.py                                 4       2  50.00%   21-22
rest/cache.py                                   33      22  33.33%   35-37, 44-70
rest/config.py                                  25       1  96.00%   35
rest/constants.py                                6       0  100.00%
rest/core.py                                   260     153  41.15%   157, 159, 161, 164-165, 179-189, 198-199, 205, 208, 249-303, 316-352, 383-420, 435-451, 467-476, 483-522, 539, 581-666, 705, 712-765
rest/errors.py                                  69      49  28.99%   60, 65-100, 105-108, 115-118, 126-147, 155-160, 175-181
rest/server.py                                 192     192  0.00%    18-600
rest/stac.py                                   319      68  78.68%   240, 306, 328, 380-383, 410-437, 468-470, 493, 525-526, 608-648, 670-686, 713, 778-782, 789, 843-844, 850, 905, 943, 976, 995-997
rest/types/__init__.py                           0       0  100.00%
rest/types/collections_search.py                13      13  0.00%    18-44
rest/types/eodag_search.py                     180      20  88.89%   225-229, 262-264, 282, 285, 291, 295, 353, 371-386
rest/types/queryables.py                        57      13  77.19%   50-51, 58-59, 66-67, 93-98, 107-108, 162
rest/types/stac_search.py                      125      11  91.20%   136-138, 184, 199-201, 209, 213, 261, 264
rest/utils/__init__.py                          94      30  68.09%   82-88, 108, 111-112, 131-133, 146, 153, 178-186, 193-214
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           23       5  78.26%   43-44, 48, 60, 62
types/__init__.py                              128      44  65.62%   66, 70, 79-83, 94-106, 134-136, 143-148, 189, 203, 208, 230, 240-256, 261, 263, 285, 290, 298, 308
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                            108       1  99.07%   63
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
types/whoosh.py                                 82      16  80.49%   131-134, 138-145, 157-163, 176-178
utils/__init__.py                              520      47  90.96%   85, 201-202, 211-238, 241, 256, 336-340, 413-417, 436-438, 451-465, 544, 559, 599-600, 964-967, 975-976, 1014-1015, 1062-1063, 1197
utils/exceptions.py                             46       0  100.00%
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                28       1  96.43%   41
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   38       8  78.95%   51, 53, 57, 98, 122-129
utils/requests.py                               55      29  47.27%   51-52, 64, 85-96, 107-124, 128
utils/rest.py                                   36       1  97.22%   55
utils/stac_reader.py                           111      45  59.46%   56-57, 63-85, 95-97, 101, 137, 153-156, 203-212, 222-252
TOTAL                                         9971    2097  78.97%

Diff against develop

Filename      Stmts    Miss  Cover
----------  -------  ------  --------
TOTAL             0       0  +100.00%

Results for commit: 3e56dd1

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@amarandon amarandon requested a review from jlahovnik December 10, 2024 15:49
@amarandon amarandon self-assigned this Dec 10, 2024
jlahovnik
jlahovnik requested changes Dec 10, 2024
View reviewed changes
docs/add_product_type.rst Outdated Show resolved Hide resolved
docs/add_product_type.rst Outdated Show resolved Hide resolved
@amarandon amarandon requested a review from jlahovnik December 11, 2024 08:58
sbrunato
sbrunato requested changes Dec 18, 2024
View reviewed changes
docs/add_product_type.rst Outdated Show resolved Hide resolved
docs/add_product_type.rst Outdated Show resolved Hide resolved
docs/add_product_type.rst Outdated Show resolved Hide resolved
docs/add_product_type.rst Outdated Show resolved Hide resolved
@sbrunato sbrunato marked this pull request as draft December 18, 2024 16:34
amarandon and others added 4 commits December 19, 2024 15:54
@amarandon amarandon marked this pull request as ready for review December 19, 2024 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sbrunato sbrunato sbrunato requested changes

@jlahovnik jlahovnik jlahovnik approved these changes

Requested changes must be addressed to merge this pull request.

Assignees

@amarandon amarandon

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add documentation about adding a new product to an existing provider
3 participants
@amarandon @sbrunato @jlahovnik