Reject products with missing expected layers

[sssangha]

In cases where there are multiple scenes for a given pair and at least one has missing expected layers, then this leads to a crash later when the program attempts to sort the dictionary of products.

Specifically, compare this product with missing tropo/SET/iono layers:

(base) ➜  products gdalinfo S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc
Driver: netCDF/Network Common Data Format
Files: S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc
Size is 512, 512
Metadata:
  NC_GLOBAL#author=David Bekaert, Charlie Marshak, Simran Sangha, Joe Kennedy, Andrew Johnston
  NC_GLOBAL#Conventions=CF-1.6
  NC_GLOBAL#institution=Jet Propulsion Laboratory
  NC_GLOBAL#ogr_geometry_field=productBoundingBox
  NC_GLOBAL#ogr_layer_name=productBoundingBox
  NC_GLOBAL#ogr_layer_type=POLYGON
  NC_GLOBAL#product_type=UNW GEO IFG
  NC_GLOBAL#references=https://aria.jpl.nasa.gov/
  NC_GLOBAL#source=Contains modified Copernicus Sentinel data processed by ESA and ARIA NASA/JPL using the DockerizedTopsApp HyP3 plugin version 0.2.1
  NC_GLOBAL#title=ARIA standard product UNW GEO IFG
  NC_GLOBAL#version=1c
Subdatasets:
  SUBDATASET_1_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":productBoundingBox
  SUBDATASET_1_DESC=[1x460] productBoundingBox (8-bit character)
  SUBDATASET_2_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/data/unwrappedPhase
  SUBDATASET_2_DESC=[2461x3845] unwrappedPhase (32-bit floating-point)
  SUBDATASET_3_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/data/coherence
  SUBDATASET_3_DESC=[2461x3845] coherence (32-bit floating-point)
  SUBDATASET_4_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/data/connectedComponents
  SUBDATASET_4_DESC=[2461x3845] connectedComponents (32-bit floating-point)
  SUBDATASET_5_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/data/amplitude
  SUBDATASET_5_DESC=[2461x3845] amplitude (32-bit floating-point)
  SUBDATASET_6_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/imagingGeometry/perpendicularBaseline
  SUBDATASET_6_DESC=[4x27x37] perpendicularBaseline (32-bit floating-point)
  SUBDATASET_7_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/imagingGeometry/parallelBaseline
  SUBDATASET_7_DESC=[4x27x37] parallelBaseline (32-bit floating-point)
  SUBDATASET_8_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/imagingGeometry/incidenceAngle
  SUBDATASET_8_DESC=[4x27x37] incidenceAngle (32-bit floating-point)
  SUBDATASET_9_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/imagingGeometry/lookAngle
  SUBDATASET_9_DESC=[4x27x37] lookAngle (32-bit floating-point)
  SUBDATASET_10_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/imagingGeometry/azimuthAngle
  SUBDATASET_10_DESC=[4x27x37] azimuthAngle (32-bit floating-point)
  SUBDATASET_11_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/corrections/external/troposphere/troposphereWet
  SUBDATASET_11_DESC=[20x43x64] troposphereWet (32-bit floating-point)
  SUBDATASET_12_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/corrections/external/troposphere/troposphereHydrostatic
  SUBDATASET_12_DESC=[20x43x64] troposphereHydrostatic (32-bit floating-point)

vs this overlapping product with all expected layers available:

(base) ➜  products gdalinfo S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc
Driver: netCDF/Network Common Data Format
Files: S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc
Size is 512, 512
Metadata:
  NC_GLOBAL#author=David Bekaert, Charlie Marshak, Simran Sangha, Joe Kennedy, Andrew Johnston
  NC_GLOBAL#Conventions=CF-1.6
  NC_GLOBAL#institution=Jet Propulsion Laboratory
  NC_GLOBAL#ogr_geometry_field=productBoundingBox
  NC_GLOBAL#ogr_layer_name=productBoundingBox
  NC_GLOBAL#ogr_layer_type=POLYGON
  NC_GLOBAL#product_type=UNW GEO IFG
  NC_GLOBAL#references=https://aria.jpl.nasa.gov/
  NC_GLOBAL#source=Contains modified Copernicus Sentinel data processed by ESA and ARIA NASA/JPL using the DockerizedTopsApp HyP3 plugin version 0.2.5
  NC_GLOBAL#title=ARIA standard product UNW GEO IFG
  NC_GLOBAL#version=1c
Subdatasets:
  SUBDATASET_1_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":productBoundingBox
  SUBDATASET_1_DESC=[1x466] productBoundingBox (8-bit character)
  SUBDATASET_2_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/data/unwrappedPhase
  SUBDATASET_2_DESC=[2222x3916] unwrappedPhase (32-bit floating-point)
  SUBDATASET_3_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/data/coherence
  SUBDATASET_3_DESC=[2222x3916] coherence (32-bit floating-point)
  SUBDATASET_4_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/data/connectedComponents
  SUBDATASET_4_DESC=[2222x3916] connectedComponents (32-bit floating-point)
  SUBDATASET_5_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/data/amplitude
  SUBDATASET_5_DESC=[2222x3916] amplitude (32-bit floating-point)
  SUBDATASET_6_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/imagingGeometry/perpendicularBaseline
  SUBDATASET_6_DESC=[4x29x38] perpendicularBaseline (32-bit floating-point)
  SUBDATASET_7_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/imagingGeometry/parallelBaseline
  SUBDATASET_7_DESC=[4x29x38] parallelBaseline (32-bit floating-point)
  SUBDATASET_8_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/imagingGeometry/incidenceAngle
  SUBDATASET_8_DESC=[4x29x38] incidenceAngle (32-bit floating-point)
  SUBDATASET_9_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/imagingGeometry/lookAngle
  SUBDATASET_9_DESC=[4x29x38] lookAngle (32-bit floating-point)
  SUBDATASET_10_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/imagingGeometry/azimuthAngle
  SUBDATASET_10_DESC=[4x29x38] azimuthAngle (32-bit floating-point)
  SUBDATASET_11_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/corrections/derived/ionosphere/ionosphere
  SUBDATASET_11_DESC=[201x355] ionospherePhaseCorrection (32-bit floating-point)
  SUBDATASET_12_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/corrections/derived/ionosphereBurstRamps/ionosphereBurstRamps
  SUBDATASET_12_DESC=[2222x3916] ionosphereBurstRamps (32-bit floating-point)
  SUBDATASET_13_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/corrections/external/troposphere/HRRR/reference/troposphereWet
  SUBDATASET_13_DESC=[20x40x67] troposphereWet (32-bit floating-point)
  SUBDATASET_14_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/corrections/external/troposphere/HRRR/reference/troposphereHydrostatic
  SUBDATASET_14_DESC=[20x40x67] troposphereHydrostatic (32-bit floating-point)
  SUBDATASET_15_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/corrections/external/troposphere/HRRR/secondary/troposphereWet
  SUBDATASET_15_DESC=[20x40x67] troposphereWet (32-bit floating-point)
  SUBDATASET_16_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/corrections/external/troposphere/HRRR/secondary/troposphereHydrostatic
  SUBDATASET_16_DESC=[20x40x67] troposphereHydrostatic (32-bit floating-point)
  SUBDATASET_17_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/corrections/external/tides/solidEarth/reference/solidEarthTide
  SUBDATASET_17_DESC=[4x29x38] solidEarthTide (32-bit floating-point)
  SUBDATASET_18_NAME=NETCDF:"S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/corrections/external/tides/solidEarth/secondary/solidEarthTide
  SUBDATASET_18_DESC=[4x29x38] solidEarthTide (32-bit floating-point)

Downstream sorting of these products for the same IFG leads to this error (note e.g. the eroneous assignment of the productBoundbox layer to the solidEarthTide in the absence of a solidEarthTide layer for this product):

item[1] {'productBoundingBox': ['NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":productBoundingBox', 'NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":productBoundingBox'], 'unwrappedPhase': ['NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/data/unwrappedPhase', 'NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/data/unwrappedPhase'], 'coherence': ['NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/data/coherence', 'NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/data/coherence'], 'connectedComponents': ['NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/data/connectedComponents', 'NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/data/connectedComponents'], 'amplitude': ['NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/data/amplitude', 'NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/data/amplitude'], 'bPerpendicular': ['NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/imagingGeometry/perpendicularBaseline', 'NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/imagingGeometry/perpendicularBaseline'], 'bParallel': ['NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/imagingGeometry/parallelBaseline', 'NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/imagingGeometry/parallelBaseline'], 'incidenceAngle': ['NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/imagingGeometry/incidenceAngle', 'NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/imagingGeometry/incidenceAngle'], 'lookAngle': ['NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/imagingGeometry/lookAngle', 'NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/imagingGeometry/lookAngle'], 'azimuthAngle': ['NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/imagingGeometry/azimuthAngle', 'NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":/science/grids/imagingGeometry/azimuthAngle'], 'ionosphere': ['NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/corrections/derived/ionosphere/ionosphere', '20220623_20220611'], 'solidEarthTide': ['NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015847-00121W_00033N-PP-d5bd-v3_0_0.nc":/science/grids/corrections/external/tides/solidEarth/reference/solidEarthTide', 'NETCDF:"/u/leffe-data2/dbekaert/APS_analysis/HRRR_CA_tracks/A137/products/S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc":productBoundingBox']}
Traceback (most recent call last):
  File "/u/leffe-data/ssangha/conda_installation/stable_july30_2020/ARIA-tools/tools/ARIAtools/ARIAProduct.py", line 725, in __continuous_time__
    [item[1] for item in sorted_products \
  File "/u/leffe-data/ssangha/conda_installation/stable_july30_2020/ARIA-tools/tools/ARIAtools/ARIAProduct.py", line 726, in <listcomp>
    if (item[1]['pair_name'][0] \
KeyError: 'pair_name'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/u/leffe-data/ssangha/conda_installation/stable_july30_2020/envs/ARIA-tools/bin/ariaExtract.py", line 7, in <module>
    exec(compile(f.read(), __file__, 'exec'))
  File "/u/leffe-data/ssangha/conda_installation/stable_july30_2020/ARIA-tools/tools/bin/ariaExtract.py", line 21, in <module>
    main(inps)
  File "/u/leffe-data/ssangha/conda_installation/stable_july30_2020/ARIA-tools/tools/ARIAtools/extractProduct.py", line 1921, in main
    standardproduct_info = ARIA_standardproduct(inps.imgfile,
  File "/u/leffe-data/ssangha/conda_installation/stable_july30_2020/ARIA-tools/tools/ARIAtools/ARIAProduct.py", line 208, in __init__
    self.__run__()
  File "/u/leffe-data/ssangha/conda_installation/stable_july30_2020/ARIA-tools/tools/ARIAtools/ARIAProduct.py", line 804, in __run__
    self.products = self.__continuous_time__()
  File "/u/leffe-data/ssangha/conda_installation/stable_july30_2020/ARIA-tools/tools/ARIAtools/ARIAProduct.py", line 733, in __continuous_time__
    print('pair_name!!!!', item[1]['pair_name'][0]) #!#
KeyError: 'pair_name'

[sssangha]

I should also mention for at least this case, this particular problematic product (red transparent frame below) is unusual in that it doesn't appear to be an fixed frame adjacent to others, but rather spanning between two overlapping products

@dbekaert @bbuzz31 @mgovorcin @cmarshak

[bbuzz31]

@sssangha interesting as I've processed thousands of ifgs over the East Coast and have not hit this. Are you certain it's not unique to this ifg / frame?

[cmarshak]

The product you mentioned was processed during earlier validation and testing this year. Please filter these out. I think we must have a small handful that were delivered to UAT as a test and/or by accident.

This is a good thing to keep in mind. Sorry for this extra overhead. You can filter out using ASF metadata as indicated below, use products that were processed/generated in the last week.

import asf_search as asf
opts = asf.ASFSearchOptions(host='cmr.uat.earthdata.nasa.gov', 
                            platform=asf.SENTINEL1,
                            processingLevel=asf.constants.GUNW_STD,
                            )
resp = asf.granule_search(['S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6'], opts=opts)
resp[0].properties

{'beamModeType': 'slc',
 'browse': ['https://grfn-public-test.asf.alaska.edu/S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.png'],
 'bytes': 57082850,
 'centerLat': None,
 'centerLon': None,
 'faradayRotation': None,
 'fileID': 'S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6',
 'flightDirection': 'ascending',
 'groupID': 'S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6',
 'granuleType': None,
 'insarStackId': None,
 'md5sum': None,
 'offNadirAngle': None,
 'orbit': 43784,
 'pathNumber': 137,
 'platform': 'Sentinel-1 Interferogram (BETA)',
 'pointingAngle': None,
 'polarization': 'HH',
 'processingDate': '2023-01-31T03:16:58.796Z',
 'processingLevel': 'GUNW_STD',
 'sceneName': 'S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6',
 'sensor': 'SENTINEL-1A C-Band SAR',
 'startTime': '2022-06-23T01:58:47.000Z',
 'stopTime': '2022-06-23T01:59:15.000Z',
 'url': 'https://grfn-test.asf.alaska.edu/door/download/S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc',
 'pgeVersion': None,
 'fileName': 'S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc',
 'frameNumber': None}

Note the processing date of this problematic product. Also, regarding the spatial issue, note that the product doesn't have a frame number in the json umm metadata accessed via resp[0].umm meaning it's likely delivered by accident.

{'TemporalExtent': {'RangeDateTime': {'BeginningDateTime': '2022-06-23T01:58:47.000Z',
   'EndingDateTime': '2022-06-23T01:59:15.000Z'}},
 'OrbitCalculatedSpatialDomains': [{'OrbitNumber': 43784},
  {'OrbitNumber': 43609}],
 'GranuleUR': 'S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6',
 'AdditionalAttributes': [{'Name': 'GROUP_ID',
   'Values': ['S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6']},
  {'Name': 'ASCENDING_DESCENDING', 'Values': ['ascending']},
  {'Name': 'BEAM_MODE_TYPE', 'Values': ['slc']},
  {'Name': 'BEAM_MODE', 'Values': ['IW']},
  {'Name': 'BEAM_MODE_DESC', 'Values': ['interferogram']},
  {'Name': 'POLARIZATION', 'Values': ['HH']},
  {'Name': 'LOOK_DIRECTION', 'Values': ['right']},
  {'Name': 'PATH_NUMBER', 'Values': ['137']},
  {'Name': 'BYTES', 'Values': ['57082850']},
  {'Name': 'NEAR_START_LON', 'Values': ['-121.142609']},
  {'Name': 'NEAR_START_LAT', 'Values': ['33.39402660826663']},
  {'Name': 'FAR_START_LON', 'Values': ['-118.024261']},
  {'Name': 'FAR_START_LAT', 'Values': ['33.39402660826663']},
  {'Name': 'FAR_END_LON', 'Values': ['-118.024261']},
  {'Name': 'FAR_END_LAT', 'Values': ['35.47822967971899']},
  {'Name': 'NEAR_END_LON', 'Values': ['-121.142609']},
  {'Name': 'NEAR_END_LAT', 'Values': ['35.47822967971899']},
  {'Name': 'ASF_PLATFORM', 'Values': ['Sentinel-1 Interferogram (BETA)']},
  {'Name': 'PROCESSING_TYPE', 'Values': ['GUNW_STD']},
  {'Name': 'PROCESSING_TYPE_DISPLAY', 'Values': ['Standard Product, NetCDF']},
  {'Name': 'PROCESSING_DESCRIPTION',
   'Values': ['Sentinel-1 SLC interferometric products generated by JPL using ISCE v2.0.0, delivered by ASF']},
  {'Name': 'THUMBNAIL_URL',
   'Values': ['https://grfn-public-test.asf.alaska.edu/S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.png']},
  {'Name': 'PERPENDICULAR_BASELINE', 'Values': ['46.5763']},
  {'Name': 'MISSION_NAME', 'Values': ['S1 I-grams (BETA) - Southern CA']},
  {'Name': 'VERSION', 'Values': ['2.0.6']}],
 'SpatialExtent': {'HorizontalSpatialDomain': {'Geometry': {'GPolygons': [{'Boundary': {'Points': [{'Longitude': -118.3591877637817,
         'Latitude': 35.47822967971899},
        {'Longitude': -121.142609, 'Latitude': 35.074505},
        {'Longitude': -120.74997073103683, 'Latitude': 33.39402660826663},
        {'Longitude': -118.024261, 'Latitude': 33.799786},
        {'Longitude': -118.3591877637817,
         'Latitude': 35.47822967971899}]}}]}}},
 'ProviderDates': [{'Date': '2023-04-28T00:31:41.000Z', 'Type': 'Insert'},
  {'Date': '2023-04-28T00:31:41.000Z', 'Type': 'Update'}],
 'CollectionReference': {'EntryTitle': 'Sentinel-1 Interferograms (BETA)'},
 'RelatedUrls': [{'Format': 'Not provided',
   'Type': 'GET DATA',
   'URL': 'https://grfn-test.asf.alaska.edu/door/download/S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.nc'},
  {'Format': 'Not provided',
   'Type': 'GET RELATED VISUALIZATION',
   'URL': 'https://grfn-public-test.asf.alaska.edu/S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6.png'}],
 'DataGranule': {'DayNightFlag': 'Unspecified',
  'Identifiers': [{'Identifier': 'S1-GUNW-A-R-137-tops-20220623_20220611-015901-00121W_00033N-PP-9a0a-v2_0_6',
    'IdentifierType': 'ProducerGranuleId'}],
  'ProductionDateTime': '2023-01-31T03:16:58.796Z',
  'ArchiveAndDistributionInformation': [{'Name': 'Not provided',
    'Size': 54.43844795227051,
    'SizeUnit': 'MB',
    'Format': 'Not provided'}]},
 'Platforms': [{'ShortName': 'SENTINEL-1A',
   'Instruments': [{'ShortName': 'SENTINEL-1A C-Band SAR',
     'ComposedOf': [{'ShortName': 'IW'}]}]}]}

The products that were meant to be there will have FRAME_NUMBER in attribute.

If there are a number of products, we can chat, but I would filter out by processing date. You could probably use ASF search directly using one of the keyword arguments directly.

I definitely did several hyp3 tests over one of the Los Angeles tracks. I can't imagine there are more than a handful of products because the tests were done without UAT delivery, I think. If there are more, please let us know. Will be good to have an idea when we officially deliver them to the DAAC.