Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RuntimeError: Terminating translation prematurely after failed translation of layer #46

Open
JinIgarashi opened this issue Nov 2, 2023 · 3 comments
Assignees
Labels
bug Something isn't working

Comments

@JinIgarashi
Copy link
Contributor

JinIgarashi commented Nov 2, 2023

I uploaded the following ESRI geodatabase as test, it failed to ingest

https://undpgeohub.blob.core.windows.net/userdata/a85516c81c0b78d3e89d3f00099b8b15/raw/GIS Parcels_20231102130408.gdb.zip

2023-11-02 13:08:25-ingest.py:sync_ingest:239:INFO:Starting to ingest https://undpgeohub.blob.core.windows.net/userdata/a85516c81c0b78d3e89d3f00099b8b15/raw/GIS Parcels_20231102130408.gdb.zip
2023-11-02 13:08:26-azblob.py:download_blob_sync:467:INFO:Downloading a85516c81c0b78d3e89d3f00099b8b15/raw/GIS Parcels_20231102130408.gdb.zip
2023-11-02 13:08:32-azblob.py:_progress_:448:INFO:downloaded - 10%
2023-11-02 13:08:32-azblob.py:_progress_:448:INFO:downloaded - 10%
2023-11-02 13:08:34-azblob.py:_progress_:448:INFO:downloaded - 20%
2023-11-02 13:08:34-azblob.py:_progress_:448:INFO:downloaded - 20%
2023-11-02 13:08:34-azblob.py:_progress_:448:INFO:downloaded - 20%
2023-11-02 13:08:36-azblob.py:_progress_:448:INFO:downloaded - 30%
2023-11-02 13:08:36-azblob.py:_progress_:448:INFO:downloaded - 30%
2023-11-02 13:08:36-azblob.py:_progress_:448:INFO:downloaded - 30%
2023-11-02 13:08:36-azblob.py:_progress_:448:INFO:downloaded - 30%
2023-11-02 13:08:37-azblob.py:_progress_:448:INFO:downloaded - 40%
2023-11-02 13:08:37-azblob.py:_progress_:448:INFO:downloaded - 40%
2023-11-02 13:08:37-azblob.py:_progress_:448:INFO:downloaded - 40%
2023-11-02 13:08:39-azblob.py:_progress_:448:INFO:downloaded - 50%
2023-11-02 13:08:39-azblob.py:_progress_:448:INFO:downloaded - 50%
2023-11-02 13:08:39-azblob.py:_progress_:448:INFO:downloaded - 50%
2023-11-02 13:08:39-azblob.py:_progress_:448:INFO:downloaded - 50%
2023-11-02 13:08:41-azblob.py:_progress_:448:INFO:downloaded - 60%
2023-11-02 13:08:41-azblob.py:_progress_:448:INFO:downloaded - 60%
2023-11-02 13:08:41-azblob.py:_progress_:448:INFO:downloaded - 60%
2023-11-02 13:08:43-azblob.py:_progress_:448:INFO:downloaded - 70%
2023-11-02 13:08:45-azblob.py:_progress_:448:INFO:downloaded - 80%
2023-11-02 13:08:47-azblob.py:_progress_:448:INFO:downloaded - 90%
2023-11-02 13:08:47-azblob.py:_progress_:448:INFO:downloaded - 90%
2023-11-02 13:08:49-azblob.py:_progress_:448:INFO:downloaded - 100%
2023-11-02 13:08:50-azblob.py:set_blob_metadata:227:INFO:Successfully updated metadata for a85516c81c0b78d3e89d3f00099b8b15/raw/GIS Parcels_20231102130408.gdb.zip
2023-11-02 13:08:50-utils.py:get_progress:117:INFO:/vsizip//tmp/tmpiafxybj5/GIS Parcels_20231102130408.gdb.zip contains:

2023-11-02 13:08:50-utils.py:get_progress:118:INFO:        1 vector layers
2023-11-02 13:08:50-utils.py:get_progress:119:INFO:        0 subdatasets
2023-11-02 13:08:50-utils.py:get_progress:120:INFO:        0 raster bands
2023-11-02 13:08:50-processing.py:process_geo_file:576:INFO:Opened /vsizip//tmp/tmpiafxybj5/GIS Parcels_20231102130408.gdb.zip with OpenFileGDB vector driver
2023-11-02 13:08:50-processing.py:process_geo_file:580:INFO:Found 1 vector layers
2023-11-02 13:08:51-processing.py:process_geo_file:586:INFO:Ingesting vector layer "All_Parcels".
2023-11-02 13:08:51-processing.py:should_reproject:49:ERROR:Failed to compare src and dst projections using osgeo.osr.GetAuthorityCode. Trying using osgeo.osr.IsSame
2023-11-02 13:08:52-processing.py:dataset2fgb:177:ERROR:dataset: https://undpgeohub.blob.core.windows.net/userdata/a85516c81c0b78d3e89d3f00099b8b15/raw/GIS Parcels_20231102130408.gdb.zip
layer: All_Parcels
gdal_error_message: Traceback (most recent call last):
  File "/usr/src/app/ingest/processing.py", line 149, in dataset2fgb
    fgb_ds = gdal.VectorTranslate(destNameOrDestDS=dst_path,
  File "/usr/lib/python3/dist-packages/osgeo/gdal.py", line 1472, in VectorTranslate
    return wrapper_GDALVectorTranslateDestName(destNameOrDestDS, srcDS, opts, callback, callback_data)
  File "/usr/lib/python3/dist-packages/osgeo/gdal.py", line 6085, in wrapper_GDALVectorTranslateDestName
    return _gdal.wrapper_GDALVectorTranslateDestName(*args)
RuntimeError: Terminating translation prematurely after failed
translation of layer All_Parcels (use -skipfailures to skip errors)

2023-11-02 13:08:52-processing.py:dataset2fgb:183:INFO:Uploading error message to a85516c81c0b78d3e89d3f00099b8b15/raw/GIS Parcels_20231102130408.gdb.zip.error
2023-11-02 13:08:53-azblob.py:upload_content_to_blob:206:INFO:Successfully wrote content to a85516c81c0b78d3e89d3f00099b8b15/raw/GIS Parcels_20231102130408.gdb.zip.error
2023-11-02 13:08:53-azblob.py:set_blob_metadata:227:INFO:Successfully updated metadata for a85516c81c0b78d3e89d3f00099b8b15/raw/GIS Parcels_20231102130408.gdb.zip
2023-11-02 13:08:53-processing.py:process_geo_file:639:INFO:/vsizip//tmp/tmpiafxybj5/GIS Parcels_20231102130408.gdb.zip does not contain raster GIS data
2023-11-02 13:08:53-ingest.py:sync_ingest:267:INFO:Finished ingesting https://undpgeohub.blob.core.windows.net/userdata/a85516c81c0b78d3e89d3f00099b8b15/raw/GIS Parcels_20231102130408.gdb.zip

image

┆Issue is synchronized with this Wrike task by Unito

@JinIgarashi JinIgarashi added the bug Something isn't working label Nov 2, 2023
@iferencik
Copy link
Contributor

@JinIgarashi, this is not an bug, it is by design

The dataset has one layer only and in this way the user knows the data is BAD. If the projection is unknown it is not possible to process the data and be sure the results are correct. We can discuss about this case and if we decide to process the data i can make small changes (use -skipfailures to skip errors). This however will produce an empty layer or an unusable layer.

@JinIgarashi
Copy link
Contributor Author

that is okay if this is not a bug, but the error message should be enough for users to understand what they need to do

@JinIgarashi JinIgarashi reopened this Nov 2, 2023
@JinIgarashi
Copy link
Contributor Author

@iferencik please also update supported format page of GeoHub for these spec you have decided

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants