The latest numpy release 1.24.0 broke TFDV

[daikeshi]

TFDV allows 'numpy>=1.16,<2'. However, the latest numpy version 1.24.0 breaks TFDV. I encountered the following error in a TFDV-related component via TFX.

...
venv/lib/python3.8/site-packages/tfx/components/__init__.py:22: in <module>
    from tfx.components.example_validator.component import ExampleValidator
venv/lib/python3.8/site-packages/tfx/components/example_validator/component.py:20: in <module>
    from tfx.components.example_validator import executor
venv/lib/python3.8/site-packages/tfx/components/example_validator/executor.py:20: in <module>
    import tensorflow_data_validation as tfdv
venv/lib/python3.8/site-packages/tensorflow_data_validation/__init__.py:18: in <module>
    from tensorflow_data_validation.api.stats_api import GenerateStatistics
venv/lib/python3.8/site-packages/tensorflow_data_validation/api/stats_api.py:50: in <module>
    from tensorflow_data_validation.statistics import stats_impl
venv/lib/python3.8/site-packages/tensorflow_data_validation/statistics/stats_impl.py:28: in <module>
    from tensorflow_data_validation.statistics.generators import image_stats_generator
venv/lib/python3.8/site-packages/tensorflow_data_validation/statistics/generators/image_stats_generator.py:99: in <module>
    class TfImageDecoder(ImageDecoderInterface):
venv/lib/python3.8/site-packages/tensorflow_data_validation/statistics/generators/image_stats_generator.py:146: in TfImageDecoder
    def get_formats(self, values: List[np.object]) -> np.ndarray:
venv/lib/python3.8/site-packages/numpy/__init__.py:284: in __getattr__
    raise AttributeError("module {!r} has no attribute "
E   AttributeError: module 'numpy' has no attribute 'object'

Python version: 3.8.12
TFX version: 1.6.2
TFDV version: 1.6.0
numpy version: 1.24.0

[gaikwadrahul8]

Hi, @daikeshi

As per release notes if you're using TensorFlow Data Validation 1.6.0 with TFX ==1.6.0 then you should go with numpy version as per this requirement numpy>=1.16,<2, It seems like numpy version greater than and equal to 2.x does not support at the moment with TFX and shared libraries, even I tried with latest versions of TFX and tensorflow-data-validation and checked installation log for numpy version and it's showing the same requirement numpy>=1.16,<2, you can check Gist file

I would recommend you to please go with numpy version numpy>=1.16,<2 and everything will work as expected

Could you please confirm if this issue is resolved for you ? Please feel free to close the issue if it is resolved ?

If issue still persists after trying above workaround, In order to expedite the trouble-shooting process, please provide a code snippet to reproduce the issue reported here.

Thank you!

[daikeshi]

@gaikwadrahul8, the numpy version incompatible with TFDV is 1.24.0 which IS in the range of numpy>=1.16,<2. Maybe numpy>=1.16,<2 should be numpy>=1.16,<1.24 before this incompatibility issue gets resolved.

[gaikwadrahul8]

Hi, @daikeshi

I'm really sorry for the confusion and I tried to install both TFX==1.12.0 and tensorflow-data-validation==1.12.0 with latest versions and it seems like currently we only support numpy==1.22.4 at the moment, even I tried to install numpy separately with version 1.24.0 but TFX and tensorflow-data-validation by default taking numpy==1.22.4, for your reference I have added gist file here

I tried to install TFX and tensorflow-data-validation with version 1.6.x and by default it's taking numpy==1.21.6

We'll update you about numpy==1.24.0 support with TFX and tensorflow-data-validation here once I got update from our release team

Thank you!