RMS_Norm operand

[Samir-atra]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am using the latest TensorFlow Model Garden release and TensorFlow 2.
• I am reporting the issue to the correct repository. (Model Garden official or research directory)
• I checked to make sure that this issue has not been filed already.

1. The entire URL of the file you are using

https://github.com/tensorflow/models/blob/master/official/nlp/modeling/layers/transformer_encoder_block.py

2. Describe the bug

when I install the tensorflow models package using the command
pip3 install tf-models-official
and try to import it using
import tensorflow_models as tfm

I get a type error and as follows:

TypeError: unsupported operand type(s) for |: 'type' and '_GenericAlias'

3. Steps to reproduce

install the tensorflow model garden using pip and then try to import it to your project.

4. Expected behavior

I expected the model to be loaded and code execution finishes

5. Additional context

I tried to install the 2.16.0 version which is one version older then tried to install the current nightly version but still I get the same error.

6. System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 22.04.4 LTS
• TensorFlow installed from (source or binary): binary
• TensorFlow version (use command below): 2024-09-03 17:06:42.133393: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable TF_ENABLE_ONEDNN_OPTS=0.
  2024-09-03 17:06:42.159286: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
  WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
  E0000 00:00:1725376002.193099 32510 cuda_dnn.cc:8322] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
  E0000 00:00:1725376002.203587 32510 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
  2024-09-03 17:06:42.236849: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
  To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
  v1.12.1-115043-gb3d54ea5d80 2.18.0-dev20240828
• Python version: Python 3.9.19
• Bazel version (if compiling from source):
• GCC/Compiler version (if compiling from source):
• CUDA/cuDNN version: CUDA Version: 12.4
• GPU model and memory: NVIDIA GeForce RTX 3050 Laptop GPU / total memory: 4096 MB

[bharatjetti]

Hi @Samir-atra
Thanks for reporting the issue. I was able to reproduce this issue, it is due to new typing feature of python 3.10. I've opened a PR #11259 to make tensorflow_models compatible with python3.7+.

[Samir-atra]

Hello, @bharatjetti
great to hear that you have solved it.
Thanks and Kind regards
Samer