Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature request] TF2.0 Support #239

Closed
BrikerMan opened this issue Sep 13, 2019 · 8 comments
Closed

[Feature request] TF2.0 Support #239

BrikerMan opened this issue Sep 13, 2019 · 8 comments
Assignees
@BrikerMan
Labels
enhancement New feature or request pinned

Comments

@BrikerMan
Copy link
Owner

TF 2.0 Support eager mode by default, easy to debug. Also, UnifiedLSTM is really cool.
@BrikerMan BrikerMan added the enhancement New feature or request label Sep 13, 2019
@BrikerMan BrikerMan self-assigned this Sep 13, 2019
@stale stale bot added the wontfix This will not be worked on label Oct 3, 2019
@BrikerMan BrikerMan added pinned and removed wontfix This will not be worked on labels Oct 3, 2019
Repository owner deleted a comment from stale bot Oct 3, 2019
@lsgrep
Copy link
Contributor

lsgrep commented Oct 5, 2019

TF 2.0 support is a big upgrade, and probably not backward compatible. I think we should create a new branch and release a major version for this feature.

@BrikerMan
Copy link
Owner Author

I have created the TF2.0 branch. After testing with the test cases, here are the issues I think we need to fix first.

  • Fix Custom Layers
    • AttentionWeightedAverageLayer
    • AttentionWeightedAverageLayer
    • CRF Layers
    • DPCNN_Model
  • utils.convert_to_saved_model function
  • tasks.base_model.BaseModel.build_tpu_model function
  • tasks.base_model.BaseModel.build_multi_gpu_model function

@lsgrep @alexwwang

@lsgrep
Copy link
Contributor

lsgrep commented Oct 8, 2019 

TensorFlow 2.0 Upgrade Script
-----------------------------
Converted 33 files
Detected 8 issues that require attention
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
File: kashgari/tasks/base_model.py
--------------------------------------------------------------------------------
kashgari/tasks/base_model.py:178:40: ERROR: tf.contrib.distribute.TPUStrategy requires manual check. (Manual edit required) tf.contrib.distribute.TPUStrategy has been migrated to tf.distribute.experimental.TPUStrategy. Note the slight changes in constructor. If you're using the strategy with a custom training loop, note the following changes in methods: make_dataset_iterator->experimental_distribute_dataset, experimental_make_numpy_iterator->experimental_make_numpy_dataset, extended.call_for_each_replica->experimental_run_v2, reduce requires an axis argument, unwrap->experimental_local_results experimental_initialize and experimenta_finalize no longer needed 
kashgari/tasks/base_model.py:178:40: WARNING: Using member tf.contrib.distribute.TPUStrategy in deprecated module tf.contrib.distribute. (Manual edit required) tf.contrib.distribute.* have been migrated totf.distribute.*. Please check out the new module for updates APIs.
kashgari/tasks/base_model.py:178:40: ERROR: Using member tf.contrib.distribute.TPUStrategy in deprecated module tf.contrib. tf.contrib.distribute.TPUStrategy cannot be converted automatically. tf.contrib will not be distributed with TensorFlow 2.0, please consider an alternative in non-contrib TensorFlow, a community-maintained repository such as tensorflow/addons, or fork the required code.
kashgari/tasks/base_model.py:207:32: ERROR: Using member tf.contrib.tpu.keras_to_tpu_model in deprecated module tf.contrib. tf.contrib.tpu.keras_to_tpu_model cannot be converted automatically. tf.contrib will not be distributed with TensorFlow 2.0, please consider an alternative in non-contrib TensorFlow, a community-maintained repository such as tensorflow/addons, or fork the required code.
kashgari/tasks/base_model.py:475:4: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
--------------------------------------------------------------------------------
File: kashgari/layers/crf.py
--------------------------------------------------------------------------------
kashgari/layers/crf.py:91:30: ERROR: Using member tf.contrib.crf.crf_decode in deprecated module tf.contrib. tf.contrib.crf.crf_decode cannot be converted automatically. tf.contrib will not be distributed with TensorFlow 2.0, please consider an alternative in non-contrib TensorFlow, a community-maintained repository such as tensorflow/addons, or fork the required code.
kashgari/layers/crf.py:98:43: ERROR: Using member tf.contrib.crf.crf_log_likelihood in deprecated module tf.contrib. tf.contrib.crf.crf_log_likelihood cannot be converted automatically. tf.contrib will not be distributed with TensorFlow 2.0, please consider an alternative in non-contrib TensorFlow, a community-maintained repository such as tensorflow/addons, or fork the required code.
kashgari/layers/crf.py:118:34: ERROR: Using member tf.contrib.crf.crf_decode in deprecated module tf.contrib. tf.contrib.crf.crf_decode cannot be converted automatically. tf.contrib will not be distributed with TensorFlow 2.0, please consider an alternative in non-contrib TensorFlow, a community-maintained repository such as tensorflow/addons, or fork the required code.
================================================================================
Detailed log follows:

================================================================================
================================================================================
Input tree: 'kashgari'
================================================================================
--------------------------------------------------------------------------------
Processing file 'kashgari/version.py'
 outputting to 'kashgari2/version.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/__init__.py'
 outputting to 'kashgari2/__init__.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/utils.py'
 outputting to 'kashgari2/utils.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/macros.py'
 outputting to 'kashgari2/macros.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/callbacks.py'
 outputting to 'kashgari2/callbacks.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/corpus.py'
 outputting to 'kashgari2/corpus.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/tasks/__init__.py'
 outputting to 'kashgari2/tasks/__init__.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/tasks/base_model.py'
 outputting to 'kashgari2/tasks/base_model.py'
--------------------------------------------------------------------------------

178:40: ERROR: tf.contrib.distribute.TPUStrategy requires manual check. (Manual edit required) tf.contrib.distribute.TPUStrategy has been migrated to tf.distribute.experimental.TPUStrategy. Note the slight changes in constructor. If you're using the strategy with a custom training loop, note the following changes in methods: make_dataset_iterator->experimental_distribute_dataset, experimental_make_numpy_iterator->experimental_make_numpy_dataset, extended.call_for_each_replica->experimental_run_v2, reduce requires an axis argument, unwrap->experimental_local_results experimental_initialize and experimenta_finalize no longer needed 
178:40: WARNING: Using member tf.contrib.distribute.TPUStrategy in deprecated module tf.contrib.distribute. (Manual edit required) tf.contrib.distribute.* have been migrated totf.distribute.*. Please check out the new module for updates APIs.
178:40: ERROR: Using member tf.contrib.distribute.TPUStrategy in deprecated module tf.contrib. tf.contrib.distribute.TPUStrategy cannot be converted automatically. tf.contrib will not be distributed with TensorFlow 2.0, please consider an alternative in non-contrib TensorFlow, a community-maintained repository such as tensorflow/addons, or fork the required code.
207:32: ERROR: Using member tf.contrib.tpu.keras_to_tpu_model in deprecated module tf.contrib. tf.contrib.tpu.keras_to_tpu_model cannot be converted automatically. tf.contrib will not be distributed with TensorFlow 2.0, please consider an alternative in non-contrib TensorFlow, a community-maintained repository such as tensorflow/addons, or fork the required code.
208:45: INFO: Renamed 'tf.train.AdamOptimizer' to 'tf.compat.v1.train.AdamOptimizer'
475:4: WARNING: *.save requires manual check. (This warning is only applicable if the code saves a tf.Keras model) Keras model.save now saves to the Tensorflow SavedModel format by default, instead of HDF5. To continue saving to HDF5, add the argument save_format='h5' to the save() function.
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/tasks/classification/models.py'
 outputting to 'kashgari2/tasks/classification/models.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/tasks/classification/dpcnn_model.py'
 outputting to 'kashgari2/tasks/classification/dpcnn_model.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/tasks/classification/__init__.py'
 outputting to 'kashgari2/tasks/classification/__init__.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/tasks/classification/base_model.py'
 outputting to 'kashgari2/tasks/classification/base_model.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/tasks/labeling/experimental.py'
 outputting to 'kashgari2/tasks/labeling/experimental.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/tasks/labeling/models.py'
 outputting to 'kashgari2/tasks/labeling/models.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/tasks/labeling/__init__.py'
 outputting to 'kashgari2/tasks/labeling/__init__.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/tasks/labeling/base_model.py'
 outputting to 'kashgari2/tasks/labeling/base_model.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/embeddings/gpt_2_embedding.py'
 outputting to 'kashgari2/embeddings/gpt_2_embedding.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/embeddings/stacked_embedding.py'
 outputting to 'kashgari2/embeddings/stacked_embedding.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/embeddings/numeric_feature_embedding.py'
 outputting to 'kashgari2/embeddings/numeric_feature_embedding.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/embeddings/word_embedding.py'
 outputting to 'kashgari2/embeddings/word_embedding.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/embeddings/bare_embedding.py'
 outputting to 'kashgari2/embeddings/bare_embedding.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/embeddings/__init__.py'
 outputting to 'kashgari2/embeddings/__init__.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/embeddings/bert_embedding.py'
 outputting to 'kashgari2/embeddings/bert_embedding.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/embeddings/base_embedding.py'
 outputting to 'kashgari2/embeddings/base_embedding.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/layers/crf.py'
 outputting to 'kashgari2/layers/crf.py'
--------------------------------------------------------------------------------

85:24: INFO: Added keywords to args of function 'tf.convert_to_tensor'
88:24: INFO: Added keywords to args of function 'tf.convert_to_tensor'
89:20: INFO: Added keywords to args of function 'tf.shape'
91:30: ERROR: Using member tf.contrib.crf.crf_decode in deprecated module tf.contrib. tf.contrib.crf.crf_decode cannot be converted automatically. tf.contrib will not be distributed with TensorFlow 2.0, please consider an alternative in non-contrib TensorFlow, a community-maintained repository such as tensorflow/addons, or fork the required code.
97:17: INFO: Added keywords to args of function 'tf.convert_to_tensor'
98:43: ERROR: Using member tf.contrib.crf.crf_log_likelihood in deprecated module tf.contrib. tf.contrib.crf.crf_log_likelihood cannot be converted automatically. tf.contrib will not be distributed with TensorFlow 2.0, please consider an alternative in non-contrib TensorFlow, a community-maintained repository such as tensorflow/addons, or fork the required code.
103:15: INFO: Added keywords to args of function 'tf.reduce_mean'
116:20: INFO: Added keywords to args of function 'tf.shape'
118:34: ERROR: Using member tf.contrib.crf.crf_decode in deprecated module tf.contrib. tf.contrib.crf.crf_decode cannot be converted automatically. tf.contrib will not be distributed with TensorFlow 2.0, please consider an alternative in non-contrib TensorFlow, a community-maintained repository such as tensorflow/addons, or fork the required code.
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/layers/non_masking_layer.py'
 outputting to 'kashgari2/layers/non_masking_layer.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/layers/__init__.py'
 outputting to 'kashgari2/layers/__init__.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/layers/att_wgt_avg_layer.py'
 outputting to 'kashgari2/layers/att_wgt_avg_layer.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/layers/kmax_pool_layer.py'
 outputting to 'kashgari2/layers/kmax_pool_layer.py'
--------------------------------------------------------------------------------

65:28: INFO: Added keywords to args of function 'tf.transpose'
72:15: INFO: Added keywords to args of function 'tf.transpose'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/processors/labeling_processor.py'
 outputting to 'kashgari2/processors/labeling_processor.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/processors/__init__.py'
 outputting to 'kashgari2/processors/__init__.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/processors/base_processor.py'
 outputting to 'kashgari2/processors/base_processor.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Processing file 'kashgari/processors/classification_processor.py'
 outputting to 'kashgari2/processors/classification_processor.py'
--------------------------------------------------------------------------------


--------------------------------------------------------------------------------

@lsgrep
Copy link
Contributor

lsgrep commented Oct 8, 2019

As the migrate script shows, base models and crf layers require fixes at least.

@BrikerMan
Copy link
Owner Author

@lsgrep we should try to support both 1.14 and 2.0.

@BrikerMan
Copy link
Owner Author

BrikerMan commented Oct 9, 2019 •
edited
Loading

I think the kashagri-tf naming is a bad decision. Maybe we will fix this issue with the new version. Yea, rename again... Hear me out

The final result looks like this.

Backend pypi version desc
TensorFlow 2.0 pip install 'kashgari>=2.0.0' edge version
TensorFlow 1.14 pip install 'kashgari>=1.0.0,<2.0.0' recommended stable version
Keras pip install 'kashgari>0.2.5,<1.0.0' discarded version

Users can install TF 1.14 version using pip install kashgari-tf, but recommend to use pip install 'kashgari>=1.0.0,<2.0.0'.

Release the current tf 1.14 version kashgari-tf as kashgari==1.0.0. Then release the new tf 2.0 version as kashgari==2.0.0. Add update warnings before like this

╭────────────────────────────────────────────────────────────────────────────╮
   │                                                                            │
   │                       New tf.keras version available!                      │
   │    Changelog: https://github.com/BrikerMan/Kashgari/releases/tag/v0.5.0    │
   │     Run `pip uninstall kashgari && pip install kashgari-tf` to install     │
   │                 Documents: https://kashgari.readthedocs.io                 │
   │                                                                            │ 
   ╰────────────────────────────────────────────────────────────────────────────╯

@lsgrep @alexwwang

@BrikerMan
Copy link
Owner Author

Updated name changing propose at #265

@BrikerMan
Copy link
Owner Author

Migrate discussion to the #340

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@BrikerMan BrikerMan

Labels
enhancement New feature or request pinned
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lsgrep @BrikerMan