Skip to content
This repository was archived by the owner on Jul 7, 2023. It is now read-only.
This repository was archived by the owner on Jul 7, 2023. It is now read-only.

Unable to instantiate problem instance when calling use_vocab_from_other_problem #1913

@colmantse

Description

@colmantse

Description

use_vocab_from_other_problem seems to return a function instead of a problem instance
I could directly change vocab_filename, but since this is the recommended approach, guess i better ask.
...

Environment information

OS: <ubuntu-16.14> $ pip freeze | grep tensor # graphsurgeon @ file:///opt/anaconda/conda-bld/graphsurgeon_1600268700975/work/tensorrt/graphsurgeon/graphsurgeon-0.4.1-py2.py3-none-any.whl mesh-tensorflow==0.0.5 tensor2tensor==1.15.7 tensorboard @ file:///opt/anaconda3/conda-bld/tensorboard_1602323996819/work/tmp_pip_dir tensorboard-plugin-profile==2.8.0 tensorflow==1.15.2 tensorflow-addons==0.13.0 tensorflow-datasets==1.0.1 tensorflow-estimator @ file:///opt/anaconda/conda-bld/tensorflow-estimator_1602324006015/work/tensorflow_estimator_pkg/tensorflow_estimator-1.15.2-py2.py3-none-any.whl tensorflow-gan==2.0.0 tensorflow-hub==0.7.0 tensorflow-metadata==0.13.0 tensorflow-probability==0.7.0 tensorflow-tensorboard==1.5.1 tensorrt @ file:///opt/anaconda/conda-bld/tensorrt_1600268909119/work/tensorrt/python/tensorrt-6.0.1.5-cp36-none-linux_x86_64.whl uff @ file:///opt/anaconda/conda-bld/uff_1600268848689/work/tensorrt/uff/uff-0.6.5-py2.py3-none-any.whl $ python -V # import _frozen_importlib # frozen import _imp # builtin import sys # builtin import '_warnings' # <class '_frozen_importlib.BuiltinImporter'> import '_thread' # <class '_frozen_importlib.BuiltinImporter'> import '_weakref' # <class '_frozen_importlib.BuiltinImporter'> import '_frozen_importlib_external' # <class '_frozen_importlib.FrozenImporter'> import '_io' # <class '_frozen_importlib.BuiltinImporter'> import 'marshal' # <class '_frozen_importlib.BuiltinImporter'> import 'posix' # <class '_frozen_importlib.BuiltinImporter'> import _thread # previously loaded ('_thread') import '_thread' # <class '_frozen_importlib.BuiltinImporter'> import _weakref # previously loaded ('_weakref') import '_weakref' # <class '_frozen_importlib.BuiltinImporter'> # installing zipimport hook import 'zipimport' # <class '_frozen_importlib.BuiltinImporter'> # installed zipimport hook # /home/colman/anaconda3/lib/python3.6/encodings/__pycache__/__init__.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/encodings/__init__.py # code object from '/home/colman/anaconda3/lib/python3.6/encodings/__pycache__/__init__.cpython-36.pyc' # /home/colman/anaconda3/lib/python3.6/__pycache__/codecs.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/codecs.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/codecs.cpython-36.pyc' import '_codecs' # <class '_frozen_importlib.BuiltinImporter'> import 'codecs' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf4ef128> # /home/colman/anaconda3/lib/python3.6/encodings/__pycache__/aliases.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/encodings/aliases.py # code object from '/home/colman/anaconda3/lib/python3.6/encodings/__pycache__/aliases.cpython-36.pyc' import 'encodings.aliases' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf4fdac8> import 'encodings' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf563c50> # /home/colman/anaconda3/lib/python3.6/encodings/__pycache__/utf_8.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/encodings/utf_8.py # code object from '/home/colman/anaconda3/lib/python3.6/encodings/__pycache__/utf_8.cpython-36.pyc' import 'encodings.utf_8' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf50a8d0> import '_signal' # <class '_frozen_importlib.BuiltinImporter'> # /home/colman/anaconda3/lib/python3.6/encodings/__pycache__/latin_1.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/encodings/latin_1.py # code object from '/home/colman/anaconda3/lib/python3.6/encodings/__pycache__/latin_1.cpython-36.pyc' import 'encodings.latin_1' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf50f400> # /home/colman/anaconda3/lib/python3.6/__pycache__/io.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/io.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/io.cpython-36.pyc' # /home/colman/anaconda3/lib/python3.6/__pycache__/abc.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/abc.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/abc.cpython-36.pyc' # /home/colman/anaconda3/lib/python3.6/__pycache__/_weakrefset.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/_weakrefset.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/_weakrefset.cpython-36.pyc' import '_weakrefset' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf515390> import 'abc' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf50f9e8> import 'io' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf50f630> # /home/colman/anaconda3/lib/python3.6/__pycache__/site.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/site.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/site.cpython-36.pyc' # /home/colman/anaconda3/lib/python3.6/__pycache__/os.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/os.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/os.cpython-36.pyc' import 'errno' # <class '_frozen_importlib.BuiltinImporter'> # /home/colman/anaconda3/lib/python3.6/__pycache__/stat.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/stat.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/stat.cpython-36.pyc' import '_stat' # <class '_frozen_importlib.BuiltinImporter'> import 'stat' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf4ba668> # /home/colman/anaconda3/lib/python3.6/__pycache__/posixpath.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/posixpath.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/posixpath.cpython-36.pyc' # /home/colman/anaconda3/lib/python3.6/__pycache__/genericpath.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/genericpath.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/genericpath.cpython-36.pyc' import 'genericpath' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf4c5080> import 'posixpath' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf4bad30> # /home/colman/anaconda3/lib/python3.6/__pycache__/_collections_abc.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/_collections_abc.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/_collections_abc.cpython-36.pyc' import '_collections_abc' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf4c56a0> import 'os' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf4a9f60> # /home/colman/anaconda3/lib/python3.6/__pycache__/_sitebuiltins.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/_sitebuiltins.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/_sitebuiltins.cpython-36.pyc' import '_sitebuiltins' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf4ac390> # /home/colman/anaconda3/lib/python3.6/__pycache__/sysconfig.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/sysconfig.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/sysconfig.cpython-36.pyc' import 'sysconfig' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf479b00> # /home/colman/anaconda3/lib/python3.6/__pycache__/_sysconfigdata_m_linux_x86_64-linux-gnu.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/_sysconfigdata_m_linux_x86_64-linux-gnu.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/_sysconfigdata_m_linux_x86_64-linux-gnu.cpython-36.pyc' import '_sysconfigdata_m_linux_x86_64-linux-gnu' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf48f3c8> # /home/colman/anaconda3/lib/python3.6/__pycache__/_bootlocale.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/_bootlocale.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/_bootlocale.cpython-36.pyc' import '_locale' # <class '_frozen_importlib.BuiltinImporter'> import '_bootlocale' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf49b3c8> # /home/colman/anaconda3/lib/python3.6/__pycache__/types.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/types.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/types.cpython-36.pyc' # /home/colman/anaconda3/lib/python3.6/__pycache__/functools.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/functools.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/functools.cpython-36.pyc' import '_functools' # <class '_frozen_importlib.BuiltinImporter'> # /home/colman/anaconda3/lib/python3.6/collections/__pycache__/__init__.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/collections/__init__.py # code object from '/home/colman/anaconda3/lib/python3.6/collections/__pycache__/__init__.cpython-36.pyc' # /home/colman/anaconda3/lib/python3.6/__pycache__/operator.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/operator.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/operator.cpython-36.pyc' import '_operator' # <class '_frozen_importlib.BuiltinImporter'> import 'operator' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68ce208208> # /home/colman/anaconda3/lib/python3.6/__pycache__/keyword.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/keyword.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/keyword.cpython-36.pyc' import 'keyword' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68ce212470> # /home/colman/anaconda3/lib/python3.6/__pycache__/heapq.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/heapq.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/heapq.cpython-36.pyc' # extension module '_heapq' loaded from '/home/colman/anaconda3/lib/python3.6/lib-dynload/_heapq.cpython-36m-x86_64-linux-gnu.so' # extension module '_heapq' executed from '/home/colman/anaconda3/lib/python3.6/lib-dynload/_heapq.cpython-36m-x86_64-linux-gnu.so' import '_heapq' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7f68ce19a1d0> import 'heapq' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68ce212c88> import 'itertools' # <class '_frozen_importlib.BuiltinImporter'> # /home/colman/anaconda3/lib/python3.6/__pycache__/reprlib.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/reprlib.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/reprlib.cpython-36.pyc' import 'reprlib' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68ce20c860> import '_collections' # <class '_frozen_importlib.BuiltinImporter'> import 'collections' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68ce1e88d0> # /home/colman/anaconda3/lib/python3.6/__pycache__/weakref.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/weakref.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/weakref.cpython-36.pyc' import 'weakref' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68ce1e8cc0> import 'functools' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68ce1dd160> # /home/colman/anaconda3/lib/python3.6/collections/__pycache__/abc.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/collections/abc.py # code object from '/home/colman/anaconda3/lib/python3.6/collections/__pycache__/abc.cpython-36.pyc' import 'collections.abc' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68ce1ddeb8> import 'types' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf4a14a8> # /home/colman/anaconda3/lib/python3.6/importlib/__pycache__/__init__.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/importlib/__init__.py # code object from '/home/colman/anaconda3/lib/python3.6/importlib/__pycache__/__init__.cpython-36.pyc' # /home/colman/anaconda3/lib/python3.6/__pycache__/warnings.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/warnings.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/warnings.cpython-36.pyc' import 'warnings' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68ce1e4c88> import 'importlib' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf4a15c0> # /home/colman/anaconda3/lib/python3.6/importlib/__pycache__/util.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/importlib/util.py # code object from '/home/colman/anaconda3/lib/python3.6/importlib/__pycache__/util.cpython-36.pyc' # /home/colman/anaconda3/lib/python3.6/importlib/__pycache__/abc.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/importlib/abc.py # code object from '/home/colman/anaconda3/lib/python3.6/importlib/__pycache__/abc.cpython-36.pyc' # /home/colman/anaconda3/lib/python3.6/importlib/__pycache__/machinery.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/importlib/machinery.py # code object from '/home/colman/anaconda3/lib/python3.6/importlib/__pycache__/machinery.cpython-36.pyc' import 'importlib.machinery' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68ce1c95f8> import 'importlib.abc' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68ce1bdfd0> # /home/colman/anaconda3/lib/python3.6/__pycache__/contextlib.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/contextlib.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/contextlib.cpython-36.pyc' import 'contextlib' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68ce1d7128> import 'importlib.util' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68ce1bd978> # possible namespace for /home/colman/anaconda3/lib/python3.6/site-packages/google # possible namespace for /home/colman/anaconda3/lib/python3.6/site-packages/google # destroy google # possible namespace for /home/colman/anaconda3/lib/python3.6/site-packages/google # destroy google # possible namespace for /home/colman/anaconda3/lib/python3.6/site-packages/google/logging # possible namespace for /home/colman/anaconda3/lib/python3.6/site-packages/mpl_toolkits # possible namespace for /home/colman/anaconda3/lib/python3.6/site-packages/google # destroy google # possible namespace for /home/colman/anaconda3/lib/python3.6/site-packages/google # destroy google # possible namespace for /home/colman/anaconda3/lib/python3.6/site-packages/google # destroy google # destroy sphinxcontrib # destroy sphinxcontrib # destroy sphinxcontrib # destroy sphinxcontrib # destroy sphinxcontrib # destroy sphinxcontrib # destroy zope # /home/colman/anaconda3/lib/python3.6/encodings/__pycache__/cp437.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/encodings/cp437.py # code object from '/home/colman/anaconda3/lib/python3.6/encodings/__pycache__/cp437.cpython-36.pyc' import 'encodings.cp437' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68ce176198> # zipimport: found 15 names in '/home/colman/anaconda3/lib/python3.6/site-packages/tflms-2.0.2-py3.6.egg' import 'site' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68cf51acc0> Python 3.6.12 |Anaconda custom (64-bit)| (default, Sep 8 2020, 23:10:56) [GCC 7.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. # extension module 'readline' loaded from '/home/colman/anaconda3/lib/python3.6/lib-dynload/readline.cpython-36m-x86_64-linux-gnu.so' # extension module 'readline' executed from '/home/colman/anaconda3/lib/python3.6/lib-dynload/readline.cpython-36m-x86_64-linux-gnu.so' import 'readline' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7f68ce176400> import 'atexit' # <class '_frozen_importlib.BuiltinImporter'> # /home/colman/anaconda3/lib/python3.6/__pycache__/rlcompleter.cpython-36.pyc matches /home/colman/anaconda3/lib/python3.6/rlcompleter.py # code object from '/home/colman/anaconda3/lib/python3.6/__pycache__/rlcompleter.cpython-36.pyc' import 'rlcompleter' # <_frozen_importlib_external.SourceFileLoader object at 0x7f68ce176518> 

For bugs: reproduction and error logs

# Steps to reproduce: Defined a custom problem which has no problem running. Then try to use vocab from said custom problem on a new custom problem as follow. 

def use_vocab_from_other_problem(self):
return WmtReorder32k()

# Error logs: ...Traceback (most recent call last): File "/home/colman/anaconda3/bin/t2t-trainer", line 33, in <module> tf.app.run(main) File "/home/colman/anaconda3/lib/python3.6/site-packages/tensorflow_core/python/platform/app.py", line 40, in run _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef) File "/home/colman/anaconda3/lib/python3.6/site-packages/absl/app.py", line 300, in run _run_main(main, args) File "/home/colman/anaconda3/lib/python3.6/site-packages/absl/app.py", line 251, in _run_main sys.exit(main(argv)) File "/home/colman/anaconda3/bin/t2t-trainer", line 28, in main t2t_trainer.main(argv) File "/home/colman/anaconda3/lib/python3.6/site-packages/tensor2tensor/bin/t2t_trainer.py", line 409, in main generate_data() File "/home/colman/anaconda3/lib/python3.6/site-packages/tensor2tensor/bin/t2t_trainer.py", line 302, in generate_data registry.problem(problem_name).generate_data(data_dir, tmp_dir) File "/home/colman/anaconda3/lib/python3.6/site-packages/tensor2tensor/data_generators/text_problems.py", line 363, in generate_data self.generate_encoded_samples(data_dir, tmp_dir, split), paths) File "/media/colman/testmount/projects/trans/usr/t2t_usr/word_order.py", line 2602, in generate_encoded_samples encoder = self.get_or_create_vocab(data_dir, tmp_dir) File "/home/colman/anaconda3/lib/python3.6/site-packages/tensor2tensor/data_generators/text_problems.py", line 249, in get_or_create_vocab return other_problem.get_or_create_vocab(data_dir, tmp_dir, force_get) AttributeError: 'function' object has no attribute 'get_or_create_vocab' 

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions