"ValueError: attempted relative import beyond top-level package" -- pytho相对路径引用问题的解决

work python

阅读材料
- beyond top level package error in relative import

0. 目录和文件

0.1 tree ./nmt

当前文件目录结构：
nmt/
├── ···
├── ···
├── scripts
│   ├── ···
│   ├── ···
│   └── bleu.py
└── utils
├── ···
├── ···
└── evaluation_utils.py

0.2 evaluation_utils.py

···
# 相对路径引用
from ..scripts import bleu
from ..scripts import rouge
···

1. 问题

想要调试 evaluation_utils.py 中的一个函数的时候，我是这么做的：

1. cd nmt
2. ipytohn
3. from utils.evaluation_utils import xx

至此，抛出错误，原因是 evaluation_utils.py 中存在相对路径引用问题：

In [5]: from utils.evaluation_utils import _bleu as bleu                                                        
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-12-07e864825e38> in <module>
----> 1 from utils.evaluation_utils import _bleu as bleu
~/URun.ResearchPrototype/People/Xiaojie/MachineTranslation/nmt/nmt/utils/evaluation_utils.py in <module>
     22 import tensorflow as tf
     23 
---> 24 from ..scripts import bleu
     25 from ..scripts import rouge
     26 
ValueError: attempted relative import beyond top-level package

2. 为什么

问题的根本原因，就在于在 nmt 目录下时，执行 from utils.evaluation_utils import xx 时，当前python只有 utils 里的目录信息，他并不知道 utils 同级和外层的路径信息，utils/ 是当前执行环境的 top-level。详细的解答引用stack overflow中的一个回答：

Q：

I have a package shown below

package/
   __init__.py
   A/
      __init__.py
      foo.py
   test_A/
      __init__.py
      test.py

and I have a single line in test.py:

from ..A import foo

now, I am in the folder of package, and I run

python -m test_A.test

I got message

"ValueError: attempted relative import beyond top-level package"

but if I am in the parent folder of package, e.g., I run:

cd ..
python -m package.test_A.test

everything is fine.

A：

Why doesn't it work? It's because python doesn't record where a package was loaded from. So when you do python -m test_A.test, it basically just discards the knowledge that test_A.test is actually stored in package (i.e. package is not considered a package). Attempting from ..A import foo is trying to access information it doesn't have any more (i.e. sibling directories of a loaded location). It's conceptually similar to allowing from ..os import path in a file in math. This would be bad because you want the packages to be distinct. If they need to use something from another package, then they should refer to them globally with from os import path and let python work out where that is with $PATH and $PYTHONPATH.

When you use python -m package.test_A.test, then using from ..A import foo resolves just fine because it kept track of what's in package and you're just accessing a child directory of a loaded location.

Why doesn't python consider the current working directory to be a package? NO CLUE, but gosh it would be useful.

3. 解决办法

既然 from utils.evaluation_utils import xx 只能知道 utils 里的路径信息，那就返回上一层吧。

1. cd ..
2. from nmt.utils.evaluation_utils import xx

此时，python已经知道了 nmt 下的路径结构，所以 evaluation_utils.py 里可以获取到和 utils 同级的路径，问题解决。

# evaluation_utils.py
from ..scripts import bleu
from ..scripts import rouge

4. One More Thing

当你读到这里，已经理解了造成上述问题的原因之后，你应该还能想到另一种解决办法 —— 告诉python解释器额外的路径信息。具体做法就是将路径添加到 sys.path 中：

# from utils.evaluation_utils xx
# raise ERROR 
>>> import sys
>>> import os
# your project dir
>>> PROJ_DIR = 'xxxx'
# append to sys.path
>>> sys.path.append(os.path.join(PROJ_DIR, 'nmt'))
# no ERROR
from nmt.utils.evaluation_utils xx