Module not found

[barseghyanartur]

Hello Timothy.

I'm getting the following error when running portray as_html command:

portray as_html
/home/me/.virtualenvs/graphene-elastic/lib/python3.6/site-packages/portray/config.py:101: UserWarning: Error (malformed node or string: <_ast.Call object at 0x7f32e20d9c18>) occurred trying to parse setup.py file: /home/me/repos/graphene-elastic/setup.py
  warnings.warn(f"Error ({error}) occurred trying to parse setup.py file: {location}")
Done Copying source documentation to temporary compilation directory
Module not found: graphene-elastic

Virtualenv I'm in has the graphene-elastic installed (python setup.py develop).
Installing it using pip install graphene-elastic or python setup.py install does not help.

Is there specific directory structure expected? I run it from the project root here. Anything else I'm missing?

Portray version: portray==1.3.0.

Any tips and help are appreciated. Thanks in advance!

[timothycrosley]

Hi @barseghyanartur,

Sorry you ran into this issue! How do you import graphene-elastic for use within Python code?

is it import graphene_elastic?

[timothycrosley]

From the error, portray incorrectly guesses your module is named "graphene-elastic" which can't be the case, as valid Python module names cannot contain dashes.

This should be fixed automatically for you in the just release 1.3.1 release: https://timothycrosley.github.io/portray/CHANGELOG/#131-september-22-2019.

To work around this in 1.3.0, simply do portay as_html -m graphene_elastic

Hope this helps!

Thanks!

~Timothy

[barseghyanartur]

@timothycrosley:

Thanks for your quick reaction and efforts! Unfortunately that didn't fix my issue. I still do get the same error after upgrading to 1.3.1. Work around using -m graphene_elastic does not work for me either on both 1.3.0 and 1.3.1 versions.

When I build docs using sphinx, I do add extra paths (in the docs/conf.py). Since that doesn't work for all the cases (when conf.py isn't taken into consideration), so I added .pth file with path alterations to the project root (yet, locally), but that didn't fix the issue either.

Please, let me know if you need more input or perhaps help from my side.

Thanks!

[timothycrosley]

@barseghyanartur,

The other common issue is that portray is installed in a different env (such as a global installation) from the project itself. Can you do a which portray and ensure it's installed in the projects venv?

Thanks!

~Timothy

[barseghyanartur]

@timothycrosley:

I got it working in my fork. The only change I made was this. I used to get ValueError: malformed node or string: <_ast.Call object at ...> on ast.literal_eval() error otherwise.

Inside of my setup.py, I have the following code for packages:

    packages=find_packages(
        where="./src",
        exclude=["tests"]
    ),

That's where it fails. Please, tell me if you want a PR.

What does not yet work fine for me yet: I would expect portray to automatically do everything nicely, as it is intended to do (as far as I understand).

I'm a big fan of RestructuredText and I prefer it to MarkDown. At the moment that (format conversion) does not happen.

Why not automatically convert all .rst files to .md (using, let's say pypandoc) before building docs?

[timothycrosley]

Glad you were able to get to the bottom of of it! I'm a little surprised that would make a difference, as that block is already wrapped in a try except block: https://github.com/timothycrosley/portray/blob/master/portray/config.py#L105. However, what it would do is suppress the warning - however everything should work fine with the warning in place (which can also be silenced by explicitly setting the module list).

I am very open to the idea of preconverting .rst to .md! I would happily accept a pull-request that added this functionality, or otherwise aim to include something to enable it for the 2.0.0 release.

Thanks!

~Timothy

[barseghyanartur]

@timothycrosley:

I think the reason it fails is that it might go wrong for one of the items in the list, but not for all. Say, there are multiple packages (and sub-packages) identified by find_packages and something fails for one of them. Then the main package wouldn't be identified either. While the top try/except block would stand for exceptions arising from reading the setup file and parsing the its' lines, the second (added) try/except block would be more error agnostic/tolerant.

In any case, I think you have a good point that warning shall be shown in any case, thus, I'll add it in the second try/except block as well. I can submit a PR for that part only. Tell me if I should.

As for preconverting the .rst to .md, I'll try to find time for it this week. That would be the second PR.

Thanks so far!

[timothycrosley]

@barseghyanartur Merge requests are always welcomed :)! Really appreciate the willingness to do so!

Thanks!

~Timothy

[amenezes]

@timothycrosley first thanks for this project is really amazing.

I'm using portray 1.3.1 locally for some tests, but after I put a new file on my project portray has stopped working. Either through the command portray as_html -m rabbit or portray in_browser -m rabbit All attempts to display the message: Module not found: rabbit.observer.

If I remove this file portray work's smoothly, but it's not the case. This bug has any relation to this issue? Or it's better open a new one? Is there anything that I could do to to fix this or get some more information?

This happen here.

[dhimmel]

I'm interested in using portray to generate docs for our manubot python package. However, as shown in the CI builds for manubot/manubot#153, portray is returning a nonzero exit code = 1 (although at least some HTML output is generated). The full output is:

$ portray as_html --overwrite --output_dir=docs
/home/travis/virtualenv/python3.7.1/lib/python3.7/site-packages/portray/config.py:137: UserWarning: No /home/travis/build/manubot/manubot/pyproject.toml config file found
  warnings.warn(f"No {location} config file found")
Done Copying source documentation to temporary compilation directory
Module not found: manubot.process.tests.test_bibliography

I am guessing the Module not found error is what's causing the problem? But I can't figure out what is wrong with manubot.process.tests.test_bibliography to cause this problem.

[amenezes]

@timothycrosley first thanks for this project is really amazing.

I'm using portray 1.3.1 locally for some tests, but after I put a new file on my project portray has stopped working. Either through the command portray as_html -m rabbit or portray in_browser -m rabbit All attempts to display the message: Module not found: rabbit.observer.

If I remove this file portray work's smoothly, but it's not the case. This bug has any relation to this issue? Or it's better open a new one? Is there anything that I could do to to fix this or get some more information?

This happen here.

@timothycrosley, Investigating this problem a little more seemed to me that problem occurs on the pdocs, extract.py module. In my case I was able to correct the error by setting PYTHONPATH environment variable to the current directory.

For example: export PYTHONPATH=.

I'm looking for a better understanding of the issue and, if relevant, submit a pull request.

[whitelynx]

I'm running into this too. From what I'm seeing, it seems like this is due to using ast.literal_eval on the find_packages call. Trying it in ipython, I get the same error when trying to run ast.literal_eval(kw.value), where kw.value is an _ast.Call object.

The docstring of ast.literal_eval says:

Safely evaluate an expression node or a string containing a Python
expression.  The string or node provided may only consist of the following
Python literal structures: strings, bytes, numbers, tuples, lists, dicts,
sets, booleans, and None.

which means it's not valid to pass it a function call like that. Since a function call expression also depends on things in the current scope (specifically find_packages in this case) we'd need some way to get the import included so we can actually evaluate the function call.

I'm curious if it would make more sense to instead simply mock the setup function from setuptools, and just execute the setup.py instead of parsing it. That would ensure that the other imports would work as expected, and we wouldn't have to try and reconstruct the scope.

[leblancfg]

Same problem here. Unsure if relevant, but it was working a minute ago on Linux w/ Python 3.8; I'm now on macOS w/ Python 3.6.

[romanows]

Just ran into this same error. Someone added a new module to an existing project at foo/bar.py, nothing special, and portray raised the Module not found: foo.bar error. A version of amenezes solution worked for me. Setting PYTHONPATH=. did NOT work but setting PYTHONPATH=foo did make it work.

Given the other debugging comments people have made in this thread make me worry that the PYTHONPATH setting isn't a real fix. In fact, I would guess that this would break in my project if I ever had two modules with the same name in different packages (e.g. foo.bar and qux.bar).

Edit: I'm going to try changing all of my CI scripts to something like:

PYTHONPATH=$(find . -maxdepth 1 -type d -printf '%p:') portray as_html ...

Which works if your version of find supports -printf. This captures all of the directories in the current working dir including .git directories and the like but seems to work for me.