Generate docs of project fails with virtual environment dependencies
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 146
- PR merge metrics
- No merged PRs in 30d
Description
Adapted from the pdocs programmed recursive documentation generation example, I have a script that runs the following methods:
tl_util.py
import pdoc
# documentation
def pdoc_module_path(m:pdoc.Module, ext:str='.html', dir_path:str=DOCS_PATH):
return os.path.join(
dir_path,
*regex.sub(r'\.html$', ext, m.url()).split('/'))
# end pdoc_module_path
def document_modules(mod:pdoc.Module) -> Generator[Tuple[pdoc.Module,str],None,None]:
"""Generate documentation for pdoc-wrapped module and submodules.
Args:
mod = pdoc.Module instance
Yields tuple:
module
cleaned module html
"""
yield (
mod,
mod.html().replace('\u2011','-').replace(u'\xa0', u' ')
)
for submod in mod.submodules():
yield from document_modules(submod)
# end document_modules
main.py
from typing import *
import pdoc
import os
import logging
from logging import Logger
import tl_util
log:Logger = logging.getLogger(__name__)
# omitted code
def document(dir_path:str=DOCS_PATH):
"""Recursively generate documentation using pdoc.
Adapted from
[pdoc documentation](https://pdoc3.github.io/pdoc/doc/pdoc/#programmatic-usage).
Args:
dir_path = documentation output directory; default=`algo_trader.DOCS_PATH`
"""
ctx = pdoc.Context()
modules:List[pdoc.Module] = [
pdoc.Module(mod)
for mod in [
'.' # this script resides within the package that I want to create docs for
]
]
pdoc.link_inheritance(ctx)
for mod in modules:
for submod, html in tl_util.document_modules(mod):
# write to output location
ext:str = '.html'
filepath = tl_util.pdoc_module_path(submod, ext, dir_path)
dirpath = os.path.dirname(filepath)
if not os.access(dirpath, os.R_OK):
os.makedirs(dirpath)
with open(filepath,'w') as f:
if ext == '.html':
try:
f.write(html)
except:
log.error(traceback.format_exc())
elif ext == '.md':
f.write(mod.text())
# close f
log.info('generated doc for {} at {}'.format(
submod.name,
filepath))
# end for module_name, html
# end for mod in modules
# end document
if __name__ == '__main__':
# omitted logic...
document()
My project filesystem is like this:
my_package/
env/
Lib/
site-packages/
<installed dependencies, including pdoc3>
main.py
tl_util.py
Below is the error that I currently get:
(env) PS C:\<path>\my_package python .\main.py --document
=== Program Name ===
set logger <RootLogger root (DEBUG)> to level 10
C:\<path>\my_package\env\lib\site-packages\pdoc\__init__.py:643: UserWarning: Module <Module 'my_package
.env.Lib.site-packages.dateutil'> doesn't contain identifier `easter` exported in `__all__`
warn("Module {!r} doesn't contain identifier `{}` "
Traceback (most recent call last):
File “.\main.py", line 608, in <module>
main()
File “.\main.py", line 469, in main
document()
File “.\main.py", line 399, in document
modules:List[pdoc.Module] = [
File “.\main.py", line 400, in <listcomp>
pdoc.Module(mod)
File "C:\Users\Owen\Documents\my_package\env\lib\site-packages\pdoc\__init__.py", line 708, in __init__
m = Module(import_module(fullname),
File "C:\Users\Owen\Documents\my_package\env\lib\site-packages\pdoc\__init__.py", line 708, in __init__
m = Module(import_module(fullname),
File "C:\Users\Owen\Documents\my_package\env\lib\site-packages\pdoc\__init__.py", line 708, in __init__
m = Module(import_module(fullname),
[Previous line repeated 1 more time]
File "C:\Users\Owen\Documents\my_package\env\lib\site-packages\pdoc\__init__.py", line 646, in __init__
obj = inspect.unwrap(obj)
UnboundLocalError: local variable 'obj' referenced before assignment
The referenced installed package __init__.py file for dateutil is as follows:
# -*- coding: utf-8 -*-
try:
from ._version import version as __version__
except ImportError:
__version__ = 'unknown'
__all__ = ['easter', 'parser', 'relativedelta', 'rrule', 'tz',
'utils', 'zoneinfo']
I’ve confirmed that the relevant virtual environment has been activated. I’ve also confirmed that all modules in the dateutil package are where I expect them to be, like so:
dateutil/
__init__.py
easter.py
parser.py
relativedelta.py
rrule.py
tz/
...
utils.py
zoneinfo/
...
...
Why is the attempt to document the dateutil dependency failing this way? If documentation of dependencies is not supported, how should I skip them?
Additional info
- pdoc version: 0.9.2
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with the documented main.py and tl_util.py example in a virtual environment containing pdoc 0.9.2 and dateutil. Read pdoc/init.py around Module.init and the dateutil init.py export list to determine why dependency traversal reaches the unbound variable. Done means documenting the project no longer crashes, with dependency-skipping behavior defined or documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100