pdoc3 / pdoc3/pdoc

Generate docs of project fails with virtual environment dependencies

Open
#299 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
1.2k
Forks
146
PR merge metrics
No merged PRs in 30d

Description

Adapted from the pdocs programmed recursive documentation generation example, I have a script that runs the following methods:

tl_util.py
import pdoc

# documentation

def pdoc_module_path(m:pdoc.Module, ext:str='.html', dir_path:str=DOCS_PATH):
    return os.path.join(
        dir_path, 
        *regex.sub(r'\.html$', ext, m.url()).split('/'))
# end pdoc_module_path

def document_modules(mod:pdoc.Module) -> Generator[Tuple[pdoc.Module,str],None,None]:
    """Generate documentation for pdoc-wrapped module and submodules.
    
    Args:
        mod = pdoc.Module instance
    
    Yields tuple:
        module
        
        cleaned module html
    """
    
    yield (
        mod, 
        mod.html().replace('\u2011','-').replace(u'\xa0', u' ')
    )
    
    for submod in mod.submodules():
        yield from document_modules(submod)
# end document_modules
main.py
from typing import *
import pdoc
import os
import logging
from logging import Logger
import tl_util

log:Logger = logging.getLogger(__name__)

# omitted code

def document(dir_path:str=DOCS_PATH):
    """Recursively generate documentation using pdoc.
    
    Adapted from 
    [pdoc documentation](https://pdoc3.github.io/pdoc/doc/pdoc/#programmatic-usage).
                         
    Args:
        dir_path = documentation output directory; default=`algo_trader.DOCS_PATH`
    """
    
    ctx = pdoc.Context()
    
    modules:List[pdoc.Module] = [
        pdoc.Module(mod)
        for mod in [
            '.' # this script resides within the package that I want to create docs for
        ]
    ]
    
    pdoc.link_inheritance(ctx)
    
    for mod in modules:
        for submod, html in tl_util.document_modules(mod):
            # write to output location
            ext:str = '.html'
            filepath = tl_util.pdoc_module_path(submod, ext, dir_path)
            
            dirpath = os.path.dirname(filepath)
            if not os.access(dirpath, os.R_OK):
                os.makedirs(dirpath)
            
            with open(filepath,'w') as f:
                if ext == '.html':
                    try:
                        f.write(html)
                    except:
                        log.error(traceback.format_exc())
                elif ext == '.md':
                    f.write(mod.text())
            # close f
            
            log.info('generated doc for {} at {}'.format(
                submod.name,
                filepath))
        # end for module_name, html
    # end for mod in modules
# end document

if __name__ == '__main__':
    # omitted logic...
    document()

My project filesystem is like this:

my_package/
    env/
        Lib/
            site-packages/
                <installed dependencies, including pdoc3>
    main.py
    tl_util.py

Below is the error that I currently get:

(env) PS C:\<path>\my_package python .\main.py --document
=== Program Name ===

set logger <RootLogger root (DEBUG)> to level 10
C:\<path>\my_package\env\lib\site-packages\pdoc\__init__.py:643: UserWarning: Module <Module 'my_package
.env.Lib.site-packages.dateutil'> doesn't contain identifier `easter` exported in `__all__`
  warn("Module {!r} doesn't contain identifier `{}` "
Traceback (most recent call last):
  File “.\main.py", line 608, in <module>
    main()
  File “.\main.py", line 469, in main
    document()
  File “.\main.py", line 399, in document
    modules:List[pdoc.Module] = [
  File “.\main.py", line 400, in <listcomp>
    pdoc.Module(mod)
  File "C:\Users\Owen\Documents\my_package\env\lib\site-packages\pdoc\__init__.py", line 708, in __init__
    m = Module(import_module(fullname),
  File "C:\Users\Owen\Documents\my_package\env\lib\site-packages\pdoc\__init__.py", line 708, in __init__
    m = Module(import_module(fullname),
  File "C:\Users\Owen\Documents\my_package\env\lib\site-packages\pdoc\__init__.py", line 708, in __init__
    m = Module(import_module(fullname),
  [Previous line repeated 1 more time]
  File "C:\Users\Owen\Documents\my_package\env\lib\site-packages\pdoc\__init__.py", line 646, in __init__
    obj = inspect.unwrap(obj)
UnboundLocalError: local variable 'obj' referenced before assignment

The referenced installed package __init__.py file for dateutil is as follows:

# -*- coding: utf-8 -*-
try:
    from ._version import version as __version__
except ImportError:
    __version__ = 'unknown'

__all__ = ['easter', 'parser', 'relativedelta', 'rrule', 'tz',
           'utils', 'zoneinfo']

I’ve confirmed that the relevant virtual environment has been activated. I’ve also confirmed that all modules in the dateutil package are where I expect them to be, like so:

dateutil/
    __init__.py
    easter.py
    parser.py
    relativedelta.py
    rrule.py
    tz/
        ...
    utils.py
    zoneinfo/
        ...
    ...

Why is the attempt to document the dateutil dependency failing this way? If documentation of dependencies is not supported, how should I skip them?

Additional info
  • pdoc version: 0.9.2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with the documented main.py and tl_util.py example in a virtual environment containing pdoc 0.9.2 and dateutil. Read pdoc/init.py around Module.init and the dateutil init.py export list to determine why dependency traversal reaches the unbound variable. Done means documenting the project no longer crashes, with dependency-skipping behavior defined or documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.