Import error with w3lib.encoding (from requests-html)

Open
#657 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
javascript, python
Domain
compilers

Research direction

Reproduce the compilation with rqhtstest.py and the shown requests_html.py and w3lib/encoding.py imports, then inspect how Transcrypt resolves re.translate and generated files in target. Confirm the import-resolution behavior and make the failing example compile without reporting existing generated modules as missing.

Written by the indexing model from the issue text.

Description

IS: bug? IS: limitation

This may be related to #443 (?) but given how far the compilation process is progressing I suspect #443 has been resolved and can be closed?
I am using the requests-html package to scrape data from other pages (with JS content) to be inserted into a section of my page.
Test code is:

from requests_html import AsyncHTMLSession

asession = AsyncHTMLSession()

async def get_page1():
    r = await asession.get('https://python.org/')
    await r.html.arender()
    return r

async def get_page2():
    r = await asession.get('https://reddit.com/')
    await r.html.arender()
    return r

pages = asession.run(get_page1, get_page2)

and gives the following output/error:

(webdev) C:\Users\ustauss\Anaconda3Projects\ts_test>transcrypt -a -b -m -v rqhtstest.py

Transcrypt (TM) Python to JavaScript Small Sane Subset Transpiler Version 3.7.16
Copyright (C) Geatec Engineering. License: Apache 2.0

Parsing module: c:/users/ustauss/anaconda3/envs/webdev/lib/site-packages/transcrypt/modules/org/transcrypt/__runtime__.py
Generating code for module: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/org.transcrypt.__runtime__.js
Saving source map in: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/org.transcrypt.__runtime__.map
Saving target code in: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/org.transcrypt.__runtime__.js
Saving minified target code in: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/org.transcrypt.__runtime__.js
Saving multi-level sourcemap in: {}
Parsing module: C:/Users/ustauss/Anaconda3Projects/ts_test/rqhtstest.py
Generating code for module: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/rqhtstest.js
Parsing module: c:/users/ustauss/anaconda3/envs/webdev/lib/site-packages/requests_html.py
Generating code for module: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/requests_html.js
Parsing module: c:/users/ustauss/anaconda3/envs/webdev/lib/site-packages/w3lib/encoding.py
Generating code for module: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/w3lib.encoding.js
Parsing module: c:/users/ustauss/anaconda3/envs/webdev/lib/site-packages/transcrypt/modules/sys/__init__.py
Generating code for module: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/sys.js
Saving source map in: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/sys.map
Saving target code in: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/sys.js
Saving minified target code in: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/sys.js
Saving multi-level sourcemap in: {}
Parsing module: c:/users/ustauss/anaconda3/envs/webdev/lib/site-packages/transcrypt/modules/re/__init__.py
Generating code for module: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/re.js
Parsing module: c:/users/ustauss/anaconda3/envs/webdev/lib/site-packages/transcrypt/modules/re/translate.py
Generating code for module: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/re.translate.js
Saving source map in: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/re.translate.map
Saving target code in: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/re.translate.js
Saving minified target code in: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/re.translate.js
Saving multi-level sourcemap in: {}
Saving source map in: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/re.map
Saving target code in: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/re.js
Saving minified target code in: C:/Users/ustauss/Anaconda3Projects/ts_test/__target__/re.js
Saving multi-level sourcemap in: {}

Error while compiling (offending file last):
        File 'C:/Users/ustauss/Anaconda3Projects/ts_test/rqhtstest.py', line 1, at import of:
        File 'c:/users/ustauss/anaconda3/envs/webdev/lib/site-packages/requests_html.py', line 22, at import of:
        File 'w3lib.encoding', line 5, namely:

        Import error, can't find any of:
                C:/Users/ustauss/Anaconda3Projects/ts_test/re/translate/translate.py
                C:/Users/ustauss/Anaconda3Projects/ts_test/re/translate/translate.js
                c:/users/ustauss/anaconda3/envs/webdev/lib/site-packages/transcrypt/modules/re/translate/translate.py
                c:/users/ustauss/anaconda3/envs/webdev/lib/site-packages/transcrypt/modules/re/translate/translate.js
                C:/Users/ustauss/Anaconda3/envs/webdev/Scripts/transcrypt.exe/re/translate/translate.py
                C:/Users/ustauss/Anaconda3/envs/webdev/Scripts/transcrypt.exe/re/translate/translate.js
                c:/users/ustauss/anaconda3/envs/webdev/re/translate/translate.py
                c:/users/ustauss/anaconda3/envs/webdev/re/translate/translate.js
                c:/users/ustauss/anaconda3/envs/webdev/lib/site-packages/re/translate/translate.py
                c:/users/ustauss/anaconda3/envs/webdev/lib/site-packages/re/translate/translate.js
                C:/Users/ustauss/Anaconda3Projects/ts_test/re/translate.py
                C:/Users/ustauss/Anaconda3Projects/ts_test/re/translate.js
                C:/Users/ustauss/Anaconda3Projects/ts_test/codecs.py
                C:/Users/ustauss/Anaconda3Projects/ts_test/codecs.js
                c:/users/ustauss/anaconda3/envs/webdev/lib/site-packages/transcrypt/modules/codecs.py
                c:/users/ustauss/anaconda3/envs/webdev/lib/site-packages/transcrypt/modules/codecs.js
                C:/Users/ustauss/Anaconda3/envs/webdev/Scripts/transcrypt.exe/codecs.py
                C:/Users/ustauss/Anaconda3/envs/webdev/Scripts/transcrypt.exe/codecs.js
                c:/users/ustauss/anaconda3/envs/webdev/codecs.py
                c:/users/ustauss/anaconda3/envs/webdev/codecs.js
                c:/users/ustauss/anaconda3/envs/webdev/lib/site-packages/codecs.py
                c:/users/ustauss/anaconda3/envs/webdev/lib/site-packages/codecs.js

Aborted

The __target__ subdirectory contains the following files:

org.transcrypt.__runtime__.js
org.transcrypt.__runtime__.map
org.transcrypt.__runtime__.pretty.map
org.transcrypt.__runtime__.py
re.js
re.map
re.pretty.map
re.py
re.translate.js
re.translate.map
re.translate.pretty.map
re.translate.py
sys.js
sys.map
sys.pretty.map
sys.py

Is Transcypt forgetting somewhere that it put those files (except codecs.*) into __target__?
What is going on?
Happy to provide any of the files in __target__ if necessary.

Dominant language
Python
Stars
2.9k
Forks
218
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from TranscryptOrg/Transcrypt

All issues in TranscryptOrg/Transcrypt

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.