python-trio / python-trio/unasync

0.5.0 a couple issues on first attempt to use

Open
#78 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
108
Forks
16
PR merge metrics
No merged PRs in 30d

Description

  1. trying to use this on Windows is producing bad EOL in output file. This is because the file is opened binary on read and pulls in '\r\n' but then the file is opened as text on output so ends up writing '\r\r\n' to the output file

  2. need to support older versions of python but the following code:
    from typing_extensions import AsyncIterator
    gets transformed to the invalid:
    from typing_extensions import Iterator

  3. unasync output files are invalid if they use tabbed indentation. Source tabs are getting replaced with a single space on some but not all lines. Actual error trying to run a converted file:

  File "C:\cvs\gravitas_v1\incpy\esl_sync.py", line 48
    if value is None:
                     ^
IndentationError: unindent does not match any outer indentation level
  1. a simple way to run this by hand (setup.py is overkill for my needs) to convert a file to another file in the same directory. In my case I want to convert 'esl_async.py' to 'esl_sync.py' in the same folder. This is what I did in order to get this far:
# stdlib imports:
import os
from pathlib import Path
import sys

# 3rd-party imports:
import unasync # pip install unasync
from unasync import std_tokenize, _tokenize, _makedirs_existok

def _untokenize(tokens):
	return "".join(
		space + '\n' if tokval == '\r\n' else space + tokval
		for space, tokval in tokens
	)

class Rule( unasync.Rule ):
	def __init__( self, fromdir, todir, suffix ) -> None:
		self.suffix = suffix
		super().__init__( fromdir, todir )
	
	def _outfilepath( self, filepath ):
		path = Path( self.todir ).absolute() / f'{Path(filepath).stem}{self.suffix}.py'
		#assert False, f'{filepath=} {path=} {os.path.dirname(str(path))=}'
		return str( path )
	
	def _unasync_file( self, filepath ):
		with open(filepath, "rb") as f:
			write_kwargs = {}
			if sys.version_info[0] >= 3:
				encoding, _ = std_tokenize.detect_encoding(f.readline)
				write_kwargs["encoding"] = encoding
				f.seek(0)
			tokens = _tokenize(f)
			tokens = self._unasync_tokens(tokens)
			result = _untokenize(tokens)
			outfilepath = self._outfilepath( filepath )
			_makedirs_existok(os.path.dirname(outfilepath))
			with open(outfilepath, "w", **write_kwargs) as f:
				print(result, file=f, end="")

rule = Rule( '.', '.', '_sync' )
rule._unasync_file( 'esl.py' )

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the Windows conversion using esl_async.py and esl_sync.py, then inspect the tokenization and output path used by the unasync Rule example. Check the handling of line endings, typing_extensions imports, and tab indentation; done means converted files are valid on Windows and older Python versions, with a simple same-directory invocation supported.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.