* Make most of blib2to3 directly typed and mypyc-compatible
This used a combination of retype and pytype's merge-pyi to do the
initial merges of the stubs, which then required manual tweaking to
make actually typecheck and work with mypyc.
Co-authored-by: Sanjit Kalapatapu <sanjitkal@gmail.com>
Co-authored-by: Michael J. Sullivan <sully@msully.net>
* Make black able to compile and run with mypyc
The changes made fall into a couple categories:
* Fixing actual type mistakes that slip through the cracks
* Working around a couple mypy bugs (the most annoying of which being
that we need to add type annotations in a number of places where
variables are initialized to None)
Co-authored-by: Sanjit Kalapatapu <sanjitkal@gmail.com>
Co-authored-by: Michael J. Sullivan <sully@msully.net>
* Blacken .py files in blib2to3
This is in preparation for adding type annotations to blib2to3 in
order to compiling it with mypyc (#1009, which I can rebase on top of
this).
To enforce that it stays blackened, I just cargo-culted the existing
test code used for validating formatting. It feels pretty clunky now,
though, so I can abstract the common logic out into a helper if that
seems better. (But error messages might be less clear then?)
* Tidy up the tests
Based on the feedback in
https://github.com/python/black/pull/845#issuecomment-490622711
- Remove TokenizerConfig, and add a field to Grammar instead.
- Pass the Grammar to the tokenizer.
- Rename `ASYNC_IS_RESERVED_KEYWORD` to `ASYNC_KEYWORDS` and
`ASYNC_IS_VALID_IDENTIFIER` to `ASYNC_IDENTIFIERS`.
Fixes#593
I looked into this bug with @ambv and @carljm, and we reached the
conclusion was that it's not possible for the tokenizer to determine if
async/await is a keyword inside all possible generators without breaking
the grammar for older versions of Python.
Instead, we introduce a new tokenizer mode for Python 3.7+ that will
cause all async/await instances to get parsed as a reserved keyword,
which should fix async/await inside generators.
... to stop it from spamming the log when black is used as a library in another
python application.
When used indirectly by black the logger initiated in `driver.py` will emit
thousands of debug messages making the debug level of the root logger virtually
useless. By getting a named logger instead the verbosity of logging from this
module can easily be controlled by setting its log level.
Fixes#715
