In #1040 I had convinced myself that the type ignore logic didn't
need anything like the ignored_ids from the type comment logic, but I
was wrong, and we do.
We hit these cases in practice a bunch.
The code introduced in #1027 to detect whether a type comment appeared
after a regular comment in a Line would spuriously misfire when a leaf
was in the comments dict but had an empty list of comments. This can
occur as an artifact of how comments on trailing commas are handled,
it seems.
(This was discovered trying to test black out on mypy.)
Type comments only apply if they are the first comment on the line,
which means that allowing them to be pushed behind a regular comment
when joining lines is a semantic change (and, indeed, one that black
catches and fails on).
* Parse `:=` properly
* never unwrap parenthesis around `:=`
* When checking for AST-equivalence, use `ast` instead of `typed-ast` when running on python >=3.8
* Assume code that uses `:=` is at least 3.8
Modified maybe_remove_trailing_comma to remove trailing commas for
typedarglists (in addition to arglists), and updated line split logic
to ensure that all lines in a function definition that contain only one
arg have a trailing comma.
Based on the feedback in
https://github.com/python/black/pull/845#issuecomment-490622711
- Remove TokenizerConfig, and add a field to Grammar instead.
- Pass the Grammar to the tokenizer.
- Rename `ASYNC_IS_RESERVED_KEYWORD` to `ASYNC_KEYWORDS` and
`ASYNC_IS_VALID_IDENTIFIER` to `ASYNC_IDENTIFIERS`.
Fixes#593
I looked into this bug with @ambv and @carljm, and we reached the
conclusion was that it's not possible for the tokenizer to determine if
async/await is a keyword inside all possible generators without breaking
the grammar for older versions of Python.
Instead, we introduce a new tokenizer mode for Python 3.7+ that will
cause all async/await instances to get parsed as a reserved keyword,
which should fix async/await inside generators.
Closes#494
Task completion should also remove the task from `pending`.
Only replicates on some platforms. (eg. Can replicate on Python 3.7+, with either Windows or whatever default Linux distro Travis uses.)
