... in the middle of an expression or code block by adding a missing return.
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
When the Leaf node with `# fmt: skip` is a NEWLINE inside a `suite`
Node, the nodes to ignore should be from the siblings of the parent
`suite` Node.
There is a also a special case for the ASYNC token, where it expands
to the grandparent Node where the ASYNC token is.
This fixes GH-2646, GH-3126, GH-2680, GH-2421, GH-2339, and GH-2138.
The former was a regression I introduced a long time ago. To avoid
changing the stable style too much, the regression is only fixed if
--preview is enabled
Annoyingly enough, as we currently always enforce a second format pass if
changes were made, there's no good way to prove the existence of the
docstring quote normalization instability issue. For posterity, here's
one failing example:
--- source
+++ first pass
@@ -1,7 +1,7 @@
def some_function(self):
- ''''<text here>
+ """ '<text here>
<text here, since without another non-empty line black is stable>
- '''
+ """
pass
--- first pass
+++ second pass
@@ -1,7 +1,7 @@
def some_function(self):
- """ '<text here>
+ """'<text here>
<text here, since without another non-empty line black is stable>
"""
pass
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Doing so is invalid. Note this only fixes the preview style since the
logic putting closing docstring quotes on their own line if they violate
the line length limit is quite new.
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Otherwise they'd be deleted which was a regression in 22.1.0 (oops! my
bad!). Also type comments are now tracked in the AST safety check on all
compatible platforms to error out if this happens again.
Overall the line rewriting code has been rewritten to do "the right
thing (tm)", I hope this fixes other potential bugs in the code (fwiw I
got to drop the bugfix in blib2to3.pytree.Leaf.clone since now bracket
metadata is properly copied over).
Fixes#2873
* Add run_self environment in tox
* Add run_self task as part of the lint CI flow
* Remove hard coded sources list
* Remove black from pre-commit
Co-authored-by: Cooper Lees <me@cooperlees.com>
This is a tricky one as await is technically an expression and therefore
in certain situations requires brackets for operator precedence.
However, the vast majority of await usage is just await some_coroutine(...)
and similar in format to return statements. Therefore this PR removes
redundant parens around these await expressions.
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
Allows us to better control placement of return annotations by:
a) removing redundant parens
b) moves very long type annotations onto their own line
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
The 8.0.x series renamed its "die on LANG=C" function and the 8.1.x
series straight up deleted it.
Unfortunately this makes this test type check cleanly hard, so we'll
just lint with click 8.1+ (the pre-commit hook configuration was changed
mostly to just evict any now unsupported mypy environments)
aiohttp.test_utils.unittest_run_loop was deprecated since aiohttp 3.8
and aiohttp 4 (which isn't a thing quite yet) removes it. To maintain
compatibility with the full range of versions we declare to support,
test_blackd.py will now define a no-op replacement if it can't be
imported.
Also, mypy is painfully slow to use without a cache, let's reenable it.
It was causing stability issues because the first pass
could cause a "magic trailing comma" to appear, meaning
that the second pass might get a different result. It's
not critical.
Some things format differently (with extra parens)
Fixes#2651. Fixes#2754. Fixes#2518. Fixes#2321.
This adds a test that lists a number of cases of unstable formatting
that we have seen in the issue tracker. Checking it in will ensure
that we don't regress on these cases.
Since power operators almost always have the highest binding power in expressions, it's often more readable to hug it with its operands. The main exception to this is when its operands are non-trivial in which case the power operator will not hug, the rule for this is the following:
> For power ops, an operand is considered "simple" if it's only a NAME, numeric CONSTANT, or attribute access (chained attribute access is allowed), with or without a preceding unary operator.
Fixes GH-538.
Closes GH-2095.
diff-shades results: https://gist.github.com/ichard26/ca6c6ad4bd1de5152d95418c8645354b
Co-authored-by: Diego <dpalma@evernote.com>
Co-authored-by: Felix Hildén <felix.hilden@gmail.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Closes#2360: I'd like to make passing SRC or `--code` mandatory and the arguments mutually exclusive. This will change our (partially already broken) promises of CLI behavior, but I'll comment below.
Fixes#2506
``XDG_CACHE_HOME`` does not work on Windows. To allow for users to set a custom cache directory on all systems I added a new environment variable ``BLACK_CACHE_DIR`` to set the cache directory. The default remains the same so users will only notice a change if that environment variable is set.
The specific use case I have for this is I need to run black on in different processes at the same time. There is a race condition with the cache pickle file that made this rather difficult. A custom cache directory will remove the race condition.
I created ``get_cache_dir`` function in order to test the logic. This is only used to set the ``CACHE_DIR`` constant.
Fixes#2742.
This PR adds the ability to configure additional python cell magics. This
will allow formatting cells in Jupyter Notebooks that are using custom (python)
magics.
Black would now echo the location that it determined as the root path
for the project if `--verbose` is enabled by the user, according to
which it chooses the SRC paths, i.e. the absolute path of the project
is `{root}/{src}`.
Closes#1880
*blib2to3's support was left untouched because: 1) I don't want to touch
parsing machinery, and 2) it'll allow us to provide a more useful error
message if someone does try to format Python 2 code.
error: cannot format <string>: ('EOF in multi-line statement', (2, 0))
▲ before ▼ after
error: cannot format <string>: Cannot parse: 2:0: EOF in multi-line statement
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
* Treat functions/classes in blocks as if they're nested
One curveball is that we still want two preceding newlines before blocks
that are probably logically disconnected. In other words:
if condition:
def foo():
return "hi"
# <- aside: this is the goal of this commit
else:
def foo():
return "cya"
# <- the two newlines spacing here should stay
# since this probably isn't related
with open("db.json", encoding="utf-8") as f:
data = f.read()
Unfortunately that means we have to special case specific clause types
instead of just being able to just for a colon leaf. The hack used here
is to check whether we're adding preceding newlines for a standalone or
dependent clause. "Standalone" being a clause that doesn't need another
clause to be valid (eg. if) and vice versa.
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Fixes https://github.com/psf/black/issues/2627 , a non-Python cell magic such as `%%writeline` can legitimately contain "incorrect" indentation, however this causes `tokenize-rt` to return an error. To avoid this, `validate_cell` should early detect cell magics (just like it detects `TransformerManager` transformations).
Test added too, in the shape of a "badly indented" `%%writefile` within `test_non_python_magics`.
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Marco Edward Gorelli <marcogorelli@protonmail.com>
The implementation of the new backtracking logic depends heavily on deepcopying the current state of the parser before seeing one of the new keywords, which by default is an very expensive operations. On my system, formatting these 3 files takes 1.3 seconds.
```
$ touch tests/data/pattern_matching_*; time python -m black -tpy310 tests/data/pattern_matching_* 19ms
All done! ✨🍰✨
3 files left unchanged.
python -m black -tpy310 tests/data/pattern_matching_* 2,09s user 0,04s system 157% cpu 1,357 total
```
which can be optimized 3X if we integrate the existing copying logic (`clone`) to the deepcopy system;
```
$ touch tests/data/pattern_matching_*; time python -m black -tpy310 tests/data/pattern_matching_* 1ms
All done! ✨🍰✨
3 files left unchanged.
python -m black -tpy310 tests/data/pattern_matching_* 0,66s user 0,02s system 147% cpu 0,464 total
```
This still might have some potential, but that would be way trickier than this initial patch.
* Improve Python 2 only syntax detection
First of all this fixes a mistake I made in Python 2 deprecation PR
using token.* to check for print/exec statements. Turns out that
for nodes with a type value higher than 256 its numeric type isn't
guaranteed to be constant. Using syms.* instead fixes this.
Also add support for the following cases:
print "hello, world!"
exec "print('hello, world!')"
def set_position((x, y), value):
pass
try:
pass
except Exception, err:
pass
raise RuntimeError, "I feel like crashing today :p"
`wow_these_really_did_exist`
10L
* Add octal support, more test cases, and fixup long ints
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
* Prepare for Python 2 depreciation
- Use BlackRunner and .stdout in command line test
So the next commit won't break this test. This is in its own commit so
we can just revert the depreciation commit when dropping Python 2
support completely.
* Deprecate Python 2 formatting support
Existing test was actually running a full black-primer
run which could be slow. This goes from 8 seconds to
0.4 seconds on my machine.
Needed to move to top level scope to leverage the caplog
feature of pytest in order to test that the command line
was parsing the bogus arguments and dumping to stderr.
* fix: allow tests to be run from the tests/ directory
* fix: try fixing windows build with MarcoGorelli's suggestion
* Windows hotfix + better respect test's spirit
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
If the individual failures are verbose, it's useful to have
the summary at the end. Otherwise, it can be really difficult
to figure out which projects have an issue.
It currently prints both ASTs - this also
adds the line diff, making it much easier to visualize
the changes as well. Not too verbose since it's only a diff.
The main goals of this commit include:
* improving consistency on how strict the test suite is -- Jelle has
seen cases where a test did not fail to an incomplete test setup
even though it should've
* simplifying tests for both ease of creation and reading via
parametrization and helpers
* reorganizing the test suite by grouping more tests
* dropping test suite dependencies that aren't strictly necessary
The test suite could definitely do with more refactoring, but this is a
good first pass. Anyway it would've gotten too big to review effectively
if I did continue on this PR.
Commit history before squash merge:
* Drop parameterized dep and refactor format tests
Since the test suite is already using pytest-only features we can drop
the parameterized test dependency in favour of pytest's own offering.
I also added an utility function called assert_format that makes it
even easier to verify Black formats some code correctly. We already
have great tooling if the case is very simple in test_format.py but
any sort of complication makes it hard to use. Also if you're writing
a non-standard test case, you have to be careful to include all of
the steps so issues don't go undetected. assert_format aims to
1) improve consistency, 2) avoid wasted CPU cycles, and 3) avoid
logical errors that hide issues.
Finally, quite a few tests were either moved and/or simplified with
the new setup.
* Move file collection tests
* Add assert_collected_sources helper function
Testing source collection involves a lot of repetitive boilerplate,
something that black.files.get_sources's signature does not help with.
So to cut down on boilerplate like `report=black.Report()` I added
a convenience function to tests/test_black.py which wraps
black.get_sources. Its signature is designed to be much more lax to
make it much easier to use. Somehow this leads to cutting 100 lines!
Also IMO the test cases are much easier to read since it's more
declarative than really procedural now.
* Run isort on some test files
* Move cache tests
* Use pytest-style asserts & add parametrization
* Drop now unnecessary test dependencies
*pytest-cases might be interesting for further refactoring but I
haven't been able to wrap my head around it for the time being. We
can always revisit anyway.
* re-implement simple CORS middleware for blackd
* remove aiohttp-cors from setup.py
* Remove aiohttp-cors from Pipfile.lock
Co-authored-by: Richard Si <63936253+ichard26@users.noreply.github.com>
Implementation stolen from PR davidhalter/parso#162. Thanks parso!
I could add support for these newer syntactical constructs in the
target version detection logic, but until I get diff-shades up
and running I don't feel very comfortable adding the code.
This fixes a bug where a trailing comma would be added to a
parenthesized return annotation changing its type to a tuple.
Here's one case where this bug shows up:
```
def spam() -> (
this_is_a_long_type_annotation_which_should_NOT_get_a_trailing_comma
):
pass
```
The root problem was that the type annotation was treated as if it was
a parameter & import list (is_body=True to linegen::bracket_split_build_line)
where a trailing comma is usually fine. Now there's another check in the
aforementioned function to make sure the body it's operating on isn't
a return annotation before truly adding a trailing comma.
